Name	Size	Mode	Actions
adadelta.cpython-36.pyc	4829	0644	editdlrm
adagrad.cpython-36.pyc	4767	0644	editdlrm
adam.cpython-36.pyc	6250	0644	editdlrm
adamax.cpython-36.pyc	4869	0644	editdlrm
adamw.cpython-36.pyc	6258	0644	editdlrm
asgd.cpython-36.pyc	2848	0644	editdlrm
lbfgs.cpython-36.pyc	8802	0644	editdlrm
lr_scheduler.cpython-36.pyc	63792	0644	editdlrm
nadam.cpython-36.pyc	5727	0644	editdlrm
optimizer.cpython-36.pyc	11827	0644	editdlrm
radam.cpython-36.pyc	5712	0644	editdlrm
rmsprop.cpython-36.pyc	6543	0644	editdlrm
rprop.cpython-36.pyc	5182	0644	editdlrm
sgd.cpython-36.pyc	6267	0644	editdlrm
sparse_adam.cpython-36.pyc	3329	0644	editdlrm
swa_utils.cpython-36.pyc	11370	0644	editdlrm
_functional.cpython-36.pyc	10407	0644	editdlrm
__init__.cpython-36.pyc	1049	0644	editdlrm

3 �Eg1 �@s<dd
lZdd
l
Z
ddlmZ
ddlmZ
Gdd�de�Zd
S)�N�
)
�_functional)
� Optimizercs4eZ
dZd
Zd�f
dd� Zej�d d d�
�
Z�ZS)�ASGDa�Implements Averaged Stochastic Gradient Descent. It has been proposed in `Acceleration of stochastic approximation by averaging`_. Args: params (iterable): iterable of parameters to optimize or dicts defining parameter groups lr (float, optional): learning rate (default: 1e-2) lambd (float, optional): decay term (default: 1e-4) alpha (float, optional): power for eta update (default: 0.75) t0 (float, optional): point at which to start averaging (default: 1e6) weight_decay (float, optional): weight decay (L2 penalty) (default: 0) .. _Acceleration of stochastic approximation by averaging: https://dl.acm.org/citation.cfm?id=131098 �{�G�z�?�-C��6?��?���.Ar
csTd
|k
stdj
|�
�
�
d
|k
s,tdj
|�
�
�
t|||||d�}tt|�j|
|�
dS)NgzInvalid learning rate: {}zInvalid weight_decay value: {})�lr�lambd�alpha�t0�weight_decay)� ValueError�format�dict�superr�__init__)�self�paramsr rrr r�defaults)
� __class__��
s