_lr_lambda returns a multiplicative factor given an interger parameter epochs. Decaying criteria: last_epoch
(current_step)
| 232 | ''' |
| 233 | |
| 234 | def _lr_lambda(current_step): |
| 235 | ''' |
| 236 | _lr_lambda returns a multiplicative factor given an interger parameter epochs. |
| 237 | Decaying criteria: last_epoch |
| 238 | ''' |
| 239 | |
| 240 | if current_step < num_warmup_steps: |
| 241 | _lr = float(current_step) / float(max(1, num_warmup_steps)) |
| 242 | else: |
| 243 | num_cos_steps = float(current_step - num_warmup_steps) |
| 244 | num_cos_steps = num_cos_steps / float(max(1, num_training_steps - num_warmup_steps)) |
| 245 | _lr = max(0.0, math.cos(math.pi * num_cycles * num_cos_steps)) |
| 246 | return _lr |
| 247 | |
| 248 | return LambdaLR(optimizer, _lr_lambda, last_epoch) |
| 249 |
nothing calls this directly
no outgoing calls
no test coverage detected