MCPcopy Create free account
hub / github.com/LiteSSLHub/DisCo / BertAdam

Class BertAdam

src/pytorch_pretrained_bert/optimization.py:45–162  ·  view source on GitHub ↗

Implements BERT version of Adam algorithm with weight decay fix. Params: lr: learning rate warmup: portion of t_total for the warmup, -1 means no warmup. Default: -1 t_total: total number of training steps for the learning rate schedule, -1 means constant le

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected