Calculates the learning rate at batch index. This function treats `self.last_epoch` as the last batch index. If `self.cycle_momentum`

Check if overflows were detected by any call to step(...) method. Clears the overflow flag.

Check if overflows were detected by any call to step(...) method. Clears the overflow flag.

init_hidden()

init_hidden()

init_inference()

Initialize your models, optimizers, and the Torch tensor and functional namespace according to the chosen ``opt_level`` and overridden proper

When running with :class:`FP16_Optimizer`, ``.grad`` attributes of a model's fp16 leaves should not be regarded as truthful,

Call this method to determine if sparsity is enabled in the model. The typical use case is right after checkpoint has been loaded.

r"""Loads pretrianed weights to model. Features:: - Incompatible layers (unmatched in name or size) will be ignored. - Can automa

Loads a state_dict created by an earlier call to state_dict(). If an DistributedFusedAdam instance was constructed from some ``init_o

Loads the schedulers state. Arguments: state_dict (dict): scheduler state. Should be an object returned from a ca

Loads the schedulers state. Arguments: state_dict (dict): scheduler state. Should be an object returned from a ca

self.log({'loss':1,'lr':0.0001})

Gathers the stats from self.trainer.stats and passes them into self.log, as a list

Compute knowledge-distillation (KD) loss given [scores] and [target_scores]. Both [scores] and [target_scores] should be tensors, although [t

:class:`mLSTM`

mLSTMCell