Set direction (fprop, bprop) based on PyTorch sequence markers. It is a heuristic and not a foolproof method.

Detect and set the class/module (mod) and operation (op) of the kernel e.g. torch.nn.functional / linear, torch / sigmoid. The lookup sequence

Set global scale.

Shows dataset statistics.

Returns a dict containing the current state of this :class:`FP16_Optimizer` instance. This dict contains attributes of :class:`FP16_O

If no closure is supplied, :attr:`step` should be called after ``fp16_optimizer_obj.backward(loss)``. :attr:`step` updates t

Strided check for overflow. You can get status by calling has_overflow.

Copy the ``.grad`` attribute from stored references to fp16 parameters to the ``.grad`` attribute of the fp32 master parameters that

Visualizes ranked results. Args: dismat (numpy.ndarray): distance matrix of shape (nq, ng) dataset (tupple): a 2-tuple including (

:class:`GRU`

:class:`LSTM`

:class:`ReLU`

:class:`Tanh`