Call this method to monkey patch optimizer step function so that masks can be applied to gradients and weights during training. You mu

Initializes GEMM state from arguments.

Load a model checkpoint and return the iteration. strict (bool): whether to strictly enforce that the keys in :attr:`state_dict` of the ch

Log a group of timers.

Utility function to print customMatmulPerf_t structure

Simple scaling.

Sets the predicate mask, overriding value stored in predicate iterator

Sets the predicate mask, overriding value stored in predicate iterator

Need to get the thread start row from the tile iterator

Returns whether access is valid or not

Adds a pointer offset in units of Element

Appends an entry

Compute the number of contiguous elements needed to store a tensor with the given size

Compute the number of contiguous elements needed to store a tensor with the given size

calculate min/max values from collector if :func:`MaxCollector <quantization.MaxCollector>` is used, kwargs should be None. if :func:`Hist

Perform dropout. Args: input_tensor: float Tensor. dropout_prob: Python float. The probability of dropping out a value (NOT of *keepi

Returns the biased exponent

Returns the biased exponent

Returns the biased exponent

Constructs a `BertConfig` from a json file of parameters.

A unified checkpoint name.

Tracker file rescords the latest chckpoint during training to restart from.

Get a yacs CfgNode object with default values.

Maps a CUTLASS tensor layout to a cuDNN cudnnTensorFormat_t

Build the model.

Determine the number of gemm_k iterations for conv2d problem using implicit gemm algorithm

Set global variables, initialize distributed, and set autoresume and random seeds. `allow_no_cuda` should not be set unless using megatron for

Determine if the source is needed. May return false if

Returns true if source is needed based on state of runtime arguments

Mapping functions (A,B,C -> activation,filter,output)