Sort symbols by frequency in descending order, ignoring special ones. Args: - threshold defines the minimum word count

Whether the iterator has been exhausted.

Maximum input length supported by the encoder.

Args: cfg (omegaconf.DictConfig): fairseq args params (iterable): iterable of parameters to optimize

copy the refs.

Generate a batch of translations. Args: sample (dict): batch prefix_tokens (torch.LongTensor, optional): force decode

Helper to wrap layers/modules in FSDP. This falls back to a no-op if fairscale is not available. Args: module (nn.Module): modul

Evaluate language model based on the current lm state and new word Parameters: ----------- state: current lm state

Build the :class:`~fairseq.criterions.FairseqCriterion` instance for this task. Args: cfg (omegaconf.DictConfig)

Return a loaded dataset split. Args: split (str): name of the split (e.g., train, valid, test) Returns:

Copies parameters and buffers from *state_dict* into this module and its descendants. Overrides the method in :class:`nn.Module`. Com

Get the (non-wrapped) model instance.

Prepare model for inference.

Truncate the iterator to n elements at most.

Updates counts from new dictionary.

Weight-normalized Linear layer (input: B x T x C)

Weight-normalized Linear layer (input: N x T x C)

query: tgt_len, bsz, embed_dim key: src_len, bsz, embed_dim value: src_len, bsz, embed_dim

Maximum length supported by the model.

Pad Dictionary size to be a multiple of *padding_factor*.

Args: append_eos: if True, each input sentence in the source tokens tensor will have an EOS appended to the end.

This verifies that with a given x, x_len, max_shuffle_distance, and vocab, we get the expected shuffle result. Args:

A friendlier wrapper for performing activation checkpointing. Compared to the PyTorch version, this version: - wraps an nn.Module, so th

Get the (non-wrapped) model instance.

Get normalized probabilities (or log probs) from a net's output.

Load all training state from a checkpoint file. rank = 0 will load the checkpoint, and then broadcast it to all other ranks.

Load a given dataset split. Args: split (str): name of the split (e.g., train, valid, test)

Load the dictionary from the filename Args: filename (str): the filename

Print end-of-epoch stats.

To leverage the efficiency of distributed training, samples for all layers are computed at once for each logit_idx. Logits are parameters

Linear layer (input: N x T x C)