r"""Move internal buffers of the ExponentialMovingAverage to `device`. Args: device: like `device` argument to `torch.Tensor.to`

Retrieves a single item from the dataset with the option to only return a portion of the item. get(idx) is the same as [idx] but get(

Save the current parameters for restoring later. Args: parameters: Iterable of `torch.nn.Parameter`; the parameters to be

Defines the operation to perform on a list of the `inst.resps` properties of `Instance` objects. Should return the list of (filtered)

r"""Move internal buffers of the ExponentialMovingAverage to `device`. Args: device: like `device` argument to `torch.Tensor.to`

Random feature map

Instantiate a GPTPreTrainedModel from a pre-trained model file or a pytorch state dict. Download and cache the pre-trained model file

Assume that x already has last dim divisible by alignment_in_bytes

Save the current parameters for restoring later. Args: parameters: Iterable of `torch.nn.Parameter`; the parameters to be

TD [2021-10-27] act_fn takes precedence over act_layer if set. This is to support Pytorch 1.10 Transformer interface that construct the activa

TD [2021-10-27] act_fn takes precedence over act_layer if set. This is to support Pytorch 1.10 Transformer interface that construct the activa

r"""Returns the state of the ExponentialMovingAverage as a dict.

Safely get Weights&Biases logger from Trainer.

Hook called at end of __init__, override this instead of __init__

r""" A streamlined generate() method overriding the transformers.GenerationMixin.generate() method. This method uses the same logits

Gets the reordered array Returns: List[Any]: The reordered array

r"""Returns the state of the ExponentialMovingAverage as a dict.

num_heads_kv: can be used to toggle MQA / GQA. If None, use num_heads. return_residual: whether to return the input x along with the

Initializes multi-GPU-friendly python logger.

Method for padding a list of tensors given the maximum tensor length in the batch. Used for batching inputs and continuations in seq2seq

:return: Iterable[obj] A iterable of any object, that doc_to_text can handle

:return: Iterable[obj] A iterable of any object, that doc_to_text can handle

If process_group is not None, we're doing Tensor Parallel with sequence parallelism: we do an all_gather of x before doing the matmul

Args: u (torch.Tensor): Float tensor of shape (batch_size, d_model, seq_len) k (torch.Tensor): Float tensor of shape (d_model, l_

Generates and yields batches from the reordered array. Parameters: - n (int): The size of each batch. Defaults to 1.

Restores the original order of elements from the reordered list. Parameters: - newarr (List): The reordered array.

Whether the task has a validation set

Imports a class or function from a string. Format: "module.submodule.ClassName" or "module.submodule.function_name". Args: pat

Compute max metric between prediction and each ground truth.

Set the logits for none top-p values to -inf. Done in-place.

Set the logits for none top-p values to -inf. Done in-place.

Normalization used in official SQuAD evaluation script.

Draw the first `n` samples in order from the specified split. Used for tasks with "canonical" ordered fewshot examples, such as MMLU

This method should set self.dataset_train, self.dataset_val, and self.dataset_test.

Initializes the LongConvolution module. Args: d_model (int): The number of expected features in the input and output.

Modify the behavior of the default _collate method.