Wrap the data collator in a callable removing unused columns.

Determines the banned tokens for the current hypothesis based on previously generated n-grams. Args: banned_ngrams (`dict`):

Determines the banned tokens for the current hypothesis based on previously generated n-grams. Args: banned_ngrams (`dict`):

This class returns a [`LogitsProcessorList`] list object that contains all relevant [`LogitsWarper`] instances used for multinomial s

Prepares one `data` before feeding it to the model, be it a tensor or a nested list/dictionary of tensors.

Prepare `inputs` before feeding them to the model, converting them to tensors if they are not already and handling potential state.

This function extracts the model-specific `inputs` for generation.

Run generation in untraced mode. This should only be used for debugging purposes.

Run generation in untraced mode. This should only be used for debugging purposes.

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

A helper wrapper to group together context managers.

Instantiates a [`GenerationConfig`] from a [`PretrainedConfig`]. This function is useful to convert legacy [`PretrainedConfig`] objec

The next possible tokens that will progress the trie, given the current sequence of tokens in `current_seq`.

The next possible tokens that will progress the trie, given the current sequence of tokens in `current_seq`.

Helper to get number of tokens in a [`~torch.utils.data.DataLoader`] by enumerating dataloader.

Will save the model, so you can reload it using `from_pretrained()`. Will only save from the main process.

Updates the underlying post processor with the current `bos_token` and `eos_token`.

Updates the underlying post processor with the current `bos_token` and `eos_token`.

Converts the cache to the format expected by Bloom, i.e. to tuple(tuple([batch_size * num_heads, ...]))

Crops the past key values up to a certain maximum length.

This class returns a [`LogitsProcessorList`] list object that contains all relevant [`LogitsWarper`] instances used for multinomial s

This class returns a [`TFLogitsProcessorList`] list object that contains all relevant [`TFLogitsWarper`] instances used for multinomi

This class returns a [`LogitsProcessorList`] list object that contains all relevant [`LogitsWarper`] instances used for multinomial s

This class returns a [`TFLogitsProcessorList`] list object that contains all relevant [`TFLogitsWarper`] instances used for multinomi

Assume ngram_size=2 and prev_input_ids=tensor([[40, 2883, 2712, 4346]]). The output of generated ngrams look like this {(40,): [2883], (2883,

Assume ngram_size=2 and prev_input_ids=tensor([[40, 2883, 2712, 4346]]). The output of generated ngrams look like this {(40,): [2883], (2883,

Initializes input ids for generation, if necessary.

Initializes input ids for generation, if necessary.

Initializes input ids for generation, if necessary.

Initializes input ids for generation, if necessary.

Initializes input ids for generation, if necessary.

Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`,

This function extracts the model-specific `inputs` for generation.

r""" Generates sequences for models with a language modeling head using beam search. If `do_sample` is `False`, uses a greedy approach

r""" Generates sequences for models with a language modeling head using beam search. If `do_sample` is `False`, uses a greedy approach

Custom bias GELU function. Adapted from Megatron-DeepSpeed code. Here we use a simple implementation (inference) to make the model jitable.

How the loss is computed by Trainer. By default, all models return the loss in the first element. Subclass and override for custom b

Setup the optimizer and the learning rate scheduler. We provide a reasonable default that works well. If you want to use something e

Checks whether the passed dictionary and its nested dicts have a *torch_dtype* key and if it's not None, converts torch.dtype to a st

Checks whether the passed dictionary and its nested dicts have a *torch_dtype* key and if it's not None, converts torch.dtype to a st

Reads in a token and returns whether it creates progress.

Reads in a token and returns whether it creates progress.

Dropout add function Args: x (`torch.tensor`, *required*): input tensor residual (`torch.tensor`, *required*):