HP search setup code

Checks whether CP is the codepoint of a CJK character.

Checks whether CP is the codepoint of a CJK character.

If optimizer and scheduler states exist, load them.

Make causal mask used for bi-directional self-attention, supporting both static and dynamic shapes.

Make causal mask used for self-attention.

Make causal mask used for bi-directional self-attention, supporting both static and dynamic shapes.

Merge heads together over the last dimension Args: x (`torch.tensor`, *required*): [batch_size * num_heads, seq_length,

Gather value of `tensors` (tensor or list/tuple of nested tensors) and convert them to numpy before concatenating them to `gathered`

Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`,

Expands or crops the model's mask for decoding purposes, to the defined length

Expands or crops the model's mask for decoding purposes, to the defined length

Prepares `decoder_input_ids` for generation with encoder-decoder models

Prepares `decoder_input_ids` for generation with encoder-decoder models

Prepares `decoder_input_ids` for generation with encoder-decoder models

This function extracts the model-specific `inputs` for generation.

This function extracts the model-specific `inputs` for generation.

This function extracts the model-specific `inputs` for generation.

Expands or crops the model's token_type_ids for decoding purposes, to the defined length

Expands or crops the model's token_type_ids for decoding purposes, to the defined length

Reranks the top_k candidates based on a degeneration penalty (cosine similarity with previous tokens), as described in the paper "A Contrasti

Reranks the top_k candidates based on a degeneration penalty (cosine similarity with previous tokens), as described in the paper "A Contrasti

Reranks the top_k candidates based on a degeneration penalty (cosine similarity with previous tokens), as described in the paper "A Contrasti

Reranks the top_k candidates based on a degeneration penalty (cosine similarity with previous tokens), as described in the paper "A Contrasti

Reranks the top_k candidates based on a degeneration penalty (cosine similarity with previous tokens), as described in the paper "A Contrasti

Validate the `rope_scaling` configuration.

Generate greenlist based on current candidate next token. Reject and move on if necessary. Runs for a fixed number of steps only for

Generate greenlist based on current candidate next token. Reject and move on if necessary. Runs for a fixed number of steps only for

Split the last dimension into (num_heads, head_dim) without making any copies, results share same memory storage as `fused_qkv`

This function precomputes everything needed for the run-time checks in StopStringCriteria, and packs them into an embedding tensor that can be

This function precomputes everything needed for the run-time checks in StopStringCriteria, and packs them into an embedding tensor that can be

This function preprocesses stop strings and the tokenizer vocabulary to determine where tokens can validly appear in the stop strings. For eac

This function preprocesses stop strings and the tokenizer vocabulary to determine where tokens can validly appear in the stop strings. For eac

Performs validation related to the resulting generated length

Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use.

Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use.

Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use.

Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use.