Compare the received keys in `config.rope_scaling` against the expected and optional keys

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Computes the inverse frequencies according to the original RoPE implementation Args: config ([`~transformers.PretrainedConfig`]):

Inverse dimension formula to find the dimension based on the number of rotations

embed_dim: output dimension for each position pos: a list of positions to be encoded: size (M,) out: (M, D)

grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Rotates half the hidden dims of the input.

Given the video file and input prompt, run the model and return the response file: Video file path inp: Input prompt

Initialize the model

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, then compu

Retrieves indexing data required to repad unpadded (ragged) tensors. Arguments: attention_mask (`torch.Tensor`): Boolean

Unpads query, key, and values tensors, using a single dimension for all tokens even though they belong to different batches. This function i

Disable the redundant torch default initialization to accelerate model creation.

Evaluate your results against the ground truth Args: - your_results_path (str): Path to your results file - video_types (Optional[Li

Find dimension range bounds based on rotations

This function returns necessary arguments to call `flash_attn_varlen_func`. All three query, key, value states will be flattened. Cummula