Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

This method forwards all its arguments to SiglipTokenizer's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this me

Forward pass for generation, returns logits

Support generation forward pass with cache

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

nested_split_lens: A list of N lists of ints. Each int indicates the length of a split within a sample, where each sample contains multi

Do not add eos again if user already added it.

Generate text from fully cached context All prompts and images are already in cache Args: confidence_thr

Ensure the value is divisible by the stride.

Convert multiple conversation formats to the internal elements list format. Supported formats: 1. [{'user': '...', '

Convert multiple conversation formats to internal elements list format. Supported formats: 1. [{'user': '...', 'assi

Create a logger that writes to a log file and stdout.

embed_dim: output dimension for each position pos: a list of positions to be encoded: size (M,) out: (M, D)

Rotates half the hidden dims of the input.

data_dir_list: list of data directories contains parquet files num_used_data: list of number of sampled data paths for each data dire

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Validate the `rope_scaling` configuration.