This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Postprocess an image or batch of images tensor. Postprocess is the reverse process of preprocess. The parameters should be same as in

Rotates half the hidden dims of the input.

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Preprocess an image or batch of images. Copy of the `preprocess` method from `CLIPImageProcessor`. Args: images (`ImageI

Validate the `rope_scaling` configuration.

Rescales the image so that the following conditions are met: 1. Both dimensions (height and width) are divisible by 'factor'. 2. The total n

Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text` and `kwargs` arguments

Emu3RMSNorm is equivalent to T5LayerNorm

Converts an id to a token, special tokens included

Converts a token to an id using the vocab, special tokens included

Converts a sequence of tokens in a single string.