Iterate over shards, yield shards of size group_shards size Each shard is a tuple (shard_id, shard)

Iterate over shards, yield shards of size number_sample_per_shard or less for the last one Each shard is a tuple (shard_id, shard)

Initialize the weights

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel

Take a list of samples (as dictionary) and create a batch, preserving the keys. If `tensors` is True, `ndarray` objects are combined into ten

* `x` is the Tensor at the head of a key or a query with shape `[seq_len, batch_size, n_heads, d]`

:param self: :param inputs_embeds: shape [B, 1 + n_image + n_token, C] :param img_length: length of image tokens, not include

Process: 1. image_embeds & text_tokens as input 2. prepend [IMG] token to img features or replace <ImagePatches> in <img><Ima

if k and v is None, self-attn, else cross-attn :param q: :param k: :param v: :param attn_mask: :retur

:param x: [B, n_token, C (context_dim)] as K and V :param attn_mask: 1, 0 :return: queries: [B, n_query, d_query (d_model)]

:param image_embeds: [B, n_query, C], after projected into Language shape :param text_input: :param text_output: :pa

Self-attention (if key_value_states is None) or attention over source sentence (provided by key_value_states).

r""" Returns: Example: ```python >>> from transformers import T5Tokenizer, T5Model >>> tokenizer = T5Tokeni

r""" labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression

r""" Returns: Example: ```python >>> from transformers import T5Tokenizer, T5EncoderModel >>> tokenizer = T

:return: forward_features function returns raw features of ViT, forward with return_all_features returns normalized f

For chat generation, inputs must be List[List[str | Image.Image]] Otherwise, inputs must be List[str | Image.Image]