Append a new message.

Applies the SFT template to conversation. An example of conversation: conversation = [ { "role":

Register a new conversation template.

Get a conversation template.

Args: input_ids (torch.LongTensor): [b, T] pixel_values (torch.FloatTensor): [b, n_images, 3, h, w] im

Args: pil_img (PIL.Image): [H, W, 3] in PIL.Image in RGB Returns: x (np.ndarray): [3, self.image_size, self

Set the system message.

r"""The original timm.models.layers.weight_init.trunc_normal_ can not handle bfloat16 yet, here we first convert the tensor to float32, apply the

r"""The original timm.models.layers.weight_init.trunc_normal_ can not handle bfloat16 yet, here we first convert the tensor to float32, apply the

Get the prompt for generation.

Args: image_indices (List[int]): [index_0, index_1, ..., index_j] input_ids (torch.LongTensor): [N] Returns

Args: image_indices (List[int]): [index_0, index_1, ..., index_j] input_ids (torch.LongTensor): [N] Returns

Applies the SFT template to conversation. An example of conversation: conversation = [ { "role":

Preprocesses the inputs for multimodal inference. Args: prepare_list (List[VLChatProcessorOutput]): A list of VLChatProc

Preprocesses the inputs for multimodal inference. Args: prepare_list (List[VLChatProcessorOutput]): A list of VLChatProc

Support file path or base64 images. Args: conversations (List[Dict[str, str]]): the conversations with a list of messages. An examp

Args: prompt (str): the formatted prompt; conversations (List[Dict]): conversations with a list of messages;

Args: prompt (str): the formatted prompt; conversations (List[Dict]): conversations with a list of messages;

Args: pil_img (PIL.Image): [H, W, 3] in PIL.Image in RGB Returns: x (np.ndarray): [3, self.image_size, self

Args: prompt (str): the formatted prompt; conversations (List[Dict]): conversations with a list of messages;

Args: prompt (str): the formatted prompt; conversations (List[Dict]): conversations with a list of messages;