MCPcopy Create free account

hub / github.com/apple/ml-fastvlm / functions

Functions294 in github.com/apple/ml-fastvlm

⨍ Functions 294◇ Types & classes 53↳ Endpoints 11
↓ 35 callersFunctiontokenizer_image_token
(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None)
llava/mm_utils.py:187
↓ 18 callersMethodcopy
(self)
llava/conversation.py:214
↓ 17 callersMethodappend_message
(self, role, message)
llava/conversation.py:143
↓ 14 callersMethodget_model
(self)
llava/model/llava_arch.py:135
↓ 13 callersMethodget_prompt
(self)
llava/conversation.py:34
↓ 12 callersMethod__init__
Build MHSA module that can handle 3D or 4D input tensors. Args: dim: Number of embedding dimensions. head_dim: Number
llava/model/multimodal_encoder/mobileclip/mci.py:633
↓ 12 callersMethodto_gradio_chatbot
(self)
llava/conversation.py:196
↓ 9 callersMethodget_vision_tower
(self)
llava/model/llava_arch.py:43
↓ 7 callersMethodget_vision_tower
(self)
llava/model/llava_arch.py:138
↓ 7 callersMethodprepare_inputs_labels_for_multimodal
( self, input_ids, position_ids, attention_mask, past_key_values, labels, images, image_sizes=
llava/model/llava_arch.py:146
↓ 5 callersMethodembed_tokens
(self, x)
llava/model/language_model/llava_mpt.py:36
↓ 4 callersFunctionbuild_logger
(logger_name, logger_filename)
llava/utils.py:17
↓ 4 callersFunctionexpand2square
(pil_img, background_color)
llava/mm_utils.py:154
↓ 4 callersMethodget_images
(self, return_pil=False)
llava/conversation.py:186
↓ 4 callersFunctionget_length_grouped_indices
(lengths, batch_size, world_size, generator=None, merge=True)
llava/train/llava_trainer.py:92
↓ 4 callersMethodload_model
(self, device_map=None)
llava/model/multimodal_encoder/clip_encoder.py:31
↓ 4 callersFunctionload_pretrained_model
(model_path, model_base, model_name, load_8bit=False, load_4bit=False, device_map="auto", device="cuda", use_f
llava/model/builder.py:26
↓ 3 callersFunction_cfg
(url="", **kwargs)
llava/model/multimodal_encoder/mobileclip/mci.py:20
↓ 3 callersMethod_fuse_bn_tensor
Method to fuse batchnorm layer with preceeding conv layer. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L95 Arg
llava/model/multimodal_encoder/mobileclip/mci.py:284
↓ 3 callersFunctiondisable_torch_init
Disable the redundant torch default initialization to accelerate model creation.
llava/utils.py:94
↓ 3 callersMethodfeature_select
(self, image_forward_outs)
llava/model/multimodal_encoder/clip_encoder.py:48
↓ 3 callersFunctionget_model_name_from_path
(model_path)
llava/mm_utils.py:209
↓ 3 callersMethodget_worker_status
(self, worker_name: str)
llava/serve/controller.py:88
↓ 3 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
llava/train/train.py:123
↓ 3 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
llava/train/train_qwen.py:123
↓ 3 callersFunctionprocess_anyres_image
Process an image with variable resolutions. Args: image (PIL.Image.Image): The input image to be processed. processor: The i
llava/mm_utils.py:121
↓ 3 callersFunctionprocess_images
(images, image_processor, model_cfg)
llava/mm_utils.py:168
↓ 3 callersFunctionvote_last_response
(state, vote_type, model_selector, request: gr.Request)
llava/serve/gradio_web_server.py:82
↓ 2 callersMethod_conv_bn
Helper method to construct conv-batchnorm layers. Args: kernel_size: Size of the convolution kernel. padding: Zero-pa
llava/model/multimodal_encoder/mobileclip/mci.py:332
↓ 2 callersMethod_conv_bn
Helper method to construct conv-batchnorm layers. Args: kernel_size: Size of the convolution kernel. padding: Zero-pa
llava/model/multimodal_encoder/mobileclip/mci.py:517
↓ 2 callersMethod_fuse_bn
Method to fuse batchnorm layer with conv layer. Args: conv: Convolutional kernel weights. bn: Batchnorm 2d layer.
llava/model/multimodal_encoder/mobileclip/mci.py:495
↓ 2 callersMethod_save
(self, output_dir: Optional[str] = None, state_dict=None)
llava/train/llava_trainer.py:267
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
llava/train/train.py:257
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
llava/train/train_qwen.py:257
↓ 2 callersFunctionauto_upgrade
(config)
llava/model/utils.py:4
↓ 2 callersFunctionbuild_vision_projector
(config, delay_load=False, **kwargs)
llava/model/multimodal_projector/builder.py:17
↓ 2 callersFunctionbuild_vision_tower
(vision_tower_cfg, **kwargs)
llava/model/multimodal_encoder/builder.py:6
↓ 2 callersMethoddict
(self)
llava/conversation.py:225
↓ 2 callersMethodencode_images
(self, images)
llava/model/llava_arch.py:141
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
llava/model/multimodal_encoder/mobileclip_encoder.py:60
↓ 2 callersMethodforward_images
(self, images)
llava/model/multimodal_encoder/mobileclip_encoder.py:77
↓ 2 callersMethodforward_images
(self, images)
llava/model/multimodal_encoder/clip_encoder.py:65
↓ 2 callersMethodgenerate
( self, inputs: Optional[torch.Tensor] = None, images: Optional[torch.Tensor] = None,
llava/model/language_model/llava_qwen.py:106
↓ 2 callersFunctionget_conv_log_filename
()
llava/serve/gradio_web_server.py:32
↓ 2 callersFunctionget_model_list
()
llava/serve/gradio_web_server.py:38
↓ 2 callersMethodget_queue_length
(self)
llava/serve/model_worker.py:108
↓ 2 callersMethodget_queue_length
(self)
llava/serve/sglang_worker.py:118
↓ 2 callersMethodget_sample
(self, i)
llava/train/train_qwen.py:929
↓ 2 callersMethodget_status
(self)
llava/serve/model_worker.py:115
↓ 2 callersMethodget_status
(self)
llava/serve/sglang_worker.py:125
↓ 2 callersMethodget_worker_address
(self, model_name: str)
llava/serve/controller.py:120
↓ 2 callersMethodinitialize_vision_modules
(self, model_args, fsdp=None)
llava/model/llava_arch.py:49
↓ 2 callersMethodinitialize_vision_tokenizer
(self, model_args, tokenizer)
llava/model/llava_arch.py:334
↓ 2 callersFunctionload_image_from_base64
(image)
llava/mm_utils.py:150
↓ 2 callersMethodload_model
(self, device_map=None)
llava/model/multimodal_encoder/mobileclip_encoder.py:31
↓ 2 callersFunctionpretty_print_semaphore
(semaphore)
llava/utils.py:124
↓ 2 callersMethodprocess_image
(self, image, image_process_mode, return_pil=False, image_format='PNG', max_len=1344, min_len=672)
llava/conversation.py:146
↓ 2 callersFunctionrank0_print
(*args)
llava/train/train.py:45
↓ 2 callersMethodregister_to_controller
(self)
llava/serve/model_worker.py:75
↓ 2 callersMethodregister_to_controller
(self)
llava/serve/sglang_worker.py:85
↓ 2 callersMethodregister_worker
(self, worker_name: str, check_heart_beat: bool, worker_status: dict)
llava/serve/controller.py:69
↓ 2 callersMethodremove_worker
(self, worker_name: str)
llava/serve/controller.py:101
↓ 2 callersMethodreparameterize
(self)
llava/model/multimodal_encoder/mobileclip/mci.py:1000
↓ 2 callersFunctionselect_best_resolution
Selects the best resolution from a list of possible resolutions based on the original size. Args: original_size (tuple): The origina
llava/mm_utils.py:14
↓ 2 callersMethodsend_heart_beat
(self)
llava/serve/model_worker.py:87
↓ 2 callersMethodsend_heart_beat
(self)
llava/serve/sglang_worker.py:97
↓ 2 callersFunctiontrain
(attn_implementation=None)
llava/train/train.py:909
↓ 2 callersFunctiontrain
(attn_implementation=None)
llava/train/train_qwen.py:1041
↓ 2 callersMethodwrite
(self, buf)
llava/utils.py:74
↓ 1 callersMethod__init__
(self, config: Qwen2Config)
llava/model/language_model/llava_qwen.py:37
↓ 1 callersMethod__init__
(self, config: LlamaConfig)
llava/model/language_model/llava_llama.py:37
↓ 1 callersMethod__init__
(self, config: MptConfig)
llava/model/language_model/llava_mpt.py:32
↓ 1 callersMethod__init__
(self, config: MistralConfig)
llava/model/language_model/llava_mistral.py:38
↓ 1 callersMethod__init__
(self, vision_tower, args, delay_load=False)
llava/model/multimodal_encoder/clip_encoder.py:8
↓ 1 callersFunction_add_speaker_and_signal
Add speaker and start/end signal on each round.
llava/train/train.py:295
↓ 1 callersFunction_add_speaker_and_signal
Add speaker and start/end signal on each round.
llava/train/train_qwen.py:295
↓ 1 callersMethod_get_in_feature_dimension
Return the input feature dimension to the image classification head.
llava/model/multimodal_encoder/mobileclip/__init__.py:61
↓ 1 callersMethod_get_kernel_bias
Method to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83 Returns:
llava/model/multimodal_encoder/mobileclip/mci.py:249
↓ 1 callersFunction_mask_targets
(target, tokenized_lens, speakers)
llava/train/train.py:284
↓ 1 callersFunction_mask_targets
(target, tokenized_lens, speakers)
llava/train/train_qwen.py:284
↓ 1 callersMethod_update_image_classifier
( image_classifier: nn.Module, projection_dim: int, *args, **kwargs )
llava/model/multimodal_encoder/mobileclip/__init__.py:82
↓ 1 callersFunctionapply_delta
(base_model_path, target_model_path, delta_path)
llava/model/apply_delta.py:13
↓ 1 callersFunctionbasic_blocks
Build FastViT blocks within a stage. Args: dim: Number of embedding dimensions. block_index: block index. num_blocks: Lis
llava/model/multimodal_encoder/mobileclip/mci.py:1195
↓ 1 callersFunctionbuild_demo
(embed_mode, cur_dir=None, concurrency_count=10)
llava/serve/gradio_web_server.py:317
↓ 1 callersMethodcall_for_batch
(self, output_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs)
llava/mm_utils.py:233
↓ 1 callersFunctionconsolidate_ckpt
(src_path, dst_path)
llava/model/consolidate.py:13
↓ 1 callersFunctionconvolutional_stem
Build convolutional stem with MobileOne blocks. Args: in_channels: Number of input channels. out_channels: Number of output chann
llava/model/multimodal_encoder/mobileclip/mci.py:553
↓ 1 callersMethoddevice
(self)
llava/model/multimodal_encoder/clip_encoder.py:87
↓ 1 callersFunctiondivide_to_patches
Divides an image into patches of a specified size. Args: image (PIL.Image.Image): The input image. patch_size (int): The siz
llava/mm_utils.py:79
↓ 1 callersFunctionexport
(args)
model_export/export_vision_encoder.py:19
↓ 1 callersFunctionfind_all_linear_names
(model)
llava/train/train.py:177
↓ 1 callersFunctionfind_all_linear_names
(model)
llava/train/train_qwen.py:177
↓ 1 callersMethodforward_embeddings
(self, x: torch.Tensor)
llava/model/multimodal_encoder/mobileclip/mci.py:1427
↓ 1 callersMethodforward_tokens
(self, x: torch.Tensor, *args, **kwargs)
llava/model/multimodal_encoder/mobileclip/mci.py:1431
↓ 1 callersMethodfrom_str
(cls, name)
llava/serve/controller.py:33
↓ 1 callersMethodgenerate_stream
(self, params)
llava/serve/model_worker.py:123
↓ 1 callersMethodgenerate_stream
(self, params)
llava/serve/sglang_worker.py:132
↓ 1 callersMethodgenerate_stream_gate
(self, params)
llava/serve/model_worker.py:195
↓ 1 callersMethodgenerate_stream_gate
(self, params)
llava/serve/sglang_worker.py:172
↓ 1 callersFunctionget_anyres_image_grid_shape
Calculate the shape of the image patch grid after the preprocessing for images of any resolution. Args: image_size (tuple): The size
llava/mm_utils.py:101
next →1–100 of 294, ranked by callers