MCPcopy Create free account

hub / github.com/LLaVA-VL/LLaVA-NeXT / functions

Functions1,515 in github.com/LLaVA-VL/LLaVA-NeXT

⨍ Functions 1,515◇ Types & classes 291↳ Endpoints 23
↓ 2 callersMethod_save_checkpoint
(self)
llava-critic-r1/EasyR1/verl/trainer/ray_trainer.py:399
↓ 2 callersFunction_split_args_kwargs_data_proto
(chunks: int, *args, **kwargs)
llava-critic-r1/EasyR1/verl/single_controller/base/decorator.py:48
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
llava/train/train.py:328
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
llava/train/train_dpo.py:309
↓ 2 callersFunction_unpad_tensor
(x: Tensor, dim: int, padding_size: int)
llava-critic-r1/EasyR1/verl/utils/ulysses.py:112
↓ 2 callersFunction_unwrap_ray_remote
(cls)
llava-critic-r1/EasyR1/verl/single_controller/ray/base.py:447
↓ 2 callersMethodadd_token_per_grid
(self, image_feature)
llava/model/llava_arch.py:222
↓ 2 callersFunctionall_gather_data_proto
(data: DataProto, size: int, group: ProcessGroup)
llava-critic-r1/EasyR1/verl/protocol.py:680
↓ 2 callersFunctionall_to_all_tensor
( local_input: Tensor, scatter_dim: int, gather_dim: int, group: Optional[dist.ProcessGroup] =
llava-critic-r1/EasyR1/verl/utils/ulysses.py:134
↓ 2 callersFunctionapply_rotary_pos_emb_vision
(tensor: torch.Tensor, freqs: torch.Tensor)
llava/model/multimodal_encoder/mlcd/vit_rope2d_hf.py:19
↓ 2 callersFunctionauto_upgrade
(config)
llava/model/utils.py:4
↓ 2 callersFunctionbasic_clean
(text)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/tokenizer.py:62
↓ 2 callersFunctionbatch_collate
(features: List[Dict[str, Any]])
llava-critic-r1/EasyR1/verl/protocol.py:113
↓ 2 callersFunctionbroadcat
(tensors, dim=-1)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/rope.py:8
↓ 2 callersFunctionbuild_model_from_openai_state_dict
( state_dict: dict, quick_gelu=True, cast_dtype=torch.float16, )
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/model.py:370
↓ 2 callersFunctionbuild_prompt_chatbot
(problems, shot_qids, prompt_format, use_caption=False, options=["A", "B", "C", "D", "E"], is_test=False)
scripts/archived/convert_sqa_to_llava_base_prompt.py:220
↓ 2 callersMethodbuild_tokenized_answer
Llama tokenizer does satisfy `enc(a + b) = enc(a) + enc(b)`. It does ensure `enc(a + b) = enc(a) + enc(a + b)[len(enc(a)):]`.
trl/trainer/dpo_trainer.py:455
↓ 2 callersFunctionbuild_vision_projector
(config, delay_load=False, **kwargs)
llava/model/multimodal_projector/builder.py:32
↓ 2 callersFunctionbuild_vision_resampler
(model_args, delay_load=False, **kwargs)
llava/model/multimodal_resampler/builder.py:21
↓ 2 callersFunctionbuild_vision_tower
(vision_tower_cfg, **kwargs)
llava/model/multimodal_encoder/builder.py:13
↓ 2 callersFunctionbytes_to_unicode
Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/tokenizer.py:28
↓ 2 callersFunctioncheck_missing_images
(json_path, images_folder)
scripts/archived/quick_check.py:8
↓ 2 callersMethodcomplete
Mark the history as completed.
trl/environment/base_environment.py:116
↓ 2 callersMethodcompute_log_prob
Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in
llava-critic-r1/EasyR1/verl/workers/actor/base.py:40
↓ 2 callersMethodcompute_loss
( self, model: Union[PreTrainedModel, nn.Module], inputs: Dict[str, Union[torch.Tensor
trl/trainer/dpo_trainer.py:1005
↓ 2 callersMethodcompute_reference_log_probs
Computes log probabilities of the reference model for a single padded batch of a DPO specific dataset.
trl/trainer/dpo_trainer.py:638
↓ 2 callersMethodcompute_rewards
Compute per token rewards from scores and KL-penalty. Args: scores (`torch.FloatTensor`): Scores from th
trl/trainer/ppo_trainer.py:1031
↓ 2 callersFunctionconversations_formatting_function
r""" return a callable function that takes in a "messages" dataset and returns a formatted dataset, based on the tokenizer apply chat template
trl/extras/dataset_formatting.py:16
↓ 2 callersFunctioncreate_model
( model_name: str, pretrained: Optional[str] = None, precision: str = "fp32", device: Union[st
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/factory.py:311
↓ 2 callersFunctioncreate_one_example
(format, question, context, choice, answer, lecture, solution, test_example=True)
scripts/archived/convert_sqa_to_llava_base_prompt.py:106
↓ 2 callersFunctioncreate_one_example_gpt4
(format, question, context, choice, answer, lecture, solution, test_example=True)
scripts/archived/convert_sqa_to_llava_base_prompt.py:161
↓ 2 callersFunctioncreate_reference_model
Creates a static reference copy of a model. Note that model will be in `.eval()` mode. Args: model (`PreTrainedModelWrapper`): The m
trl/models/modeling_base.py:572
↓ 2 callersFunctiondisable_dropout_in_model
(model: torch.nn.Module)
trl/trainer/utils.py:546
↓ 2 callersFunctiondownload_pretrained_from_hf
( model_id: str, filename: str = "open_clip_pytorch_model.bin", revision=None, cache_dir: Unio
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/pretrained.py:275
↓ 2 callersFunctiondownload_pretrained_from_url
( url: str, cache_dir: Union[str, None] = None, )
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/pretrained.py:222
↓ 2 callersMethodencode_images
(self, images)
llava/model/llava_arch.py:192
↓ 2 callersMethodestimate_flops
Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis
llava-critic-r1/EasyR1/verl/utils/flops_counter.py:122
↓ 2 callersMethodexecute_all_async
(self, method_name: str, *args, **kwargs)
llava-critic-r1/EasyR1/verl/single_controller/ray/base.py:372
↓ 2 callersMethodexecute_rank_zero_async
(self, method_name: str, *args, **kwargs)
llava-critic-r1/EasyR1/verl/single_controller/ray/base.py:359
↓ 2 callersFunctionextract_patches
(image, patch_size, overlap_ratio)
llava/mm_utils.py:63
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
llava/model/multimodal_encoder/open_clip_encoder.py:63
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
llava/model/multimodal_encoder/hf_vision.py:45
↓ 2 callersMethodfeature_select
(self, image_features)
llava/model/multimodal_encoder/dev_eva_clip/eva_vit.py:78
↓ 2 callersFunctionflatten_dict
(data: Dict[str, Any], parent_key: str = "", sep: str = "/")
llava-critic-r1/EasyR1/verl/utils/py_functional.py:92
↓ 2 callersMethodforward
(self, x)
llava/model/multimodal_encoder/eva_clip/eva_vit.py:205
↓ 2 callersMethodforward_features
(self, x, return_all_features=False)
llava/model/multimodal_encoder/eva_clip/eva_vit.py:665
↓ 2 callersMethodforward_visual
(self, x, output_hidden_states=False)
llava/model/multimodal_encoder/open_clip_encoder.py:75
↓ 2 callersMethodfrom_single_dict
( cls, data: Dict[str, Union[torch.Tensor, NDArray]], meta_info: Optional[Dict[str, An
llava-critic-r1/EasyR1/verl/protocol.py:257
↓ 2 callersFunctiongather_outputs_and_unpad
( x: Tensor, gather_dim: int, unpad_dim: int = None, padding_size: int = 0, grad_scaler: b
llava-critic-r1/EasyR1/verl/utils/ulysses.py:241
↓ 2 callersMethodgenerate
Generate responses for a list of histories.
trl/environment/base_environment.py:362
↓ 2 callersMethodget_batch_loss_metrics
Compute the DPO loss and other metrics for the given batch of inputs for train or test. CHANGE: 1. add sft loss 2. all gather metrics
trl/trainer/dpo_trainer.py:902
↓ 2 callersFunctionget_cast_dtype
(precision: str)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/model.py:116
↓ 2 callersMethodget_cast_dtype
(self)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/transformer.py:473
↓ 2 callersFunctionget_conv_log_filename
()
llava/serve/gradio_multi_image.py:30
↓ 2 callersFunctionget_conv_log_filename
()
llava/serve/gradio_web_server.py:30
↓ 2 callersMethodget_extended_attention_mask
Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (
llava/model/multimodal_resampler/qformer.py:651
↓ 2 callersFunctionget_mm_adapter_state_maybe_zero_3
(named_params, keys_to_match)
llava/train/llava_trainer.py:45
↓ 2 callersFunctionget_model_list
()
llava/serve/gradio_multi_image.py:36
↓ 2 callersFunctionget_model_list
()
llava/serve/gradio_web_server.py:36
↓ 2 callersMethodget_num_gpus
Get the number of gpus in this cluster.
llava-critic-r1/EasyR1/verl/trainer/ray_trainer.py:102
↓ 2 callersFunctionget_pairs
Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/tokenizer.py:50
↓ 2 callersFunctionget_pretrained_tag
(pretrained_model)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/factory.py:158
↓ 2 callersFunctionget_pretrained_url
(model: str, tag: str)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/pretrained.py:217
↓ 2 callersMethodget_queue_length
(self)
llava/serve/model_worker.py:94
↓ 2 callersMethodget_queue_length
(self)
llava/serve/sglang_worker.py:113
↓ 2 callersFunctionget_seqlen_balanced_partitions
get order of seq lengths to make partitions balanced, this is used in balacing sum of seqlength across dp ranks and microbatches Parameter
llava-critic-r1/EasyR1/verl/utils/seqlen_balancing.py:150
↓ 2 callersMethodget_status
(self)
llava/serve/model_worker.py:100
↓ 2 callersMethodget_status
(self)
llava/serve/sglang_worker.py:119
↓ 2 callersFunctionget_tokenizer
Create a huggingface pretrained tokenizer.
llava-critic-r1/EasyR1/verl/utils/tokenizer.py:21
↓ 2 callersMethodget_worker_address
(self, model_name: str)
llava/serve/controller.py:117
↓ 2 callersFunctionhas_hf_hub
(necessary=False)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/pretrained.py:268
↓ 2 callersMethodis_image_file
(self, filename)
llava/conversation.py:244
↓ 2 callersFunctionis_rank0
()
llava-critic-r1/EasyR1/verl/utils/model_utils.py:27
↓ 2 callersMethodload_checkpoint
(self, path: str)
llava-critic-r1/EasyR1/verl/workers/fsdp_workers.py:431
↓ 2 callersFunctionload_clip_text_state_dict
(checkpoint_path: str, map_location: str = "cpu", is_openai: bool = False, skip_list: list = [])
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/factory.py:149
↓ 2 callersFunctionload_clip_visual_state_dict
(checkpoint_path: str, map_location: str = "cpu", is_openai: bool = False, skip_list: list = [])
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/factory.py:135
↓ 2 callersFunctionload_fsdp_optimizer
(optimizer: Optimizer, empty_cache: bool = True)
llava-critic-r1/EasyR1/verl/utils/fsdp_utils.py:128
↓ 2 callersFunctionload_image_from_base64
(image)
llava/mm_utils.py:296
↓ 2 callersFunctionmake_conv
(prompt, answer)
llava/train/train_dpo.py:475
↓ 2 callersFunctionmasked_whiten
Whiten values with masked values.
trl/core.py:171
↓ 2 callersFunctionpatch_device
(module)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/openai.py:97
↓ 2 callersFunctionpatch_float
(module)
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/openai.py:121
↓ 2 callersFunctionpreprocess
Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\
llava/train/train_dpo.py:849
↓ 2 callersFunctionpreprocess_multimodal
(sources: Sequence[str], data_args: DataArguments)
llava/train/train.py:381
↓ 2 callersFunctionpreprocess_qwen
(sources, tokenizer: transformers.PreTrainedTokenizer, has_image: bool = False, max_len=2048, system_message:
llava/eval/model_vqa.py:33
↓ 2 callersFunctionpretty_print_semaphore
(semaphore)
llava/utils.py:195
↓ 2 callersMethodprocess_image
(self, image, image_process_mode, return_pil=False, image_format="PNG")
llava/conversation.py:183
↓ 2 callersMethodprocess_image
(self, image_file)
llava/train/train_dpo.py:1003
↓ 2 callersMethodpush_to_hub
Overwrite the `push_to_hub` method in order to force-add the tag "sft" when pushing the model on the Hub. Please refer to `~transform
trl/trainer/dpo_trainer.py:1179
↓ 2 callersMethodrandom_masking
Perform per-sample random masking by per-sample shuffling. Per-sample shuffling is done by argsort random noise. x: [N, L, D]
llava/model/multimodal_resampler/masked_drop.py:56
↓ 2 callersMethodregister_to_controller
(self)
llava/serve/model_worker.py:69
↓ 2 callersMethodregister_to_controller
(self)
llava/serve/sglang_worker.py:88
↓ 2 callersMethodregister_worker
(self, worker_name: str, check_heart_beat: bool, worker_status: dict)
llava/serve/controller.py:69
↓ 2 callersMethodremove_worker
(self, worker_name: str)
llava/serve/controller.py:98
↓ 2 callersMethodreset_classifier
(self, num_classes, global_pool="")
llava/model/multimodal_encoder/dev_eva_clip/eva_clip/eva_vit_model.py:525
↓ 2 callersFunctionrotate_half
Rotates half the hidden dims of the input.
llava/model/language_model/modeling_llama.py:167
↓ 2 callersFunctionrotate_half
(x)
llava/model/multimodal_encoder/eva_clip/eva_vit.py:30
↓ 2 callersMethodsave_checkpoint
(self, path: str)
llava-critic-r1/EasyR1/verl/workers/fsdp_workers.py:420
↓ 2 callersMethodsave_model
Override the save_model method to also store the lightweight MeZO checkpoint.
llava/train/llava_trainer.py:448
↓ 2 callersMethodsave_pretrained
Saves all of the model weights
trl/models/modeling_sd_base.py:131
← previousnext →201–300 of 1,515, ranked by callers