MCPcopy Create free account

hub / github.com/PKU-YuanGroup/Video-LLaVA / functions

Functions670 in github.com/PKU-YuanGroup/Video-LLaVA

⨍ Functions 670◇ Types & classes 125↳ Endpoints 14
↓ 42 callersMethodfrom_pretrained
See `AutoTokenizer.from_pretrained` docstring.
videollava/model/language_model/mpt/adapt_tokenizer.py:37
↓ 28 callersMethodappend_message
(self, role, message)
videollava/conversation.py:106
↓ 26 callersFunctiontokenizer_image_token
(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None)
videollava/mm_utils.py:43
↓ 24 callersMethodcopy
(self)
videollava/conversation.py:190
↓ 16 callersMethodget_prompt
(self, qs, state)
videollava/serve/gradio_utils.py:65
↓ 15 callersFunctionload_pretrained_model
(model_path, model_base, model_name, load_8bit=False, load_4bit=False, device_map="auto", device="cuda", **kwa
videollava/model/builder.py:27
↓ 14 callersMethodbatch_decode
This method forwards all its arguments to CLIPTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of
videollava/model/multimodal_encoder/languagebind/image/processing_image.py:70
↓ 14 callersFunctionget_model_name_from_path
(model_path)
videollava/mm_utils.py:65
↓ 13 callersMethodgenerate
(self, images_tensor: list, prompt: str, first_run: bool, state)
videollava/serve/gradio_utils.py:71
↓ 13 callersMethodpreprocess
(self, images, return_tensors)
videollava/model/multimodal_encoder/languagebind/image/processing_image.py:67
↓ 13 callersMethodwrite
(self, buf)
videollava/utils.py:88
↓ 12 callersMethodget_model
(self)
videollava/model/llava_arch.py:128
↓ 9 callersFunctiondisable_torch_init
Disable the redundant torch default initialization to accelerate model creation.
videollava/utils.py:108
↓ 8 callersMethodfrom_pretrained
(cls, pretrained_model_name_or_path: Union[str, os.PathLike], **kwargs)
videollava/model/multimodal_encoder/languagebind/image/configuration_image.py:108
↓ 8 callersMethodget_video_tower
(self)
videollava/model/llava_arch.py:45
↓ 7 callersMethod__init__
(self, config: LanguageBindImageConfig)
videollava/model/multimodal_encoder/languagebind/image/modeling_image.py:346
↓ 7 callersMethod__init__
(self, config: LanguageBindAudioConfig)
videollava/model/multimodal_encoder/languagebind/audio/modeling_audio.py:346
↓ 7 callersMethod__init__
(self, config: LanguageBindDepthConfig)
videollava/model/multimodal_encoder/languagebind/depth/modeling_depth.py:346
↓ 7 callersMethod__init__
(self, config: LanguageBindThermalConfig)
videollava/model/multimodal_encoder/languagebind/thermal/modeling_thermal.py:346
↓ 7 callersMethod__init__
(self, config: LanguageBindVideoConfig)
videollava/model/multimodal_encoder/languagebind/video/modeling_video.py:346
↓ 6 callersFunctioncreate_custom_forward
(module)
videollava/model/language_model/mpt/hf_prefixlm_converter.py:207
↓ 6 callersMethodflush
(self)
videollava/utils.py:102
↓ 5 callersFunction_cast_if_autocast_enabled
(tensor)
videollava/model/language_model/mpt/norm.py:3
↓ 5 callersFunctiongeneric_param_init_fn_
(module: nn.Module, init_fn_, n_layers: int, d_model: Optional[int]=None, init_div_is_residual: Union[int, flo
videollava/model/language_model/mpt/param_init_fns.py:28
↓ 5 callersFunctionget_answer
(problem, options)
scripts/convert_sqa_to_llava_base_prompt.py:25
↓ 5 callersFunctionget_choice_text
(probelm, options)
scripts/convert_sqa_to_llava_base_prompt.py:15
↓ 5 callersFunctionget_context_text
(problem, use_caption)
scripts/convert_sqa_to_llava_base_prompt.py:6
↓ 5 callersMethodget_image_tower
(self)
videollava/model/llava_arch.py:39
↓ 5 callersMethodget_input_embeddings
(self)
videollava/model/language_model/mpt/modeling_mpt.py:81
↓ 5 callersFunctionget_lecture_text
(problem)
scripts/convert_sqa_to_llava_base_prompt.py:29
↓ 5 callersFunctionget_question_text
(problem)
scripts/convert_sqa_to_llava_base_prompt.py:1
↓ 5 callersFunctionget_solution_text
(problem)
scripts/convert_sqa_to_llava_base_prompt.py:35
↓ 5 callersMethodload_model
(self)
videollava/model/multimodal_encoder/clip_encoder.py:22
↓ 5 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
videollava/model/multimodal_encoder/languagebind/image/configuration_image.py:402
↓ 4 callersFunctioncall_og_forward
()
videollava/model/language_model/mpt/hf_prefixlm_converter.py:74
↓ 4 callersMethoddecode
This method forwards all its arguments to CLIPTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this
videollava/model/multimodal_encoder/languagebind/image/processing_image.py:77
↓ 4 callersFunctionget_length_grouped_indices
(lengths, batch_size, world_size, generator=None, merge=True)
videollava/train/llava_trainer.py:88
↓ 4 callersFunctionget_model_output
(model, video_processor, tokenizer, video, qs, args)
videollava/eval/video/run_inference_video_qa.py:49
↓ 4 callersMethodget_output_embeddings
(self)
videollava/model/language_model/mpt/modeling_mpt.py:261
↓ 4 callersFunctionlength
(r)
videollava/eval/eval_gqa.py:286
↓ 4 callersFunctionloadFile
(name)
videollava/eval/eval_gqa.py:108
↓ 4 callersFunctionorder_pick_k
(lst, k)
videollava/utils.py:18
↓ 4 callersFunctionpreprocess
Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\
videollava/train/train.py:612
↓ 4 callersFunctionprocess_images
(images, image_processor, model_cfg)
videollava/mm_utils.py:28
↓ 4 callersFunctionread_jsonl
(path: str, key: str=None)
videollava/eval/generate_webpage_data_from_table.py:10
↓ 4 callersFunctiontext2Markdown
(text)
videollava/eval/webpage/script.js:35
↓ 4 callersFunctiontoScore
(b)
videollava/eval/eval_gqa.py:168
↓ 4 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
videollava/model/multimodal_encoder/languagebind/audio/configuration_audio.py:409
↓ 4 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
videollava/model/multimodal_encoder/languagebind/depth/configuration_depth.py:404
↓ 4 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
videollava/model/multimodal_encoder/languagebind/thermal/configuration_thermal.py:402
↓ 4 callersMethodto_dict
Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str,
videollava/model/multimodal_encoder/languagebind/video/configuration_video.py:402
↓ 4 callersMethodto_gradio_chatbot
(self)
videollava/conversation.py:159
↓ 3 callersFunction_flash_attn_backward
(do, q, k, v, o, lse, dq, dk, dv, bias=None, causal=False, softmax_scale=None)
videollava/model/language_model/mpt/flash_attn_triton.py:366
↓ 3 callersFunction_flash_attn_forward
(q, k, v, bias=None, causal=False, softmax_scale=None)
videollava/model/language_model/mpt/flash_attn_triton.py:329
↓ 3 callersFunctionavg
(l)
videollava/eval/eval_gqa.py:173
↓ 3 callersFunctiondisplayQuestion
(index)
videollava/eval/webpage/script.js:110
↓ 3 callersFunctioneval_single
(result_file, eval_only_type=None)
scripts/convert_seed_for_submission.py:14
↓ 3 callersMethodget_mel
(self, audio_data)
videollava/model/multimodal_encoder/languagebind/audio/processing_audio.py:116
↓ 3 callersFunctionget_pred_idx
Get the index (e.g. 2) from the prediction (e.g. 'C')
videollava/eval/eval_science_qa_gpt4_requery.py:30
↓ 3 callersMethodget_worker_status
(self, worker_name: str)
videollava/serve/controller.py:88
↓ 3 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
videollava/train/train.py:122
↓ 3 callersFunctionpreprocess_multimodal
( sources: Sequence[str], data_args: DataArguments )
videollava/train/train.py:315
↓ 3 callersFunctiontrain
()
videollava/train/train.py:881
↓ 3 callersFunctionupdateQuestionSelect
(question_id)
videollava/eval/webpage/script.js:48
↓ 3 callersFunctionxrange
(c)
videollava/eval/eval_gqa.py:282
↓ 3 callersFunctionyrange
(c)
videollava/eval/eval_gqa.py:278
↓ 2 callersMethod__init__
(self, normalized_shape, eps=1e-05, weight=True, dtype=None, device=None)
videollava/model/language_model/mpt/norm.py:35
↓ 2 callersMethod__init__
(self, clip_type=('thermal', 'image', 'video', 'depth', 'audio'), use_temp=True, cache_dir='./cache_dir')
videollava/model/multimodal_encoder/languagebind/__init__.py:55
↓ 2 callersMethod__init__
( self, vocab_size=49408, hidden_size=512, intermediate_size=2048, pro
videollava/model/multimodal_encoder/languagebind/image/configuration_image.py:70
↓ 2 callersMethod__init__
( self, vocab_size=49408, hidden_size=512, intermediate_size=2048, pro
videollava/model/multimodal_encoder/languagebind/audio/configuration_audio.py:70
↓ 2 callersMethod__init__
( self, vocab_size=49408, hidden_size=512, intermediate_size=2048, pro
videollava/model/multimodal_encoder/languagebind/depth/configuration_depth.py:70
↓ 2 callersMethod__init__
( self, vocab_size=49408, hidden_size=512, intermediate_size=2048, pro
videollava/model/multimodal_encoder/languagebind/thermal/configuration_thermal.py:70
↓ 2 callersMethod__init__
( self, vocab_size=49408, hidden_size=512, intermediate_size=2048, pro
videollava/model/multimodal_encoder/languagebind/video/configuration_video.py:70
↓ 2 callersFunction_bwd_kernel_one_col_block
(start_n, Q, K, V, Bias, DO, DQ, DK, DV, LSE, D, softmax_scale, stride_qm, stride_kn, stride_vn, stride_bm, st
videollava/model/language_model/mpt/flash_attn_triton.py:184
↓ 2 callersFunction_bwd_store_dk_dv
(dk_ptrs, dv_ptrs, dk, dv, offs_n, offs_d, seqlen_k, headdim, EVEN_M: tl.constexpr, EVEN_N: tl.constexpr, EVEN
videollava/model/language_model/mpt/flash_attn_triton.py:168
↓ 2 callersFunction_get_attn_modules
Helper that gets a list of the model's attention modules. Each module has a `bias` buffer used for causal masking. The Prefix LM conv
videollava/model/language_model/mpt/hf_prefixlm_converter.py:45
↓ 2 callersFunction_normal_param_init_fn_
(module: nn.Module, std: float, n_layers: int, d_model: Optional[int]=None, init_div_is_residual: Union[int, f
videollava/model/language_model/mpt/param_init_fns.py:124
↓ 2 callersFunction_reset_is_causal
(num_query_tokens: int, num_key_tokens: int, original_is_causal: bool)
videollava/model/language_model/mpt/attention.py:12
↓ 2 callersMethod_set_config_defaults
(self, config, config_defaults)
videollava/model/language_model/mpt/configuration_mpt.py:90
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
videollava/train/train.py:256
↓ 2 callersMethod_validate_config
(self)
videollava/model/language_model/mpt/configuration_mpt.py:96
↓ 2 callersFunctionauto_upgrade
(config)
videollava/model/utils.py:4
↓ 2 callersFunctionbelongs
(element, group, question)
videollava/eval/eval_gqa.py:246
↓ 2 callersFunctionbuild_image_tower
(image_tower_cfg, **kwargs)
videollava/model/multimodal_encoder/builder.py:7
↓ 2 callersFunctionbuild_logger
(logger_name, logger_filename)
videollava/utils.py:32
↓ 2 callersFunctionbuild_prompt_chatbot
(problems, shot_qids, prompt_format, use_caption=False, options=["A", "B", "C", "D", "E"], is_test=False)
scripts/convert_sqa_to_llava_base_prompt.py:221
↓ 2 callersFunctionbuild_video_tower
(video_tower_cfg, **kwargs)
videollava/model/multimodal_encoder/builder.py:17
↓ 2 callersFunctionbuild_vision_projector
(config, delay_load=False, **kwargs)
videollava/model/multimodal_projector/builder.py:33
↓ 2 callersFunctioncheck_valid_inputs
(*tensors, valid_dtypes=[torch.float16, torch.bfloat16])
videollava/model/language_model/mpt/attention.py:64
↓ 2 callersFunctioncreate_one_example
(format, question, context, choice, answer, lecture, solution, test_example=True)
scripts/convert_sqa_to_llava_base_prompt.py:106
↓ 2 callersFunctioncreate_one_example_gpt4
(format, question, context, choice, answer, lecture, solution, test_example=True)
scripts/convert_sqa_to_llava_base_prompt.py:162
↓ 2 callersFunctiondisplayAnswers
(index)
videollava/eval/webpage/script.js:116
↓ 2 callersMethodembed_tokens
(self, x)
videollava/model/language_model/llava_mpt.py:41
↓ 2 callersFunctioneval_single
(annotation_file, result_file)
videollava/eval/eval_textvqa.py:35
↓ 2 callersFunctionexpand2square
(pil_img, background_color)
videollava/mm_utils.py:14
↓ 2 callersFunctionexpand2square
(pil_img, background_color)
videollava/train/train.py:659
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
videollava/model/multimodal_encoder/clip_encoder.py:29
↓ 2 callersMethodfeature_select
(self, image_forward_outs)
videollava/model/multimodal_encoder/languagebind/__init__.py:121
↓ 2 callersMethodfeature_select
(self, video_forward_outs)
videollava/model/multimodal_encoder/languagebind/__init__.py:207
↓ 2 callersFunctionformat_example
(df, idx, include_answer=True)
videollava/eval/eval_mmlu.py:100
next →1–100 of 670, ranked by callers