MCPcopy Create free account

hub / github.com/PKU-YuanGroup/Chat-UniVi / functions

Functions291 in github.com/PKU-YuanGroup/Chat-UniVi

⨍ Functions 291◇ Types & classes 52
↓ 127 callersFunctionprint
(*args, **kwargs)
ChatUniVi/model/multimodal_encoder/utils.py:25
↓ 43 callersMethodget_model
(self)
ChatUniVi/model/arch.py:91
↓ 28 callersMethodcopy
(self)
ChatUniVi/conversation.py:191
↓ 27 callersFunctiontokenizer_image_token
(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None)
ChatUniVi/mm_utils.py:18
↓ 24 callersMethodappend_message
(self, role, message)
ChatUniVi/conversation.py:106
↓ 13 callersMethodget_prompt
(self, qs, state)
ChatUniVi/demo.py:34
↓ 12 callersMethodget_vision_tower
(self)
ChatUniVi/model/arch.py:36
↓ 12 callersMethodpreprocess
(self, item, return_tensors)
ChatUniVi/model/multimodal_encoder/processor.py:48
↓ 11 callersMethod__init__
( self, config: PretrainedConfig, n_inner: Optional[int] = None, act_fn: Optio
ChatUniVi/model/language_model/language_model/modeling_phi.py:304
↓ 11 callersMethodgenerate
(self, images_tensor: list, prompt: str, first_run: bool, state)
ChatUniVi/demo.py:77
↓ 9 callersMethodload_model
(self)
ChatUniVi/model/multimodal_encoder/eva_encoder.py:22
↓ 8 callersMethodto_gradio_chatbot
(self)
ChatUniVi/conversation.py:158
↓ 7 callersFunctiondisable_torch_init
Disable the redundant torch default initialization to accelerate model creation.
ChatUniVi/utils.py:93
↓ 7 callersMethodflush
(self)
ChatUniVi/utils.py:87
↓ 7 callersFunctionload_pretrained_model
(model_path, model_base, model_name, load_8bit=False, load_4bit=False, device_map="auto")
ChatUniVi/model/builder.py:11
↓ 7 callersMethodwrite
(self, buf)
ChatUniVi/utils.py:73
↓ 6 callersMethod__init__
(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.)
ChatUniVi/model/multimodal_encoder/eva_vit.py:47
↓ 5 callersMethodget_input_embeddings
(self)
ChatUniVi/model/language_model/language_model/modeling_phi.py:906
↓ 4 callersMethodget_output_embeddings
(self)
ChatUniVi/model/language_model/language_model/modeling_phi.py:963
↓ 4 callersFunctionmerge
(image, token_dict, patch_size=14, alpha=0.2, line_color=np.array([200, 200, 200]))
visualization.py:29
↓ 4 callersFunctionread_json
(file)
data/data_generation.py:33
↓ 4 callersFunctiontrunc_normal_
r"""Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distributi
ChatUniVi/model/cluster.py:43
↓ 3 callersMethod__init__
(self, mean=None, std=None)
ChatUniVi/model/multimodal_encoder/processor.py:7
↓ 3 callersFunctionindex_points
Sample features following the index. Returns: new_points:, indexed points data, [B, S, C] Args: points: input points data, [B
ChatUniVi/model/cluster.py:91
↓ 3 callersFunctionis_dist_avail_and_initialized
()
ChatUniVi/model/multimodal_encoder/utils.py:33
↓ 3 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
ChatUniVi/train/train.py:110
↓ 3 callersFunctionpreprocess
Given a list of sources, each is a conversation list. This transform: 1. Add signal '### ' at the beginning each sentence, with end signal '\
ChatUniVi/train/train.py:701
↓ 3 callersMethodproject
(self, image_features, input_type="image")
ChatUniVi/model/arch.py:111
↓ 3 callersFunctionread_jsonl
(file)
data/data_generation.py:25
↓ 3 callersFunctionwrite_json
(file, data)
data/data_generation.py:39
↓ 2 callersMethod__init__
(self, sample_ratio, embed_dim, dim_out, k=5)
ChatUniVi/model/cluster.py:227
↓ 2 callersMethod_compute_inv_freq
(self, device: Optional[str] = None)
ChatUniVi/model/language_model/language_model/modeling_phi.py:219
↓ 2 callersMethod_forward_cross_attn
( self, x: torch.FloatTensor, past_key_values: Optional[InferenceParams], key_
ChatUniVi/model/language_model/language_model/modeling_phi.py:628
↓ 2 callersFunction_get_rawvideo_dec
(video_path, image_processor, max_frames=MAX_IMAGE_LENGTH, image_resolution=224, video_framerate=1, s=None, e=
ChatUniVi/eval/model_video_qa.py:34
↓ 2 callersMethod_get_rawvideo_dec
(self, video_path, image_processor, max_frames=MAX_IMAGE_LENGTH, image_resolution=224,
ChatUniVi/demo.py:39
↓ 2 callersFunction_tokenize_fn
Tokenize a list of strings.
ChatUniVi/train/train.py:240
↓ 2 callersMethod_update_cos_sin_cache
( self, seqlen: int, device: Optional[str] = None, dtype: Optional[torch.dtype
ChatUniVi/model/language_model/language_model/modeling_phi.py:222
↓ 2 callersFunctionbuild_vision_tower
(vision_tower_cfg, **kwargs)
ChatUniVi/model/multimodal_encoder/builder.py:5
↓ 2 callersFunctiondrop_path
Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
ChatUniVi/model/cluster.py:67
↓ 2 callersMethodencode_images
(self, images)
ChatUniVi/model/arch.py:97
↓ 2 callersMethodfeature_select
(self, image_forward_outs, select_feature='patch')
ChatUniVi/model/multimodal_encoder/eva_encoder.py:32
↓ 2 callersMethodfeature_select
(self, image_forward_outs, select_feature='patch')
ChatUniVi/model/multimodal_encoder/clip_encoder.py:34
↓ 2 callersMethodget_intermediate_layers
(self, x)
ChatUniVi/model/multimodal_encoder/eva_vit.py:360
↓ 2 callersFunctionget_pred_idx
Get the index (e.g. 2) from the prediction (e.g. 'C')
ChatUniVi/eval/evaluate/evaluate_science_qa.py:29
↓ 2 callersFunctionnorm_cdf
(x)
ChatUniVi/model/cluster.py:10
↓ 2 callersFunctionparse_args
()
ChatUniVi/eval/evaluate/summarize_gpt_review.py:8
↓ 2 callersMethodprepare_inputs_labels_for_multimodal
( self, input_ids, attention_mask, past_key_values, labels, images )
ChatUniVi/model/arch.py:220
↓ 2 callersFunctionpreprocess_multimodal
( sources: Sequence[str], data_args: DataArguments, image_token_num=1 )
ChatUniVi/train/train.py:299
↓ 2 callersFunctionrank0_print
(*args)
ChatUniVi/train/train.py:41
↓ 2 callersFunctionsave_image_to_local
(image)
main_demo_7B.py:18
↓ 2 callersFunctionsave_image_to_local
(image)
main_demo_13B.py:18
↓ 2 callersFunctiontrain
()
ChatUniVi/train/train.py:982
↓ 1 callersFunctionCOCO_CAP
(args)
data/data_generation.py:44
↓ 1 callersFunctionCOCO_REC
(args)
data/data_generation.py:78
↓ 1 callersFunctionCOCO_REG
(args)
data/data_generation.py:132
↓ 1 callersMethod__init__
(self, config: LlamaConfig)
ChatUniVi/model/language_model/llama.py:19
↓ 1 callersMethod__init__
(self, config: PhiConfig)
ChatUniVi/model/language_model/phi.py:36
↓ 1 callersMethod__init__
(self)
ChatUniVi/model/multimodal_projector/builder.py:7
↓ 1 callersFunction_add_speaker_and_signal
Add speaker and start/end signal on each round.
ChatUniVi/train/train.py:278
↓ 1 callersFunction_apply_rotary_emb
( x: torch.FloatTensor, cos: torch.FloatTensor, sin: torch.FloatTensor, )
ChatUniVi/model/language_model/language_model/modeling_phi.py:88
↓ 1 callersFunction_apply_rotary_emb_kv
( kv: torch.FloatTensor, cos: torch.FloatTensor, sin: torch.FloatTensor, cos_k: Optional[torch
ChatUniVi/model/language_model/language_model/modeling_phi.py:109
↓ 1 callersFunction_apply_rotary_emb_qkv
( qkv: torch.FloatTensor, cos: torch.FloatTensor, sin: torch.FloatTensor, cos_k: Optional[torc
ChatUniVi/model/language_model/language_model/modeling_phi.py:138
↓ 1 callersFunction_cfg
(url='', **kwargs)
ChatUniVi/model/multimodal_encoder/eva_vit.py:21
↓ 1 callersFunction_find_mha_dims
( config: PretrainedConfig, n_head: Optional[int] = None, n_head_kv: Optional[int] = None, hea
ChatUniVi/model/language_model/language_model/modeling_phi.py:462
↓ 1 callersMethod_forward_self_attn
( self, x: torch.FloatTensor, key_padding_mask: Optional[torch.BoolTensor] )
ChatUniVi/model/language_model/language_model/modeling_phi.py:595
↓ 1 callersFunction_get_rawvideo_dec
(video_path, image_processor, max_frames=64, image_resolution=224, video_framerate=1, s=None, e=None)
ChatUniVi/model/dataloader.py:9
↓ 1 callersFunction_get_rawvideo_dec
(video_path, image_processor, max_frames=MAX_IMAGE_LENGTH, image_resolution=224, video_framerate=1, s=None, e=
ChatUniVi/eval/model_video_consistency.py:29
↓ 1 callersFunction_get_rawvideo_dec
(video_path, image_processor, max_frames=MAX_IMAGE_LENGTH, image_resolution=224, video_framerate=1, s=None, e=
ChatUniVi/eval/model_video_general.py:29
↓ 1 callersFunction_mask_targets
(target, tokenized_lens, speakers)
ChatUniVi/train/train.py:267
↓ 1 callersFunction_no_grad_trunc_normal_
(tensor, mean, std, a, b)
ChatUniVi/model/cluster.py:7
↓ 1 callersMethod_save
(self, output_dir: Optional[str] = None, state_dict=None)
ChatUniVi/train/trainer.py:49
↓ 1 callersFunction_update_kv_cache
(kv: torch.FloatTensor, inference_params: InferenceParams, layer_idx: int)
ChatUniVi/model/language_model/language_model/modeling_phi.py:480
↓ 1 callersFunctionapply_delta
(base_model_path, target_model_path, delta_path)
ChatUniVi/model/apply_delta.py:9
↓ 1 callersFunctionbuild_vision_projector
(config, delay_load=False, **kwargs)
ChatUniVi/model/multimodal_projector/builder.py:33
↓ 1 callersFunctioncluster_dpc_knn
Cluster tokens with DPC-KNN algorithm. Return: idx_cluster (Tensor[B, N]): cluster index of each token. cluster_num (int): actual
ChatUniVi/model/cluster.py:111
↓ 1 callersFunctionconsolidate_ckpt
(src_path, dst_path)
ChatUniVi/model/consolidate.py:13
↓ 1 callersFunctionconvert_weights_to_fp16
Convert applicable model parameters to fp16
ChatUniVi/model/multimodal_encoder/eva_vit.py:403
↓ 1 callersFunctioncreate_eva_vit_g
(img_size=224, drop_path_rate=0.4, use_checkpoint=False, precision="fp16")
ChatUniVi/model/multimodal_encoder/eva_vit.py:421
↓ 1 callersFunctiondownload_cached_file
Download a file from a URL and cache it locally. If the file already exists, it is not downloaded again. If distributed, only the main proces
ChatUniVi/model/multimodal_encoder/utils.py:117
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_coco_vqa.py:81
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_vqa_scienceqa.py:30
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_video_consistency.py:90
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_video_general.py:90
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_vqa.py:27
↓ 1 callersFunctioneval_model
(args)
ChatUniVi/eval/model_video_qa.py:95
↓ 1 callersFunctionfind_all_linear_names
(model)
ChatUniVi/train/train.py:164
↓ 1 callersMethodfix_init_weight
(self)
ChatUniVi/model/multimodal_encoder/eva_vit.py:305
↓ 1 callersMethodforward_features
(self, x)
ChatUniVi/model/multimodal_encoder/eva_vit.py:329
↓ 1 callersFunctionget_acc
(file)
ChatUniVi/eval/model_coco_vqa.py:19
↓ 1 callersFunctionget_args
()
ChatUniVi/eval/evaluate/evaluate_science_qa.py:9
↓ 1 callersFunctionget_cached_file_path
()
ChatUniVi/model/multimodal_encoder/utils.py:123
↓ 1 callersFunctionget_chunk
(lst, n, k)
ChatUniVi/eval/model_coco_vqa.py:67
↓ 1 callersFunctionget_chunk
(lst, n, k)
ChatUniVi/eval/model_vqa_scienceqa.py:25
↓ 1 callersFunctionget_chunk
(lst, n, k)
ChatUniVi/eval/model_vqa.py:22
↓ 1 callersFunctionget_dist_info
()
ChatUniVi/model/multimodal_encoder/utils.py:93
↓ 1 callersFunctionget_eval
(content: str, max_tokens: int)
ChatUniVi/eval/evaluate/evaluate_gpt_review_visual.py:11
↓ 1 callersMethodget_images
(self, return_pil=False)
ChatUniVi/conversation.py:109
↓ 1 callersFunctionget_mm_adapter_state_maybe_zero_3
(named_params, keys_to_match)
ChatUniVi/train/train.py:158
↓ 1 callersFunctionget_mm_adapter_state_maybe_zero_3
(named_params, keys_to_match)
ChatUniVi/train/trainer.py:21
↓ 1 callersFunctionget_peft_state_maybe_zero_3
(named_params, bias)
ChatUniVi/train/train.py:125
next →1–100 of 291, ranked by callers