MCPcopy Create free account

hub / github.com/KlingAIResearch/VideoAlign / functions

Functions63 in github.com/KlingAIResearch/VideoAlign

⨍ Functions 63◇ Types & classes 9
↓ 4 callersFunctionbuild_prompt
(prompt, dimension, template_type)
prompt_template.py:100
↓ 4 callersFunctionceil_by_factor
Returns the smallest integer greater than or equal to 'number' that is divisible by 'factor'.
vision_process.py:45
↓ 4 callersFunctionround_by_factor
Returns the closest integer to 'number' that is divisible by 'factor'.
vision_process.py:40
↓ 4 callersFunctionset_requires_grad
(parameters, requires_grad)
train_reward.py:65
↓ 4 callersFunctionsmart_resize
Rescales the image so that the following conditions are met: 1. Both dimensions (height and width) are divisible by 'factor'. 2. The to
vision_process.py:55
↓ 3 callersFunctionfloor_by_factor
Returns the largest integer less than or equal to 'number' that is divisible by 'factor'.
vision_process.py:50
↓ 3 callersFunctionprocess_vision_info
( conversations: list[dict] | list[list[dict]], )
vision_process.py:358
↓ 2 callersMethod__init__
(self, config, output_dim=4, reward_token="last", special_token_ids=None)
trainer.py:60
↓ 2 callersMethod_clean_message
remove unnecessary keys from message(very very necessary)
data.py:142
↓ 2 callersMethod_pad_sequence
Pad the sequences to the maximum length.
data.py:171
↓ 2 callersMethod_prepare_inputs
Prepare `inputs` before feeding them to the model, converting them to tensors if they are not already and handling potential state.
inference.py:112
↓ 2 callersFunctioncalc_accuracy_with_ties
algorithm: https://arxiv.org/abs/2305.14324 O(N^2logN) Input: h: list of N human labels, 1 for prefer A, -1 for prefer B, 0 for t
calc_accuracy.py:37
↓ 2 callersFunctioncalc_accuracy_without_ties
Input: h: list of N human labels, 1 for prefer A, -1 for prefer B, 0 for ties m: list of N model predictions, can be obtained by
calc_accuracy.py:102
↓ 2 callersFunctioncreate_dataset
(data_config, meta_file=None)
train_reward.py:141
↓ 2 callersFunctioncreate_model_and_processor
( model_config, peft_lora_config, training_args, cache_dir=None, )
train_reward.py:69
↓ 2 callersFunctionfetch_image
(ele: dict[str, str | Image.Image], size_factor: int = IMAGE_FACTOR)
vision_process.py:84
↓ 2 callersFunctionload_model_from_checkpoint
( model, checkpoint_dir, checkpoint_step )
utils.py:163
↓ 2 callersFunctionmaybe_zero_3
(param, ignore_status=False, name=None)
utils.py:89
↓ 2 callersMethodreward
Inputs: video_paths: List[str], B paths of the videos. prompts: List[str], B prompts for the videos. eval
inference.py:176
↓ 2 callersFunctionsmart_nframes
calculate the number of frames for video used for model inputs. Args: ele (dict): a dict contains the configuration of video.
vision_process.py:129
↓ 2 callersFunctionsuff_stats
+------------+-----------+-----------+-----------+ | Notation | Model Prediction | | | < | =
calc_accuracy.py:1
↓ 1 callersFunction_convert_A_B_to_chosen_rejected
Inputs: rewards_A: [B, N] rewards_B: [B, N] scores_A: [B, N] scores_B: [B, N] chosen_label: [B, N] Ou
trainer.py:176
↓ 1 callersFunction_insert_adapter_name_into_state_dict
Utility function to remap the state_dict keys to fit the PEFT model by inserting the adapter name.
utils.py:136
↓ 1 callersMethod_norm
(self, reward)
inference.py:67
↓ 1 callersMethod_prepare_input
Prepare `inputs` before feeding them to the model, converting them to tensors if they are not already and handling potential state.
inference.py:92
↓ 1 callersFunctionadd_idx
(example, idx)
train_reward.py:145
↓ 1 callersFunctioncalc_acc
(C, D, Th, Tm, Thm)
calc_accuracy.py:33
↓ 1 callersMethodcompute_loss
( self, model, inputs, return_outputs=False, )
trainer.py:424
↓ 1 callersFunctionconvert_GSB_csv_to_reward_data
Convert Good/Same/Bad csv data to reward data. Args: example (dict): A dataframe containing the GSB csv data. data_dir (str)
data.py:31
↓ 1 callersFunctionconvert_pair_to_single
(df_pair_anno)
eval_videogen_rewardbench.py:16
↓ 1 callersFunctionconvert_single_to_pair
(df_pair_anno, df_single_pred)
eval_videogen_rewardbench.py:31
↓ 1 callersFunctionextract_vision_info
(conversations: list[dict] | list[list[dict]])
vision_process.py:340
↓ 1 callersFunctionfetch_video
(ele: dict, image_factor: int = IMAGE_FACTOR)
vision_process.py:293
↓ 1 callersFunctionfind_target_linear_names
Find the target linear modules for LoRA.
train_reward.py:43
↓ 1 callersFunctionget_peft_state_non_lora_maybe_zero_3
(named_params, require_grad_only=True)
utils.py:127
↓ 1 callersFunctionget_video_reader_backend
()
vision_process.py:282
↓ 1 callersFunctionis_decord_available
()
vision_process.py:221
↓ 1 callersFunctionload_configs_from_json
(config_path)
inference.py:18
↓ 1 callersFunctionmain
()
eval_videogen_rewardbench.py:51
↓ 1 callersMethodprepare_batch
(self, video_paths, prompts, fps=None, num_frames=None, max_pixels=None,)
inference.py:122
↓ 1 callersFunctionsave_configs_to_json
Save all configurations to a JSON file.
train_reward.py:21
↓ 1 callersFunctiontrain
()
train_reward.py:164
Method__call__
Preprocess inputs to token sequences and return a batch
data.py:187
Method__init__
(self, processor, add_noise=False, p_shuffle_frames=0.0, p_color_jitter=0.0)
data.py:132
Method__init__
(self, load_from_pretrained, load_from_pretrained_step=-1, device='cuda', dtype=torch.bfloat16)
inference.py:30
Method__init__
(self, special_token_ids)
trainer.py:223
Method__init__
(self, loss_type="regular", enable_noise_in_eval=False, *args, **kwargs)
trainer.py:245
Method__post_init__
(self)
utils.py:47
Method__post_init__
(self)
utils.py:77
Method_pad_sequence
Pad the sequences to the maximum length.
inference.py:76
Function_read_video_decord
read video using decord.VideoReader Args: ele (dict): a dict contains the configuration of video. support keys: - vid
vision_process.py:227
Function_read_video_torchvision
read video using torchvision.io.read_video Args: ele (dict): a dict contains the configuration of video. support keys:
vision_process.py:169
Method_save
(self, output_dir: Optional[str] = None, state_dict=None)
trainer.py:559
Method_save_checkpoint
(self, model, trial, metrics=None)
trainer.py:528
Functioncompute_multi_attr_accuracy
(eval_pred, metainfo_idxs=None, eval_dims=None, save_path=None)
trainer.py:602
Methodcreate_optimizer
Setup the optimizer. We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the
trainer.py:318
Methodforward
( self, input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = N
trainer.py:71
Methodget_eval_dataloader
Returns the evaluation [`~torch.utils.data.DataLoader`]. Subclass and override this method if you want to inject some custom behavio
trainer.py:257
Functionget_peft_state_maybe_zero_3
(named_params, bias)
utils.py:103
Methodon_step_end
(self, args, state, control, **kwargs)
trainer.py:232
Methodon_train_begin
(self, args, state, control, **kwargs)
trainer.py:228
Methodprediction_step
( self, model, inputs, prediction_loss_only, ignore_keys = None, )
trainer.py:500
Functionsave_video
(tensor, path)
utils.py:155