MCPcopy Create free account

hub / github.com/bytedance/LatentSync / functions

Functions512 in github.com/bytedance/LatentSync

Methodforward
x : torch.LongTensor, shape = (batch_size, <= n_ctx) the text tokens xa : torch.Tensor, shape = (batch_size, n_mels, n_au
latentsync/whisper/whisper/model.py:189
Methodforward
(self, mel: torch.Tensor, tokens: torch.Tensor)
latentsync/whisper/whisper/model.py:245
Methodforward
r""" Args: sample (`torch.FloatTensor`): (batch, channel, height, width) noisy inputs tensor timestep (`torch.FloatTen
latentsync/models/unet.py:312
Methodforward
(self, image_sequences, audio_sequences)
latentsync/models/wav2lip_syncnet.py:57
Methodforward
(self, x)
latentsync/models/wav2lip_syncnet.py:86
Methodforward
(self, image_sequences, audio_sequences)
latentsync/models/stable_syncnet.py:51
Methodforward
(self, input_tensor)
latentsync/models/stable_syncnet.py:111
Methodforward
(self, hidden_states)
latentsync/models/stable_syncnet.py:150
Methodforward
(self, hidden_states)
latentsync/models/stable_syncnet.py:219
Methodforward
(self, hidden_states, temb=None, encoder_hidden_states=None, attention_mask=None)
latentsync/models/unet_blocks.py:247
Methodforward
(self, hidden_states, temb=None, encoder_hidden_states=None, attention_mask=None)
latentsync/models/unet_blocks.py:359
Methodforward
(self, hidden_states, temb=None, encoder_hidden_states=None)
latentsync/models/unet_blocks.py:478
Methodforward
( self, hidden_states, res_hidden_states_tuple, temb=None, encoder_hid
latentsync/models/unet_blocks.py:611
Methodforward
( self, hidden_states, res_hidden_states_tuple, temb=None, upsample_si
latentsync/models/unet_blocks.py:733
Methodforward
(self, hidden_states, encoder_hidden_states=None, timestep=None, return_dict: bool = True)
latentsync/models/attention.py:82
Methodforward
( self, hidden_states, encoder_hidden_states=None, timestep=None, attention_mask=None, video_length=No
latentsync/models/attention.py:174
Methodforward
(self, hidden_states, encoder_hidden_states=None, attention_mask=None)
latentsync/models/attention.py:250
Methodforward
(self, x)
latentsync/models/resnet.py:11
Methodforward
(self, x)
latentsync/models/resnet.py:22
Methodforward
(self, hidden_states, output_size=None)
latentsync/models/resnet.py:47
Methodforward
(self, hidden_states)
latentsync/models/resnet.py:93
Methodforward
(self, input_tensor, temb)
latentsync/models/resnet.py:182
Methodforward
(self, input_tensor, temb, encoder_hidden_states, attention_mask=None, anchor_frame_idx=None)
latentsync/models/motion_module.py:68
Methodforward
(self, hidden_states, encoder_hidden_states=None, attention_mask=None)
latentsync/models/motion_module.py:126
Methodforward
(self, hidden_states, encoder_hidden_states=None, attention_mask=None, video_length=None)
latentsync/models/motion_module.py:203
Methodforward
(self, x)
latentsync/models/motion_module.py:232
Methodforward
(self, hidden_states, encoder_hidden_states=None, attention_mask=None, video_length=None)
latentsync/models/motion_module.py:262
Functionfunc
(paths, device_id)
preprocess/filter_visual_quality.py:56
Functionfunc
(sync_conf_threshold, paths, device_id, process_temp_dir)
preprocess/sync_av.py:47
Functionfunc
(paths, process_temp_dir, device_id, resolution)
preprocess/affine_transform.py:56
Methodget_all_torch
Get all the stored features as PyTorch Tensor. Returns: Concatenation of the stored features.
latentsync/trepa/utils/metric_utils.py:116
Methodget_classifier
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:115
Methodget_classifier
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:214
Methodget_classifier
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:473
Functionget_dataloader
(data_path, image_folder, resolution=128, sequence_length=16, sample_every_n_frames=1, batc
latentsync/trepa/utils/data_utils.py:22
Methodget_mean_cov
Get the mean and covariance of the stored features. Returns: Mean and covariance of the stored features.
latentsync/trepa/utils/metric_utils.py:125
Functionget_melspec_overlap
(audio_samples, melspec_length=52)
latentsync/utils/audio.py:185
Methodget_num_layers
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:108
Methodget_num_layers
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:207
Methodget_num_layers
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:332
Methodget_num_layers
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:466
Functionget_parent_dir
(path)
latentsync/trepa/utils/data_utils.py:33
Methodget_random_clip_from_video
Sample a random clip starting index from the video. Args: idx: Index of the video.
latentsync/trepa/utils/data_utils.py:180
Methodget_sliced_feature_sparse
Get sliced features based on a given index :param feature_array: :param start_idx: the start index of the feature :pa
latentsync/whisper/audio2feature.py:54
Methodinstall_hooks
(layer: nn.Module)
latentsync/whisper/whisper/model.py:280
Functioninv_preemphasis
(wav, k, inv_preemphasize=True)
latentsync/utils/audio.py:36
Methodis_full
Check if the maximum number of samples is reached. Returns: True if the storage is full, False otherwise.
latentsync/trepa/utils/metric_utils.py:54
Methodis_multilingual
(self)
latentsync/whisper/whisper/model.py:253
Methodlanguage_token
Returns the token id corresponding to the value of the `language` field
latentsync/whisper/whisper/tokenizer.py:196
Functionlibrosa_pad_lr
(x, fsize, fshift)
latentsync/utils/audio.py:104
Functionlinearspectrogram
(wav)
latentsync/utils/audio.py:50
Functionload
(filename)
eval/syncnet/syncnet.py:13
Functionload_wav
(path, sr)
latentsync/utils/audio.py:16
Methodlogits
(self, tokens: Tensor, audio_features: Tensor, include_embeddings=False)
latentsync/whisper/whisper/decoding.py:139
Methodlogits
(self, tokens: torch.Tensor, audio_features: torch.Tensor)
latentsync/whisper/whisper/model.py:242
Functionloop
(cuda_device)
tools/occupy_gpu.py:32
Functionmulti_run_wrapper
(args)
preprocess/filter_high_resolution.py:92
Functionmulti_run_wrapper
(args)
preprocess/segment_videos.py:43
Functionmulti_run_wrapper
(args)
preprocess/detect_shot.py:43
Functionmulti_run_wrapper
(args)
preprocess/resample_fps_hz.py:51
Methodno_speech
(self)
latentsync/whisper/whisper/tokenizer.py:181
Methodno_timestamps
(self)
latentsync/whisper/whisper/tokenizer.py:186
Methodno_weight_decay
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:112
Methodno_weight_decay
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:211
Methodno_weight_decay
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:336
Methodno_weight_decay
(self)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:470
Methodnon_speech_tokens
Returns the list of tokens to suppress in order to avoid any speaker tags or non-speech annotations, to prevent sampling texts that a
latentsync/whisper/whisper/tokenizer.py:237
Functionoptional_float
(string)
latentsync/whisper/whisper/utils.py:22
Functionoptional_int
(string)
latentsync/whisper/whisper/utils.py:18
Functionpad_lr
Compute left and right padding
latentsync/utils/audio.py:93
Functionparse
(x)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:80
Methodpixel_values_to_images
(pixel_values: torch.Tensor)
latentsync/pipelines/lipsync_pipeline.py:245
Methodpredict
Run a single prediction on the model
predict.py:34
Functionpreprocess_videomae
(videos)
latentsync/trepa/third_party/VideoMAEv2/utils.py:39
Functionpretrain_videomae_base_patch16_224
(pretrained=False, **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:384
Functionpretrain_videomae_giant_patch14_224
(pretrained=False, **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:450
Functionpretrain_videomae_huge_patch16_224
(pretrained=False, **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:428
Functionpretrain_videomae_large_patch16_224
(pretrained=False, **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:406
Functionpretrain_videomae_small_patch16_224
(pretrained=False, **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:362
Functionprocess_video
( video_path, audio_path, guidance_scale, inference_steps, seed, )
gradio_app.py:12
Methodrank
(self, tokens: List[List[Tensor]], sum_logprobs: List[List[float]])
latentsync/whisper/whisper/decoding.py:182
Functionread_json
(filepath: str)
latentsync/utils/util.py:40
Methodrearrange_kv_cache
(self, source_indices)
latentsync/whisper/whisper/decoding.py:158
Functionremove_broken_video
(video_path)
preprocess/remove_broken_videos.py:23
Functionremove_incorrect_affined
(video_path)
preprocess/remove_incorrect_affined.py:56
Functionremove_symbols
Replace any other markers, symbols, punctuations with a space, keeping diacritics
latentsync/whisper/whisper/normalizers/basic.py:46
Methodreset
(self)
latentsync/whisper/whisper/decoding.py:292
Methodreset_classifier
(self, num_classes, global_pool='')
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:118
Methodreset_classifier
(self, num_classes, global_pool='')
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:217
Methodreset_classifier
(self, num_classes, global_pool="")
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:476
Functionsave
(model, filename)
eval/syncnet/syncnet.py:7
Functionsave_image
(image, save_path)
latentsync/utils/util.py:223
Methodsave_to_cache
(module, _, output)
latentsync/whisper/whisper/model.py:273
Functionsave_wav
(wav, path, sr)
latentsync/utils/audio.py:20
Functionsave_wavenet_wav
(wav, path, sr)
latentsync/utils/audio.py:26
Methodscores
(logprobs, lengths)
latentsync/whisper/whisper/decoding.py:183
Functionseed_everything
(seed)
latentsync/trepa/utils/metric_utils.py:10
Methodsetup
Load the model into memory to make running multiple predictions efficient
predict.py:22
Methodsot
(self)
latentsync/whisper/whisper/tokenizer.py:166
Methodsot_lm
(self)
latentsync/whisper/whisper/tokenizer.py:171
← previousnext →401–500 of 512, ranked by callers