MCPcopy Create free account

hub / github.com/bytedance/LatentSync / functions

Functions512 in github.com/bytedance/LatentSync

↓ 136 callersMethodappend
Add the newly computed features to the list. Update the mean and covariance. Args: x: New features to record.
latentsync/trepa/utils/metric_utils.py:63
↓ 32 callersMethodload
Load the features and statistics from a pickle file. Args: pkl_file: Path to the pickle file.
latentsync/trepa/utils/metric_utils.py:149
↓ 21 callersMethodrun
(self, mel: Tensor)
latentsync/whisper/whisper/decoding.py:631
↓ 18 callersMethodencode
(self, text, **kwargs)
latentsync/whisper/whisper/tokenizer.py:137
↓ 14 callersMethodload_state_dict
(self, state_dict, strict=True)
latentsync/models/unet.py:473
↓ 13 callersMethodupdate
Specify how to select the next token, based on the current trace and logits Parameters ---------- tokens : Tensor, shape = (n
latentsync/whisper/whisper/decoding.py:203
↓ 10 callersMethodsave
Save the features and statistics to a pickle file. Args: pkl_file: Path to the pickle file.
latentsync/trepa/utils/metric_utils.py:138
↓ 9 callersMethodfrom_pretrained
(cls, model_config: dict, ckpt_path: str, device="cpu")
latentsync/models/unet.py:495
↓ 8 callersMethodclose
(self)
preprocess/filter_high_resolution.py:72
↓ 7 callersFunctioncheck_model_and_download
(ckpt_path: str, huggingface_model_id: str = "ByteDance/LatentSync-1.5")
latentsync/utils/util.py:277
↓ 7 callersMethoddevice
(self)
latentsync/whisper/whisper/model.py:249
↓ 6 callersMethod__init__
(self, in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.0)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:135
↓ 6 callersFunction_cfg
(url="", **kwargs)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:105
↓ 6 callersMethodapply
Apply any filtering or masking to logits in-place Parameters ---------- logits : Tensor, shape = (n_batch, vocab_size)
latentsync/whisper/whisper/decoding.py:372
↓ 6 callersMethoddecode
(self, token_ids: Union[int, List[int], np.ndarray, torch.Tensor], **kwargs)
latentsync/whisper/whisper/tokenizer.py:140
↓ 6 callersMethodget_batch
Get entire batch of audio samples and video frames. Parameters ---------- indices : list of integers A list of fr
latentsync/utils/av_reader.py:95
↓ 6 callersMethodsplit_heads
(self, tensor)
latentsync/models/attention.py:238
↓ 5 callersMethod_get_single_token_id
(self, text)
latentsync/whisper/whisper/tokenizer.py:267
↓ 5 callersFunctiongather_video_paths_recursively
(input_dir)
latentsync/utils/util.py:246
↓ 5 callersFunctionget_motion_module
(in_channels, motion_module_type: str, motion_module_kwargs: dict)
latentsync/models/motion_module.py:29
↓ 4 callersMethod__init__
(self, lda_out_channels, hyper_in_channels, target_in_size, target_fc1_size, target_fc2_size, target_fc3_size,
eval/hyper_iqa.py:34
↓ 4 callersMethod__init__
(self, dims: ModelDimensions)
latentsync/whisper/whisper/model.py:221
↓ 4 callersMethod__init__
( self, in_channels: int, prev_output_channel: int, out_channels: int,
latentsync/models/unet_blocks.py:670
↓ 4 callersMethod__init__
(self, d_model, dropout=0.0, max_len=24)
latentsync/models/motion_module.py:222
↓ 4 callersMethod_make_layer
(self, block, planes, blocks, stride=1)
eval/hyper_iqa.py:273
↓ 4 callersFunctionformat_timestamp
(seconds: float, always_include_hours: bool = False, decimal_marker: str = '.')
latentsync/whisper/whisper/utils.py:30
↓ 4 callersFunctionread_video
(video_path: str, change_fps=True, use_decord=True)
latentsync/utils/util.py:46
↓ 4 callersFunctionsyncnet_eval
(syncnet, syncnet_detector, video_path, temp_dir, detect_results_dir="detect_results")
eval/eval_sync_conf.py:25
↓ 3 callersMethod__init__
(self, config, gradient_checkpointing=False)
latentsync/models/stable_syncnet.py:29
↓ 3 callersMethod_audio2feat
(self, audio_path: str)
latentsync/whisper/audio2feature.py:105
↓ 3 callersMethodaffine_transform
(self, image: torch.Tensor)
latentsync/utils/image_processor.py:54
↓ 3 callersMethodaudio2feat
(self, audio_path)
latentsync/whisper/audio2feature.py:120
↓ 3 callersFunctioncosine_loss
(vision_embeds, audio_embeds, y)
latentsync/utils/util.py:215
↓ 3 callersMethodforward
(self, x: Tensor)
latentsync/whisper/whisper/model.py:35
↓ 3 callersMethodforward_features
(self, x)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:499
↓ 3 callersFunctionget_sinusoid_encoding_table
Sinusoid position encoding table
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:357
↓ 3 callersMethodget_sliced_feature
Get sliced features based on a given index :param feature_array: :param start_idx: the start index of the feature :pa
latentsync/whisper/audio2feature.py:27
↓ 3 callersMethodloadParameters
(self, path)
eval/syncnet/syncnet_eval.py:212
↓ 3 callersFunctionload_fixed_mask
(resolution: int, mask_image_path="latentsync/utils/mask.png")
latentsync/utils/image_processor.py:26
↓ 3 callersFunctionmelspectrogram
(wav)
latentsync/utils/audio.py:59
↓ 3 callersMethodset_progress_bar_config
(self, **kwargs)
latentsync/pipelines/lipsync_pipeline.py:231
↓ 3 callersFunctionwrite_video
(video_output_path: str, video_frames: np.ndarray, fps: int)
latentsync/utils/util.py:115
↓ 3 callersFunctionzero_module
(module)
latentsync/models/utils.py:15
↓ 2 callersMethod__init__
( self, img_size=224, patch_size=16, encoder_in_chans=3, encoder_num_c
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:242
↓ 2 callersMethod__init__
( self, query_dim: int, cross_attention_dim: Optional[int] = None, heads: int
latentsync/models/attention.py:203
↓ 2 callersMethod__init__
(self, channels, use_conv=False, use_conv_transpose=False, out_channels=None, name="conv")
latentsync/models/resnet.py:33
↓ 2 callersFunction_amp_to_db
(x)
latentsync/utils/audio.py:130
↓ 2 callersFunction_normalize
(S)
latentsync/utils/audio.py:139
↓ 2 callersFunction_stft
(y)
latentsync/utils/audio.py:74
↓ 2 callersMethodadd_ckpt
(self, ckpt_path, line_name)
eval/draw_syncnet_lines.py:23
↓ 2 callersFunctionaffine_transform_multi_gpus
(input_dir, output_dir, temp_dir, resolution, num_workers)
preprocess/affine_transform.py:79
↓ 2 callersMethodaffine_transform_video
(self, video_path)
latentsync/utils/image_processor.py:107
↓ 2 callersMethodaffine_transform_video
(self, video_frames: np.ndarray)
latentsync/pipelines/lipsync_pipeline.py:252
↓ 2 callersMethodappend_dataset
(self, dataset_dir: str)
tools/write_fileslist.py:25
↓ 2 callersFunctionavailable_models
Returns the names of available models
latentsync/whisper/whisper/__init__.py:70
↓ 2 callersFunctioncli
()
latentsync/whisper/whisper/transcribe.py:131
↓ 2 callersFunctioncompute_our_fvd
(videos_fake: np.ndarray, videos_real: np.ndarray, device: str = "cuda")
eval/fvd.py:29
↓ 2 callersFunctioncompute_stats
(feats: np.ndarray)
eval/fvd.py:21
↓ 2 callersMethodconcat_heads
(self, tensor)
latentsync/models/attention.py:244
↓ 2 callersFunctioncount_video_time
(video_path)
latentsync/utils/util.py:262
↓ 2 callersFunctiondetect_shot_multiprocessing
(input_dir, output_dir, num_workers)
preprocess/detect_shot.py:47
↓ 2 callersMethoddetect_video
(self, video_path)
eval/eval_fvd.py:54
↓ 2 callersMethoddetect_videos
(self, videos_dir: str)
eval/eval_fvd.py:70
↓ 2 callersFunctionfilter_visual_quality_multi_gpus
(input_dir, output_dir, num_workers)
preprocess/filter_visual_quality.py:105
↓ 2 callersMethodforward
(self, x)
eval/detectors/s3fd/nets.py:21
↓ 2 callersMethodforward
(self, hidden_states)
latentsync/models/resnet.py:227
↓ 2 callersFunctionget_hop_size
()
latentsync/utils/audio.py:42
↓ 2 callersFunctionget_tokenizer
( multilingual: bool, *, task: Optional[str] = None, # Literal["transcribe", "translate", None]
latentsync/whisper/whisper/tokenizer.py:295
↓ 2 callersFunctioninit_dist
Initializes distributed environment.
latentsync/utils/util.py:138
↓ 2 callersFunctionload_model
Load a Whisper ASR model Parameters ---------- name : str one of the official model names listed by `whisper.available_model
latentsync/whisper/whisper/__init__.py:75
↓ 2 callersMethodlogits
Perform a forward pass on the decoder and return per-token logits
latentsync/whisper/whisper/decoding.py:119
↓ 2 callersFunctionmain
(config, args)
scripts/inference.py:27
↓ 2 callersFunctionnorm_cdf
(x)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:24
↓ 2 callersFunctionplot_loss_chart
(save_path: str, *args)
latentsync/utils/util.py:188
↓ 2 callersFunctionpreemphasis
(wav, k, preemphasize=True)
latentsync/utils/audio.py:30
↓ 2 callersMethodprepare_masks_and_masked_images
(self, images: Union[torch.Tensor, np.ndarray], affine_transform=False)
latentsync/utils/image_processor.py:82
↓ 2 callersMethodprocess_images
(self, images: Union[torch.Tensor, np.ndarray])
latentsync/utils/image_processor.py:93
↓ 2 callersMethodread_audio
(self, video_path: str)
latentsync/data/unet_dataset.py:59
↓ 2 callersMethodread_audio
(self, video_path: str)
latentsync/data/syncnet_dataset.py:53
↓ 2 callersFunctionremove_broken_videos_multiprocessing
(input_dir, num_workers)
preprocess/remove_broken_videos.py:30
↓ 2 callersFunctionresample_fps_hz_multiprocessing
(input_dir, output_dir, num_workers)
preprocess/resample_fps_hz.py:55
↓ 2 callersFunctionsegment_videos_multiprocessing
(input_dir, output_dir, num_workers)
preprocess/segment_videos.py:47
↓ 2 callersFunctionsync_av_multi_gpus
(input_dir, output_dir, temp_dir, num_workers, sync_conf_threshold)
preprocess/sync_av.py:80
↓ 2 callersFunctiontrunc_normal_
(tensor, mean=0., std=1.)
latentsync/trepa/third_party/VideoMAEv2/videomaev2_pretrain.py:23
↓ 2 callersFunctiontrunc_normal_
r"""Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distributi
latentsync/trepa/third_party/VideoMAEv2/videomaev2_finetune.py:59
↓ 2 callersFunctionzero_rank_log
(logger, message: str)
latentsync/utils/util.py:156
↓ 1 callersMethod__init__
(self, num_classes=2, top_k=750, nms_thresh=0.3, conf_thresh=0.05, var
eval/detectors/s3fd/box_utils.py:135
↓ 1 callersMethod__init__
(self, n_channels, scale)
eval/detectors/s3fd/nets.py:10
↓ 1 callersMethod__init__
(self, data_path: str, image_folder: bool, resolution: int, sequence_length: int, sample_ever
latentsync/trepa/utils/data_utils.py:91
↓ 1 callersMethod__init__
(self)
latentsync/whisper/whisper/normalizers/english.py:459
↓ 1 callersMethod__init__
(self, cin, cout, kernel_size, stride, padding, residual=False, act_fn="relu", *args, **kwargs)
latentsync/models/wav2lip_syncnet.py:72
↓ 1 callersFunction_build_mel_basis
()
latentsync/utils/audio.py:119
↓ 1 callersMethod_dataloader
Initializes and returns the dataloader.
latentsync/trepa/utils/data_utils.py:117
↓ 1 callersMethod_dataset
Initializes and return the dataset.
latentsync/trepa/utils/data_utils.py:103
↓ 1 callersMethod_detect_language
(self, audio_features: Tensor, tokens: Tensor)
latentsync/whisper/whisper/decoding.py:579
↓ 1 callersFunction_download
(url: str, root: str, in_memory: bool)
latentsync/whisper/whisper/__init__.py:33
↓ 1 callersMethod_get_audio_features
(self, mel: Tensor, include_embeddings: bool = False)
latentsync/whisper/whisper/decoding.py:557
↓ 1 callersMethod_get_initial_tokens
(self)
latentsync/whisper/whisper/decoding.py:512
↓ 1 callersMethod_get_suppress_tokens
(self)
latentsync/whisper/whisper/decoding.py:534
↓ 1 callersFunction_linear_to_mel
(spectogram)
latentsync/utils/audio.py:112
next →1–100 of 512, ranked by callers