MCPcopy Create free account

hub / github.com/TMElyralab/MuseTalk / functions

Functions394 in github.com/TMElyralab/MuseTalk

↓ 15 callersMethodencode
(self, text, **kwargs)
musetalk/whisper/whisper/tokenizer.py:137
↓ 11 callersMethodrun
(self, mel: Tensor)
musetalk/whisper/whisper/decoding.py:631
↓ 9 callersMethoddevice
(self)
musetalk/whisper/whisper/model.py:249
↓ 8 callersFunctionread_imgs
(img_list)
musetalk/utils/preprocessing.py:35
↓ 7 callersMethoddecode
(self, token_ids: Union[int, List[int], np.ndarray, torch.Tensor], **kwargs)
musetalk/whisper/whisper/tokenizer.py:140
↓ 6 callersMethod__init__
(self, resnet_path='models/resnet18-5c106cde.pth', n_classes=19, *args, **kwargs)
musetalk/utils/face_parsing/model.py:231
↓ 6 callersMethodsoftmax
(self, x, axis=1)
musetalk/utils/face_detection/api.py:118
↓ 5 callersMethod_get_single_token_id
(self, text)
musetalk/whisper/whisper/tokenizer.py:267
↓ 5 callersFunctionget_landmark_and_bbox
(img_list,upperbondrange =0)
musetalk/utils/preprocessing.py:84
↓ 5 callersMethodget_latents_for_unet
Prepare latent variables for a U-Net model. :param img: The image to process. :return: A concatenated tensor of latents for U
musetalk/models/vae.py:110
↓ 5 callersFunctionset_requires_grad
(net, requires_grad=False)
musetalk/loss/basic_loss.py:22
↓ 4 callersMethod__init__
(self, num_modules=1)
musetalk/utils/face_detection/models.py:147
↓ 4 callersMethod__init__
(self, dims: ModelDimensions)
musetalk/whisper/whisper/model.py:221
↓ 4 callersMethod__init__
(self, requires_grad=False)
musetalk/loss/vgg_face.py:133
↓ 4 callersMethod_make_layer
(self, block, planes, blocks, stride=1)
musetalk/utils/face_detection/models.py:229
↓ 4 callersMethod_make_layer
(self, block, planes, blocks, stride=1)
musetalk/loss/resnet.py:109
↓ 4 callersFunctioncalculate_landmarks_similarity
Calculate the similarity between sets of facial landmarks and return the indices of the most similar faces. Parameters: landmarks_list (
musetalk/data/sample_method.py:54
↓ 4 callersFunctioncreate_layer_basic
(in_chan, out_chan, bnum, stride=1)
musetalk/utils/face_parsing/resnet.py:51
↓ 4 callersMethoddecode_latents
Decode latent variables back into an image. :param latents: The latent variables to decode. :return: A NumPy array representi
musetalk/models/vae.py:96
↓ 4 callersFunctionformat_timestamp
(seconds: float, always_include_hours: bool = False, decimal_marker: str = '.')
musetalk/whisper/whisper/utils.py:30
↓ 4 callersMethodget_audio_feature
(self, wav_path, start_index=0, weight_dtype=None)
musetalk/utils/audio_processor.py:15
↓ 4 callersFunctionget_file_type
(video_path)
musetalk/utils/utils.py:33
↓ 4 callersFunctionget_ganloss_weight
Calculate GAN loss weight based on training step
train.py:162
↓ 4 callersFunctionget_image
将裁剪的面部图像粘贴回原始图像,并进行一些处理。 Args: image (numpy.ndarray): 原始图像(身体部分)。 face (numpy.ndarray): 裁剪的面部图像。 face_box (tuple): 面
musetalk/utils/blending.py:35
↓ 4 callersFunctiontransform
Generate and affine transformation matrix. Given a set of points, a center, a scale and a targer resolution, the function generates and affin
musetalk/utils/face_detection/utils.py:56
↓ 4 callersMethodupdate
Specify how to select the next token, based on the current trace and logits Parameters ---------- tokens : Tensor, shape = (n
musetalk/whisper/whisper/decoding.py:203
↓ 3 callersFunctionPortraitDataset
Return dataset based on configuration Args: cfg: Configuration dictionary Returns: Dataset: Combined dataset
musetalk/data/dataset.py:525
↓ 3 callersMethod__init__
(self, config)
musetalk/models/syncnet.py:19
↓ 3 callersMethod__init__
(self, in_features, out_features, norm=False, kernel_size=4, pool=False, sn=False)
musetalk/loss/discriminator.py:11
↓ 3 callersFunctionconv3x3
3x3 convolution with padding
musetalk/utils/face_detection/models.py:7
↓ 3 callersFunctiondatagen
( whisper_chunks, vae_encode_latents, batch_size=8, delay_frame=0, device="cuda:0", )
musetalk/utils/utils.py:49
↓ 3 callersFunctionfast_check_ffmpeg
()
app.py:175
↓ 3 callersMethodforward
(self, x: Tensor)
musetalk/whisper/whisper/model.py:35
↓ 3 callersMethodget_detections_for_batch
(self, images)
musetalk/utils/face_detection/api.py:71
↓ 3 callersMethodget_whisper_chunk
( self, whisper_input_features, device, weight_dtype, whisper,
musetalk/utils/audio_processor.py:37
↓ 3 callersFunctionload_all_model
( unet_model_path=os.path.join("models", "musetalkV15", "unet.pth"), vae_type="sd-vae", unet_confi
musetalk/utils/utils.py:15
↓ 3 callersFunctionosmakedirs
(path_list)
scripts/realtime_inference.py:51
↓ 3 callersMethodprepare_material
(self)
scripts/realtime_inference.py:148
↓ 2 callersMethod__init__
(self, cfg, list_paths, root_path='./dataset/',
musetalk/data/dataset.py:31
↓ 2 callersMethod__init__
(self, cin, cout, kernel_size, stride, padding, residual=False, *args, **kwargs)
musetalk/loss/conv.py:6
↓ 2 callersMethod__init__
(self, block, layers, num_classes=1000, include_top=True)
musetalk/loss/resnet.py:84
↓ 2 callersFunction_amp_to_db
(x)
musetalk/data/audio.py:135
↓ 2 callersFunction_normalize
(S)
musetalk/data/audio.py:142
↓ 2 callersFunction_stft
(y)
musetalk/data/audio.py:89
↓ 2 callersMethodapply
Apply any filtering or masking to logits in-place Parameters ---------- logits : Tensor, shape = (n_batch, vocab_size)
musetalk/whisper/whisper/decoding.py:372
↓ 2 callersFunctionavailable_models
Returns the names of available models
musetalk/whisper/whisper/__init__.py:66
↓ 2 callersFunctioncli
()
musetalk/whisper/whisper/transcribe.py:131
↓ 2 callersFunctionconv3x3
3x3 convolution with padding
musetalk/utils/face_parsing/resnet.py:14
↓ 2 callersFunctionconv3x3
3x3 convolution with padding
musetalk/loss/resnet.py:6
↓ 2 callersMethodcrop_resize_img
Crop and resize image Args: img: Input image bbox: Bounding box crop_type: Type of cropping
musetalk/data/dataset.py:131
↓ 2 callersFunctiondetect
(net, img, device)
musetalk/utils/face_detection/detection/sfd/detect.py:19
↓ 2 callersMethodencode_latents
Encode an image into latent variables. :param image: The image tensor to encode. :return: The encoded latent variables.
musetalk/models/vae.py:84
↓ 2 callersFunctionensure_wav
Convert any audio (mp3/ogg/m4a/wav/…) to 16kHz mono PCM WAV via ffmpeg. Returns path to the converted .wav (original if already correct).
musetalk/utils/audio_utils.py:3
↓ 2 callersFunctionface_seg
对图像进行面部解析,生成面部区域的掩码。 Args: image (PIL.Image): 输入图像。 Returns: PIL.Image: 面部区域的掩码图像。
musetalk/utils/blending.py:16
↓ 2 callersFunctionfast_check_ffmpeg
()
scripts/realtime_inference.py:29
↓ 2 callersFunctionfast_check_ffmpeg
()
scripts/preprocess.py:17
↓ 2 callersFunctionfast_check_ffmpeg
()
scripts/inference.py:23
↓ 2 callersMethodgenerate_random_value
Generate random value Returns: float: Generated random value
musetalk/data/dataset.py:88
↓ 2 callersFunctionget_crop_box
(box, expand)
musetalk/utils/blending.py:7
↓ 2 callersFunctionget_hop_size
()
musetalk/data/audio.py:62
↓ 2 callersFunctionget_image_pred
(pixel_values, ref_pixel_values, audio_prompts, vae,
musetalk/utils/utils.py:140
↓ 2 callersMethodget_sliced_feature
Get sliced features based on a given index :param feature_array: :param start_idx: the start index of the feature :p
musetalk/whisper/audio2feature.py:16
↓ 2 callersFunctionget_tokenizer
( multilingual: bool, *, task: Optional[str] = None, # Literal["transcribe", "translate", None]
musetalk/whisper/whisper/tokenizer.py:295
↓ 2 callersFunctionget_video_fps
(video_path)
musetalk/utils/utils.py:43
↓ 2 callersMethodget_zero_tensor
(self, input)
musetalk/loss/discriminator.py:111
↓ 2 callersFunctionload_model
Load a Whisper ASR model Parameters ---------- name : str one of the official model names listed by `whisper.available_model
musetalk/whisper/whisper/__init__.py:71
↓ 2 callersMethodlogits
Perform a forward pass on the decoder and return per-token logits
musetalk/whisper/whisper/decoding.py:119
↓ 2 callersFunctionnms
(dets, thresh)
musetalk/utils/face_detection/detection/sfd/bbox.py:44
↓ 2 callersFunctionpreemphasis
(wav, k, preemphasize=True)
musetalk/data/audio.py:52
↓ 2 callersMethodpreprocess
(self, s: str)
musetalk/whisper/whisper/normalizers/english.py:381
↓ 2 callersMethodpreprocess_img
Preprocess an image for the VAE. :param img_name: The image file path or a list of image file paths. :param half_mask: Wheth
musetalk/models/vae.py:51
↓ 2 callersFunctionprocess_audio_features
(cfg, batch, wav2vec, bsz, num_frames, weight_dtype)
musetalk/utils/utils.py:177
↓ 2 callersFunctionresize_landmark
(landmark, w, h, new_w, new_h)
musetalk/data/sample_method.py:141
↓ 2 callersFunctionsave_list_to_file
Save a list of strings to a file, each string on a new line. Parameters: file_path (str): The path to the file where the list will be sa
scripts/preprocess.py:177
↓ 2 callersFunctionsave_models
(accelerator, net, save_dir, global_step, cfg, logger=None)
musetalk/utils/utils.py:234
↓ 1 callersMethod__init__
(self)
musetalk/utils/face_detection/detection/sfd/net_s3fd.py:23
↓ 1 callersMethod__init__
(self, model_path)
musetalk/utils/face_parsing/resnet.py:59
↓ 1 callersMethod__init__
(self)
musetalk/whisper/whisper/normalizers/english.py:459
↓ 1 callersMethod__init__
(self, unet_config, model_path, use_float16=False,
musetalk/models/unet.py:30
↓ 1 callersFunction_build_mel_basis
()
musetalk/data/audio.py:130
↓ 1 callersMethod_create_cheek_mask
Create cheek area mask (1/4 area on both sides)
musetalk/utils/face_parsing/__init__.py:51
↓ 1 callersMethod_detect_language
(self, audio_features: Tensor, tokens: Tensor)
musetalk/whisper/whisper/decoding.py:579
↓ 1 callersFunction_download
(url: str, root: str, in_memory: bool)
musetalk/whisper/whisper/__init__.py:33
↓ 1 callersMethod_forward
(self, level, inp)
musetalk/utils/face_detection/models.py:119
↓ 1 callersFunction_gaussian
( size=3, sigma=0.25, amplitude=1, normalize=False, width=None, height=None, sigma_horz=None,
musetalk/utils/face_detection/utils.py:11
↓ 1 callersMethod_generate_network
(self, level)
musetalk/utils/face_detection/models.py:107
↓ 1 callersMethod_get_audio_features
(self, mel: Tensor, include_embeddings: bool = False)
musetalk/whisper/whisper/decoding.py:557
↓ 1 callersMethod_get_initial_tokens
(self)
musetalk/whisper/whisper/decoding.py:512
↓ 1 callersMethod_get_suppress_tokens
(self)
musetalk/whisper/whisper/decoding.py:534
↓ 1 callersFunction_linear_to_mel
(spectogram)
musetalk/data/audio.py:124
↓ 1 callersFunction_lws_processor
()
musetalk/data/audio.py:85
↓ 1 callersMethod_main_loop
(self, audio_features: Tensor, tokens: Tensor)
musetalk/whisper/whisper/decoding.py:591
↓ 1 callersMethod_verify_options
(self, options: DecodingOptions)
musetalk/whisper/whisper/decoding.py:499
↓ 1 callersFunctionadd_segment
( *, start: float, end: float, encoder_embeddings )
musetalk/whisper/whisper/transcribe.py:88
↓ 1 callersFunctionanalyze_video
Convert video files to a specified format and save them to the destination path. Parameters: org_path (str): The directory containing th
scripts/preprocess.py:207
↓ 1 callersMethodaudio2feat
(self,audio_path)
musetalk/whisper/audio2feature.py:97
↓ 1 callersFunctionbatch_decode
Decode locations from predictions using priors to undo the encoding we did for offset regression at train time. Args: loc (tensor): lo
musetalk/utils/face_detection/detection/sfd/bbox.py:111
↓ 1 callersFunctionbatch_detect
(net, imgs, device)
musetalk/utils/face_detection/detection/sfd/detect.py:58
↓ 1 callersFunctionbuild_tokenizer
(name: str = "gpt2")
musetalk/whisper/whisper/tokenizer.py:274
↓ 1 callersMethodcleanup_caching
Clean up any resources or hooks after decoding is finished
musetalk/whisper/whisper/decoding.py:127
next →1–100 of 394, ranked by callers