Functions — github.com/antgroup/echomimic

hub / github.com/antgroup/echomimic / functions

Functions352 in github.com/antgroup/echomimic

⨍ Functions 352◇ Types & classes 83

↓ 15 callersMethodencode

(self, text, **kwargs)

src/models/whisper/whisper/tokenizer.py:137

↓ 14 callersMethoddevice

(self)

src/models/whisper/whisper/model.py:249

↓ 12 callersFunctiontorch_dfs

(model: torch.nn.Module)

src/models/mutual_self_attention.py:12

↓ 10 callersMethodfrom_pretrained_2d

( cls, pretrained_model_path: PathLike, motion_module_path: PathLike, subfolde

src/models/unet_3d.py:578

↓ 9 callersMethoddecode

(self, token_ids: Union[int, List[int], np.ndarray, torch.Tensor], **kwargs)

src/models/whisper/whisper/tokenizer.py:140

↓ 8 callersMethodclear

(self)

src/models/mutual_self_attention.py:307

↓ 8 callersFunctioncrop_and_pad

(image, rect)

src/utils/util.py:131

↓ 8 callersMethodupdate

Specify how to select the next token, based on the current trace and logits Parameters ---------- tokens : Tensor, shape = (n

src/models/whisper/whisper/decoding.py:203

↓ 7 callersMethoddetect

Performs face landmarks detection on the given image. Only use this method when the FaceLandmarker is created with the image running mode.

src/utils/face_landmark.py:3173

↓ 6 callersMethod__init__

( self, in_channels: int, prev_output_channel: int, out_channels: int,

src/models/unet_2d_blocks.py:963

↓ 6 callersFunctionsave_videos_grid

(videos: torch.Tensor, path: str, audio_path=None, rescale=False, n_rows=6, fps=8)

src/utils/util.py:86

↓ 5 callersMethod_get_single_token_id

(self, text)

src/models/whisper/whisper/tokenizer.py:267

↓ 5 callersMethodaudio2feat

(self,audio_path)

src/models/whisper/audio2feature.py:98

↓ 5 callersFunctionget_motion_module

(in_channels, motion_module_type: str, motion_module_kwargs: dict)

src/models/motion_module.py:34

↓ 5 callersFunctionload_audio_model

(model_path, device)

src/models/whisper/audio2feature.py:113

↓ 5 callersMethodpreprocess

(self, s: str)

src/models/whisper/whisper/normalizers/english.py:381

↓ 4 callersMethod__init__

( self, in_channels: int, prev_output_channel: int, out_channels: int,

src/models/unet_3d_blocks.py:758

↓ 4 callersMethod__init__

(self, d_model, dropout=0.0, max_len=24)

src/models/motion_module.py:263

↓ 4 callersMethod__init__

(self, dims: ModelDimensions)

src/models/whisper/whisper/model.py:221

↓ 4 callersMethoddraw_landmarks

(self, image_size, keypoints, normed=False)

src/utils/draw_utils.py:159

↓ 4 callersMethodfeature2chunks

(self,feature_array,fps,audio_feat_length = [2,2])

src/models/whisper/audio2feature.py:82

↓ 4 callersFunctionformat_timestamp

(seconds: float, always_include_hours: bool = False, decimal_marker: str = '.')

src/models/whisper/whisper/utils.py:30

↓ 4 callersFunctionget_context_scheduler

(name: str)

src/pipelines/context.py:45

↓ 4 callersFunctionget_tensor_interpolation_method

()

src/pipelines/utils.py:6

↓ 4 callersMethodrun

(self, mel: Tensor)

src/models/whisper/whisper/decoding.py:631

↓ 4 callersMethodupdate

(self, writer, do_classifier_free_guidance=False, dtype=torch.float16)

src/models/mutual_self_attention.py:266

↓ 3 callersFunctioncenter_crop_cv2

(cv2_pic)

src/utils/img_utils.py:15

↓ 3 callersMethodcreate_from_options

Creates the `FaceLandmarker` object from face landmarker options. Args: options: Options for the face landmarker task. Returns:

src/utils/face_landmark.py:3099

↓ 3 callersMethodforward

(self, x: Tensor)

src/models/whisper/whisper/model.py:35

↓ 3 callersFunctionget_alpha

(alphas_cumprod, timestep)

src/utils/step_func.py:4

↓ 3 callersFunctionget_down_block

( down_block_type: str, num_layers: int, in_channels: int, out_channels: int, temb_channel

src/models/unet_2d_blocks.py:20

↓ 3 callersFunctionget_up_block

( up_block_type: str, num_layers: int, in_channels: int, out_channels: int, prev_output_ch

src/models/unet_2d_blocks.py:102

↓ 3 callersFunctionpil_to_cv2

(pil)

src/utils/img_utils.py:9

↓ 3 callersFunctionproject_vertices_from_ref2tgt

(ref_lmks3d, tgt_trans_mat)

src/utils/motion_utils.py:82

↓ 3 callersMethodto_pb2

Generates an FaceLandmarkerGraphOptions protobuf object.

src/utils/face_landmark.py:3040

↓ 2 callersMethod__init__

( self, channels, use_conv=False, use_conv_transpose=False, out_channe

src/models/resnet.py:32

↓ 2 callersFunction_build_landmarker_result2

Constructs a `FaceLandmarkerResult` from output packets.

src/utils/face_landmark.py:2934

↓ 2 callersMethodapply

Apply any filtering or masking to logits in-place Parameters ---------- logits : Tensor, shape = (n_batch, vocab_size)

src/models/whisper/whisper/decoding.py:372

↓ 2 callersFunctionavailable_models

Returns the names of available models

src/models/whisper/whisper/__init__.py:66

↓ 2 callersFunctioncli

()

src/models/whisper/whisper/transcribe.py:131

↓ 2 callersMethoddraw_points

We have a custom function to draw the pupils because the mp.draw_landmarks method requires a parameter for all landmarks. Until our PR is mer

src/utils/draw_utils.py:132

↓ 2 callersMethodforward

(self, hidden_states)

src/models/resnet.py:251

↓ 2 callersMethodget_sliced_feature

Get sliced features based on a given index :param feature_array: :param start_idx: the start index of the feature :p

src/models/whisper/audio2feature.py:16

↓ 2 callersFunctionget_tokenizer

( multilingual: bool, *, task: Optional[str] = None, # Literal["transcribe", "translate", None]

src/models/whisper/whisper/tokenizer.py:295

↓ 2 callersFunctionload_model

Load a Whisper ASR model Parameters ---------- name : str one of the official model names listed by `whisper.available_model

src/models/whisper/whisper/__init__.py:71

↓ 2 callersMethodlogits

Perform a forward pass on the decoder and return per-token logits

src/models/whisper/whisper/decoding.py:119

↓ 2 callersFunctionmotion_sync

(sequence_driver_det, reference_det, per_landmark_align=True)

src/utils/motion_utils.py:135

↓ 2 callersFunctionordered_halving

(val)

src/pipelines/context.py:7

↓ 2 callersFunctionorigin_by_velocity_and_sample

(velocity, noisy_images, alphas_cumprod, timestep)

src/utils/step_func.py:29

↓ 2 callersFunctionpils_from_video

(video_path)

src/utils/img_utils.py:22

↓ 2 callersFunctionpsuedo_velocity_wrt_noisy_and_timestep

(noisy_images, noisy_images_pre, alphas_cumprod, timestep, timestep_prev)

src/utils/step_func.py:10

↓ 2 callersFunctionzero_module

(module)

src/models/motion_module.py:15

↓ 1 callersMethod__init__

( self, dim: int, num_attention_heads: int, attention_head_dim: int, d

src/models/attention.py:47

↓ 1 callersMethod__init__

(self)

src/models/whisper/whisper/normalizers/english.py:459

↓ 1 callersFunction_build_landmarker_result

Constructs a `FaceLandmarkerResult` from output packets.

src/utils/face_landmark.py:2873

↓ 1 callersMethod_detect_language

(self, audio_features: Tensor, tokens: Tensor)

src/models/whisper/whisper/decoding.py:579

↓ 1 callersFunction_download

(url: str, root: str, in_memory: bool)

src/models/whisper/whisper/__init__.py:33

↓ 1 callersMethod_get_audio_features

(self, mel: Tensor, include_embeddings: bool = False)

src/models/whisper/whisper/decoding.py:557

↓ 1 callersMethod_get_initial_tokens

(self)

src/models/whisper/whisper/decoding.py:512

↓ 1 callersMethod_get_suppress_tokens

(self)

src/models/whisper/whisper/decoding.py:534

↓ 1 callersMethod_main_loop

(self, audio_features: Tensor, tokens: Tensor)

src/models/whisper/whisper/decoding.py:591

↓ 1 callersMethod_verify_options

(self, options: DecodingOptions)

src/models/whisper/whisper/decoding.py:499

↓ 1 callersFunctionadd_segment

( *, start: float, end: float, encoder_embeddings )

src/models/whisper/whisper/transcribe.py:88

↓ 1 callersFunctionbuild_tokenizer

(name: str = "gpt2")

src/models/whisper/whisper/tokenizer.py:274

↓ 1 callersMethodcleanup_caching

Clean up any resources or hooks after decoding is finished

src/models/whisper/whisper/decoding.py:127

↓ 1 callersFunctioncompression_ratio

(text)

src/models/whisper/whisper/utils.py:26

↓ 1 callersFunctioncreate_perspective_matrix

(aspect_ratio)

src/utils/motion_utils.py:43

↓ 1 callersFunctioncv2_to_pil

(cv2_img)

src/utils/img_utils.py:12

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echo_mimic_pose.py:108

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echo_mimic.py:115

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echo_mimic_acc.py:116

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echo_mimic_pose_acc.py:109

↓ 1 callersMethoddetect_for_video

Performs face landmarks detection on the provided video frame. Only use this method when the FaceLandmarker is created with the video running

src/utils/face_landmark.py:3214

↓ 1 callersFunctionexact_div

(x, y)

src/models/whisper/whisper/utils.py:5

↓ 1 callersMethodfinalize

Finalize search and return the final candidate sequences Parameters ---------- tokens : Tensor, shape = (n_audio, n_group, cu

src/models/whisper/whisper/decoding.py:228

↓ 1 callersMethodinstall_kv_cache_hooks

The `MultiHeadAttention` module optionally accepts `kv_cache` which stores the key and value tensors calculated for the previous posi

src/models/whisper/whisper/model.py:256

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echo_mimic_pose.py:331

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echo_mimic.py:292

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echo_mimic_acc.py:293

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echo_mimic_pose_acc.py:325

↓ 1 callersFunctionload_audio

Open an audio file and read as mono waveform, resampling as necessary Parameters ---------- file: str The audio file to open

src/models/whisper/whisper/audio.py:22

↓ 1 callersFunctionlog_mel_spectrogram

Compute the log-Mel spectrogram of Parameters ---------- audio: Union[str, np.ndarray, torch.Tensor], shape = (*) The path t

src/models/whisper/whisper/audio.py:92

↓ 1 callersFunctionmain

()

infer_audio2vid_pose_acc.py:78

↓ 1 callersFunctionmain

()

infer_audio2vid_pose.py:72

↓ 1 callersFunctionmain

()

infer_audio2vid.py:91

↓ 1 callersFunctionmain

()

infer_audio2vid_acc.py:91

↓ 1 callersFunctionmel_filters

load the mel filterbank matrix for projecting STFT into a Mel spectrogram. Allows decoupling librosa dependency; saved using: np.sav

src/models/whisper/whisper/audio.py:77

↓ 1 callersMethodoutput

(result: Union[str, int])

src/models/whisper/whisper/normalizers/english.py:171

↓ 1 callersFunctionpad_or_trim

Pad or trim the audio array to N_SAMPLES, as expected by the encoder.

src/models/whisper/whisper/audio.py:52

↓ 1 callersFunctionparse_args

()

infer_audio2vid_pose_acc.py:38

↓ 1 callersFunctionparse_args

()

infer_audio2vid_pose.py:34

↓ 1 callersFunctionparse_args

()

infer_audio2vid.py:51

↓ 1 callersFunctionparse_args

()

infer_audio2vid_acc.py:51

↓ 1 callersMethodpostprocess

(self, s: str)

src/models/whisper/whisper/normalizers/english.py:410

↓ 1 callersMethodprepare_extra_step_kwargs

(self, generator, eta)

src/pipelines/pipeline_echo_mimic_pose.py:123

↓ 1 callersMethodprepare_extra_step_kwargs

(self, generator, eta)

src/pipelines/pipeline_echo_mimic.py:129

↓ 1 callersMethodprepare_extra_step_kwargs

(self, generator, eta)

src/pipelines/pipeline_echo_mimic_acc.py:130

↓ 1 callersMethodprepare_extra_step_kwargs

(self, generator, eta)

src/pipelines/pipeline_echo_mimic_pose_acc.py:124

↓ 1 callersMethodprepare_latents

( self, batch_size, num_channels_latents, width, height, video

src/pipelines/pipeline_echo_mimic_pose.py:180

↓ 1 callersMethodprepare_latents

( self, batch_size, num_channels_latents, width, height, video

src/pipelines/pipeline_echo_mimic.py:150

next →1–100 of 352, ranked by callers