Functions — github.com/antgroup/echomimic

hub / github.com/antgroup/echomimic_v2 / functions

Functions395 in github.com/antgroup/echomimic_v2

⨍ Functions 395◇ Types & classes 84

↓ 13 callersMethodencode

(self, text, **kwargs)

src/models/whisper/whisper/tokenizer.py:137

↓ 12 callersFunctiontorch_dfs

(model: torch.nn.Module)

src/models/mutual_self_attention.py:12

↓ 9 callersMethoddevice

(self)

src/models/whisper/whisper/model.py:249

↓ 8 callersMethodupdate

Specify how to select the next token, based on the current trace and logits Parameters ---------- tokens : Tensor, shape = (n

src/models/whisper/whisper/decoding.py:203

↓ 7 callersMethoddecode

(self, token_ids: Union[int, List[int], np.ndarray, torch.Tensor], **kwargs)

src/models/whisper/whisper/tokenizer.py:140

↓ 6 callersMethod__init__

( self, in_channels: int, prev_output_channel: int, out_channels: int,

src/models/unet_2d_blocks.py:964

↓ 6 callersFunctionget_alpha

(alphas_cumprod, timestep)

src/pipelines/myschedulers.py:18

↓ 6 callersMethodrun

(self, mel: Tensor)

src/models/whisper/whisper/decoding.py:631

↓ 5 callersMethod__init__

(self, d_model, dropout=0.0, max_len=24)

src/models/motion_module.py:264

↓ 5 callersMethod_get_single_token_id

(self, text)

src/models/whisper/whisper/tokenizer.py:267

↓ 5 callersMethodfrom_pretrained_2d

( cls, pretrained_model_path: PathLike, motion_module_path: PathLike, subfolde

src/models/unet_3d.py:578

↓ 5 callersFunctionget_motion_module

(in_channels, motion_module_type: str, motion_module_kwargs: dict)

src/models/motion_module.py:34

↓ 4 callersMethod__init__

( self, in_channels: int, prev_output_channel: int, out_channels: int,

src/models/unet_3d_blocks.py:758

↓ 4 callersMethod__init__

(self, dims: ModelDimensions)

src/models/whisper/whisper/model.py:221

↓ 4 callersFunctionalpha_blend_color

blend color according to point conf

src/utils/dwpose_util.py:9

↓ 4 callersMethodclear

(self)

src/models/mutual_self_attention.py:315

↓ 4 callersFunctiondraw_pose_select_v2

vis dwpose outputs Args: pose (List): DWposeDetector outputs in dwpose_detector.py H (int): height W (int): width

src/utils/dwpose_util.py:147

↓ 4 callersFunctionformat_timestamp

(seconds: float, always_include_hours: bool = False, decimal_marker: str = '.')

src/models/whisper/whisper/utils.py:30

↓ 4 callersFunctionload_audio_model

(model_path, device)

src/models/whisper/audio2feature.py:113

↓ 4 callersFunctionsave_videos_grid

(videos: torch.Tensor, path: str, audio_path=None, rescale=False, n_rows=6, fps=8)

src/utils/util.py:86

↓ 3 callersMethodaudio2feat

(self,audio_path)

src/models/whisper/audio2feature.py:98

↓ 3 callersFunctiondraw_pose

vis dwpose outputs Args: pose (List): DWposeDetector outputs in dwpose_detector.py H (int): height W (int): width

src/models/dwpose/util.py:137

↓ 3 callersMethodforward

(self, x: Tensor)

src/models/whisper/whisper/model.py:35

↓ 3 callersFunctionget_alpha

(alphas_cumprod, timestep)

src/pipelines/step_func.py:4

↓ 3 callersFunctionget_down_block

( down_block_type: str, num_layers: int, in_channels: int, out_channels: int, temb_ch

src/models/unet_2d_blocks.py:21

↓ 3 callersFunctionget_up_block

( up_block_type: str, num_layers: int, in_channels: int, out_channels: int, prev_outp

src/models/unet_2d_blocks.py:103

↓ 3 callersMethodpreprocess

(self, s: str)

src/models/whisper/whisper/normalizers/english.py:381

↓ 3 callersFunctionproject_vertices_from_ref2tgt

(ref_lmks3d, tgt_trans_mat)

src/utils/motion_utils.py:82

↓ 3 callersMethodrelease_memory

(self)

src/models/dwpose/dwpose_detector.py:25

↓ 3 callersFunctionzero_module

(module)

src/models/motion_module.py:15

↓ 2 callersMethod__init__

(self, *args, **kwargs)

src/utils/util_emo.py:160

↓ 2 callersMethod__init__

( self, channels, use_conv=False, use_conv_transpose=False, out_channe

src/models/resnet.py:32

↓ 2 callersFunction_get_3rd_point

To calculate the affine matrix, three pairs of points are required. This function is used to get the 3rd point, given 2D points a & b. The 3r

src/models/dwpose/onnxpose.py:182

↓ 2 callersFunctionalpha_blend_color

blend color according to point conf

src/models/dwpose/util.py:9

↓ 2 callersMethodapply

Apply any filtering or masking to logits in-place Parameters ---------- logits : Tensor, shape = (n_batch, vocab_size)

src/models/whisper/whisper/decoding.py:372

↓ 2 callersFunctionavailable_models

Returns the names of available models

src/models/whisper/whisper/__init__.py:66

↓ 2 callersFunctioncli

()

src/models/whisper/whisper/transcribe.py:131

↓ 2 callersMethoddraw_points

We have a custom function to draw the pupils because the mp.draw_landmarks method requires a parameter for all landmarks. Until our PR is mer

src/utils/draw_utils.py:132

↓ 2 callersMethodfeature2chunks

(self,feature_array,fps,audio_feat_length = [2,2])

src/models/whisper/audio2feature.py:82

↓ 2 callersMethodforward

(self, hidden_states)

src/models/resnet.py:251

↓ 2 callersFunctionget_context_scheduler

(name: str)

src/pipelines/context.py:45

↓ 2 callersFunctionget_sigma

(sigmas, timestep)

src/pipelines/myschedulers.py:128

↓ 2 callersMethodget_sliced_feature

Get sliced features based on a given index :param feature_array: :param start_idx: the start index of the feature :p

src/models/whisper/audio2feature.py:16

↓ 2 callersFunctionget_tensor_interpolation_method

()

src/pipelines/utils.py:6

↓ 2 callersFunctionget_tokenizer

( multilingual: bool, *, task: Optional[str] = None, # Literal["transcribe", "translate", None]

src/models/whisper/whisper/tokenizer.py:295

↓ 2 callersFunctionload_model

Load a Whisper ASR model Parameters ---------- name : str one of the official model names listed by `whisper.available_model

src/models/whisper/whisper/__init__.py:71

↓ 2 callersMethodlogits

Perform a forward pass on the decoder and return per-token logits

src/models/whisper/whisper/decoding.py:119

↓ 2 callersFunctionordered_halving

(val)

src/pipelines/context.py:7

↓ 2 callersFunctionsave_video_from_cv2_list

(pic_cv2_list, output_path, fps=30.0, rgb2bgr=False)

src/utils/img_utils.py:59

↓ 2 callersMethodupdate

(self, writer, do_classifier_free_guidance=False, dtype=torch.float16)

src/models/mutual_self_attention.py:274

↓ 1 callersMethod__init__

(self, ddpm_or_ddim_scheduler, normnoise=False)

src/pipelines/myschedulers.py:27

↓ 1 callersMethod__init__

(self, model_path)

src/models/wav_clip.py:18

↓ 1 callersMethod__init__

( self, dim: int, num_attention_heads: int, attention_head_dim: int, d

src/models/attention.py:47

↓ 1 callersMethod__init__

(self)

src/models/whisper/whisper/normalizers/english.py:459

↓ 1 callersMethod_detect_language

(self, audio_features: Tensor, tokens: Tensor)

src/models/whisper/whisper/decoding.py:579

↓ 1 callersFunction_download

(url: str, root: str, in_memory: bool)

src/models/whisper/whisper/__init__.py:33

↓ 1 callersFunction_fix_aspect_ratio

Extend the scale to match the given aspect ratio. Args: scale (np.ndarray): The image scale (w, h) in shape (2, ) aspect_ratio (f

src/models/dwpose/onnxpose.py:149

↓ 1 callersMethod_get_audio_features

(self, mel: Tensor, include_embeddings: bool = False)

src/models/whisper/whisper/decoding.py:557

↓ 1 callersMethod_get_initial_tokens

(self)

src/models/whisper/whisper/decoding.py:512

↓ 1 callersMethod_get_suppress_tokens

(self)

src/models/whisper/whisper/decoding.py:534

↓ 1 callersMethod_main_loop

(self, audio_features: Tensor, tokens: Tensor)

src/models/whisper/whisper/decoding.py:591

↓ 1 callersFunction_rotate_point

Rotate a point by an angle. Args: pt (np.ndarray): 2D point coordinates (x, y) in shape (2, ) angle_rad (float): rotation angle i

src/models/dwpose/onnxpose.py:167

↓ 1 callersMethod_verify_options

(self, options: DecodingOptions)

src/models/whisper/whisper/decoding.py:499

↓ 1 callersFunctionadd_segment

( *, start: float, end: float, encoder_embeddings )

src/models/whisper/whisper/transcribe.py:88

↓ 1 callersFunctionbbox_xyxy2cs

Transform the bbox format from (x,y,w,h) into (center, scale) Args: bbox (ndarray): Bounding box(es) in shape (4,) or (n, 4), formatted

src/models/dwpose/onnxpose.py:115

↓ 1 callersFunctionbuild_tokenizer

(name: str = "gpt2")

src/models/whisper/whisper/tokenizer.py:274

↓ 1 callersFunctioncenter_crop_cv2

(cv2_pic)

src/utils/img_utils.py:15

↓ 1 callersMethodcleanup_caching

Clean up any resources or hooks after decoding is finished

src/models/whisper/whisper/decoding.py:127

↓ 1 callersFunctioncompression_ratio

(text)

src/models/whisper/whisper/utils.py:26

↓ 1 callersFunctionconvert_fps

(src_path, tgt_path, tgt_fps=24, tgt_sr=16000)

EMTD_dataset/preprocess.py:34

↓ 1 callersFunctioncreate_perspective_matrix

(aspect_ratio)

src/utils/motion_utils.py:43

↓ 1 callersFunctioncv2_to_pil

(cv2_img)

src/utils/img_utils.py:12

↓ 1 callersFunctiondecode

Modulate simcc distribution with Gaussian. Args: simcc_x (np.ndarray[K, Wx]): model predicted simcc in x. simcc_y (np.ndarray[K,

src/models/dwpose/onnxpose.py:333

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echomimicv2.py:105

↓ 1 callersMethoddecode_latents

(self, latents)

src/pipelines/pipeline_echomimicv2_acc.py:105

↓ 1 callersFunctiondemo_postprocess

(outputs, img_size, p6=False)

src/models/dwpose/onnxdet.py:76

↓ 1 callersFunctiondownload_youtube_video

:param video_url: youtube video url :param output_dir: file path to save

EMTD_dataset/download.py:11

↓ 1 callersFunctiondraw_bodypose

(canvas, candidate, subset, score, th=0.3)

src/models/dwpose/util.py:14

↓ 1 callersFunctiondraw_facepose

(canvas, all_lmks, all_scores, th=0.3)

src/models/dwpose/util.py:88

↓ 1 callersFunctiondraw_handpose

(canvas, all_hand_peaks, all_hand_scores)

src/utils/dwpose_util.py:105

↓ 1 callersFunctiondraw_handpose

(canvas, all_hand_peaks, all_hand_scores)

src/models/dwpose/util.py:59

↓ 1 callersMethoddraw_landmarks

(self, image_size, keypoints, normed=False)

src/utils/draw_utils.py:159

↓ 1 callersFunctiondraw_pose_video

(pose_params_path, save_path, max_size, ori_frames=None)

EMTD_dataset/preprocess.py:265

↓ 1 callersFunctionexact_div

(x, y)

src/models/whisper/whisper/utils.py:5

↓ 1 callersMethodfinalize

Finalize search and return the final candidate sequences Parameters ---------- tokens : Tensor, shape = (n_audio, n_group, cu

src/models/whisper/whisper/decoding.py:228

↓ 1 callersFunctionget_pose_params

(detected_poses, max_size)

EMTD_dataset/preprocess.py:107

↓ 1 callersFunctionget_simcc_maximum

Get maximum response location and value from simcc representations. Note: instance number: N num_keypoints: K heatmap hei

src/models/dwpose/onnxpose.py:288

↓ 1 callersFunctionget_video_pose

( video_path: str, sample_stride: int=1, max_frame=None)

EMTD_dataset/preprocess.py:44

↓ 1 callersFunctionget_warp_matrix

Calculate the affine transformation matrix that can warp the bbox area in the input image to the output size. Args: center (np.ndarra

src/models/dwpose/onnxpose.py:201

↓ 1 callersFunctionimg_resize

(input_img, imh=None, imw=None, max_val=512)

src/utils/util_emo.py:92

↓ 1 callersFunctioninference

Inference RTMPose model. Args: sess (ort.InferenceSession): ONNXRuntime session. img (np.ndarray): Input image in shape. Ret

src/models/dwpose/onnxpose.py:52

↓ 1 callersFunctioninference_detector

run human detect

src/models/dwpose/onnxdet.py:116

↓ 1 callersFunctioninference_pose

run pose detect Args: session (ort.InferenceSession): ONNXRuntime session. out_bbox (np.ndarray): bbox list oriImg (np.n

src/models/dwpose/onnxpose.py:353

↓ 1 callersMethodinstall_kv_cache_hooks

The `MultiHeadAttention` module optionally accepts `kv_cache` which stores the key and value tensors calculated for the previous posi

src/models/whisper/whisper/model.py:256

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echomimicv2.py:363

↓ 1 callersMethodinterpolate_latents

( self, latents: torch.Tensor, interpolation_factor: int, device )

src/pipelines/pipeline_echomimicv2_acc.py:256

↓ 1 callersFunctionload_audio

Open an audio file and read as mono waveform, resampling as necessary Parameters ---------- file: str The audio file to open

src/models/whisper/whisper/audio.py:22

↓ 1 callersFunctionlog_mel_spectrogram

Compute the log-Mel spectrogram of Parameters ---------- audio: Union[str, np.ndarray, torch.Tensor], shape = (*) The path t

src/models/whisper/whisper/audio.py:92

↓ 1 callersFunctionmain

()

infer.py:62

↓ 1 callersFunctionmain

()

infer_acc.py:69

next →1–100 of 395, ranked by callers