Convert video files to a specified format and save them to the destination path. Parameters: org_path (str): The directory containing th

Crop audio window Args: spec: Spectrogram start_frame_num: Starting frame number fps: Frames per

Decode locations from predictions using priors to undo the encoding we did for offset regression at train time. Args: loc (tensor): lo

Detects faces in a given image. This function detects the faces present in a provided BGR(usually) image. The input can be either the

Dynamically crop image with dynamic margin Args: img: Input image original_bbox: Original bounding box

Extract audio from video files and save as WAV format. Parameters: org_path (str): The directory containing the original video files.

Finalize search and return the final candidate sequences Parameters ---------- tokens : Tensor, shape = (n_audio, n_group, cu

Get audio file features Args: wav_path: Audio file path start_index: Starting index Retu

Get mel spectrogram of audio file Args: wav_path: Audio file path start_index: Starting index

Creates a mask tensor for image processing. :return: A mask tensor.

Calculate the source index (src_idx) based on the given drive index, T, s, e, and sampling method. Parameters: - drive_idx (

Get SyncNet input features Args: video_path: Video file path Returns: ndarray: SyncNet i

Initialize training and validation dataloaders

Initialize loss functions and discriminators

Initialize models and optimizers

Initialize SyncNet model

Initialize VGG model

The `MultiHeadAttention` module optionally accepts `kv_cache` which stores the key and value tensors calculated for the previous posi

Open an audio file and read as mono waveform, resampling as necessary Parameters ---------- file: str The audio file to open

Set parameters converted from Caffe models authors of VGGFace2 provide. See https://www.robots.ox.ac.uk/~vgg/data/vgg_face2/. Arguments:

Compute the log-Mel spectrogram of Parameters ---------- audio: Union[str, np.ndarray, torch.Tensor], shape = (*) The path t

Generate anchors from features.

Extract mel spectrogram features Args: audio_input: Input audio Returns: ndarray: Mel sp

load the mel filterbank matrix for projecting STFT into a Mel spectrogram. Allows decoupling librosa dependency; saved using: np.sav

Compute number of time frames of spectrogram

Pad or trim the audio array to N_SAMPLES, as expected by the encoder.

Given a list of groups of samples and their cumulative log probabilities, return the indices of the samples in each group to select a

Update the key-value cache according to the updated beams

Replace any other markers, symbols, and punctuations with a space, and drop any diacritics (category 'Mn' and some manual mappings)

Initialize any stateful variables for decoding a new sequence

Segment video files into smaller clips of specified duration. Parameters: org_path (str): The directory containing the original video fi

Translates the data in landmark_list to the coordinates of the cropped larger face. Parameters: landmark_list (list): A list

Shuffle the points left-right according to the axis of symmetry of the object. Arguments: parts {torch.tensor} -- a 3D or 4D object c

Returns sinusoids for positional embedding

Split video files into training and validation sets based on val_list_hdtf. Parameters: video_files (List[str]): A list of video file na

Convert path (represented as a string) or torch.tensor to a numpy.ndarray Arguments: tensor_or_path {numpy.ndarray, torch.tensor

Transcribe an audio file using Whisper Parameters ---------- model: Whisper The Whisper model instance audio: Union[str

Validation function for model evaluation

Write a transcript to a file in SRT format. Example usage: from pathlib import Path from whisper.utils import write_srt

Detect faces and keypoints in the given image. Parameters: im (np.ndarray): The input image. maxface (bool): Whether