A single linear layer which does a 1x1 conv

Using Decord(https://github.com/dmlc/decord) to initialize the video_reader.

A single linear layer which does a 1x1 conv

The registry that provides name -> object mapping, to support third-party users' custom modules. To create a registry (e.g. a backbone r

r""" A cross attention layer. Parameters: query_dim (`int`): The number of channels in the query. cross_attention

r""" Pipeline for text-to-image generation using PixArt-Alpha. This model inherits from [`DiffusionPipeline`]. Check the superclass documenta

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

r""" A feed-forward layer. Parameters: dim (`int`): The number of channels in the input. dim_out (`int`, *optional*): The num

Convert tensor data type from uint8 to float, divide value by 255.0 and permute the dimensions of clip tensor

A general file client to access files in different backend. The client loads a file or text in a specified backend from its path and return i

r""" A gated self-attention dense layer that combines visual features and object features. Parameters: query_dim (`int`): The number

Utilities for training and sampling diffusion models. Original ported from this codebase: https://github.com/hojonathanho/diffusion/blob/

Temporally crop the given frame indices at a random location. Args: size (int): Desired length of frames will be seen in the model.

Generic dataset for videos files stored in folders Returns BCTHW videos in the range [-0.5, 0.5]

ODE solver class

r""" Norm layer adaptive layer norm single (adaLN-single). As proposed in PixArt-Alpha (see: https://arxiv.org/abs/2310.00426; Section 2.3).

Compatible with old versions, there are issues, use with caution.

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

r""" A basic Transformer block. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads

A class for embedding texts and images using a pretrained CLIP model.

Projects caption embeddings. Also handles dropout for classifier-free guidance. Adapted from https://github.com/PixArt-alpha/PixArt-alpha/bl

First use the short side for cropping length, center crop video, then resize to the specified size

Charbonnier loss (one variant of Robust L1Loss, a differentiable variant of L1Loss). Described in "Deep Laplacian Pyramid Networks for Fast a

For PixArt-Alpha. Reference: https://github.com/PixArt-alpha/PixArt-alpha/blob/0f55e922376d8b797edd44d25d0e7464b260dcab/diffusion/model/

takes 2 distances, puts through fc layers, spits out value between [0,1] (if use_sigmoid is True)

Dynamic Relative Position Bias. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. re

credit to @karpathy: https://github.com/karpathy/deep-vector-quantization/blob/main/model.py (thanks!) Gumbel Softmax trick quantizer Cat

Inception-v1 I3D architecture. The model is introduced in: Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset Joa

Pads images such that dimensions are divisible by divisor

L1 (mean absolute error, MAE) loss. Args: loss_weight (float): Loss weight for L1 loss. Default: 1.0. reduction (str): Specifies

to match AttnBlock usage

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

Code from https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L148

First use the long side, then resize to the specified size