Transformer model for flow matching on sequences.

r""" Attention processor for ID-Adapater. Args: hidden_size (`int`): The hidden size of the attention layer. cross

- perceiver resampler like arch (compared with previous MLP-like arch) - we concat id embedding (generated by arcface) and query tokens as la

https://arxiv.org/abs/2212.00794

A wrapper around torchsde.BrownianTree that enables batches of entropy.

A noise sampler backed by a torchsde.BrownianTree. Args: x (Tensor): The tensor whose shape, device and dtype to use to generate

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Vision Transformer with support for patch or hybrid CNN input stage

HuggingFace model adapter

HuggingFace tokenizer wrapper

A ResNet class that is similar to torchvision's but contains the following changes: - There are now 3 "stem" convolutions as opposed to 1, wi

Image to Patch Embedding

A DiT block with parallel linear layers as described in https://arxiv.org/abs/2302.05442 and adapted modulation interface.

timm model adapter # FIXME this adapter is a work in progress, may change in ways that break weight compat

An autograd function that performs allgather on a tensor. Performs all_gather operation on the provided tensors. *** Warning ***: torch.distri

r""" Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).

CLS token pooling

r""" Attention processor for ID-Adapater for PyTorch 2.0. Args: hidden_size (`int`): The hidden size of the attention laye

Subclass torch's LayerNorm (with cast back to input dtype).

Subclass torch's LayerNorm to handle fp16 (by casting to float32 and back).

Max pooling

Mean pooling