A single linear layer which does a 1x1 conv

An LLM for generating texts from given prompts and sampling parameters. This class includes a tokenizer, a language model (possibly distributed

Sampler that repeats the indices of a dataset in a structured manner. Args: data_source (`Sized`): Dataset to sample fro

A worker class that executes (a partition of) the model on a GPU. Each worker is associated with a single GPU. The worker is responsible for

Note: In vllm, PagedAttention supports head sizes [64, 80, 96, 112, 128, 256]. However, LlamaGen-3B model has head size 100 (for some his

Embeds text caption into vector representations. Also handles label dropout for classifier-free guidance.

Calculate pairwise distances between vectors. Adapted from https://github.com/kynkaat/improved-precision-and-recall-metric/blob/f60f25e5ad93

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.

Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.

A helper for comparing manifolds of feature vectors. Adapted from https://github.com/kynkaat/improved-precision-and-recall-metric/blob/f60f2

Defines a PatchGAN discriminator as in Pix2Pix --> see https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/models/networks.py

Samples the next tokens from the model's outputs. This layer does the following: 1. Discard the hidden states that are not used for sampling

see https://github.com/MishaLaskin/vqvae/blob/d761a999e2267766400dc646d82d3ac3657771d4/models/quantizer.py __________________________________