https://arxiv.org/abs/2212.00794

Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

Vision Transformer with support for patch or hybrid CNN input stage

HuggingFace model adapter

HuggingFace tokenizer wrapper

A ResNet class that is similar to torchvision's but contains the following changes: - There are now 3 "stem" convolutions as opposed to 1, wi

Image to Patch Embedding

timm model adapter # FIXME this adapter is a work in progress, may change in ways that break weight compat

An autograd function that performs allgather on a tensor. Performs all_gather operation on the provided tensors. *** Warning ***: torch.distri

CLS token pooling

Subclass torch's LayerNorm (with cast back to input dtype).

Subclass torch's LayerNorm to handle fp16 (by casting to float32 and back).

Max pooling

Mean pooling