Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).

MLP using 1x1 convs that keeps spatial dims

MLP as used in Vision Transformer, MLP-Mixer and related networks

A downsample pooling module supporting several maxpool and avgpool modes * 'max' - MaxPool2d w/ kernel_size 3, stride 2, padding 1 * 'max2' -

A downsample pooling module supporting several maxpool and avgpool modes * 'max' - MaxPool2d w/ kernel_size 3, stride 2, padding 1 * 'max2' -

Channels-last multi-head attention (B, ..., C)

Channels-last multi-head attention (B, ..., C)

Classifier head w/ configurable global pooling and dropout.

Selectable global pooling layer with dynamic input kernel size

multi-head attention for 2D NCHW tensors

multi-head attention for 2D NCHW tensors

Tensorflow like 'SAME' wrapper for 2D average pooling

BatchNorm + Activation This module performs BatchNorm + Activation in a manner that will remain backwards compatible with weights trained wit

Original CBAM channel attention module, currently avg + max pool variant only.

Conditionally Parameterized Convolution Inspired by: https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/condconv/condconv

Tensorflow like 'SAME' convolution wrapper for 2D convolutions

ConvNeXt Block There are two equivalent implementations: (1) DwConv -> LayerNorm (channels_first) -> 1x1 Conv -> GELU -> 1x1 Conv; all in (

Feature Hook Helper This module helps with the setup and extraction of hooks for extracting features from internal nodes in a model by node

An experimental 'lightweight' that sums avg + max pool first

An experimental 'lightweight' variant that sums avg_pool and max_pool results.

r"""Applies a linear transformation to the incoming data: :math:`y = xA^T + b` Wraps torch.nn.Linear to support AMP + torchscript usage by manual

Tensorflow like 'SAME' wrapper for 2D max pooling

MaxVit conv, window partition + FFN , grid partition + FFN

MaxVit conv, window partition + FFN , grid partition + FFN

Mixed Grouped Convolution Based on MDConv and GroupedConv in MixNet impl: https://github.com/tensorflow/tpu/blob/master/models/official/mn

MaxVit block with parallel cat(window + grid), one FF Experimental timm block.

MaxVit block with parallel cat(window + grid), one FF Experimental timm block.

Experimental. Grid and Block partition + single FFN NxC tensor layout.

Experimental. Grid and Block partition + single FFN NxC tensor layout.

2D Image to Patch Embedding

Relative Position Embedding As per: https://gist.github.com/aravindsrinivas/56359b79f0ce4449bcb04ab4b56a57a2 Originally from: `Attention Augm

Relative Position Embedding As per: https://gist.github.com/aravindsrinivas/56359b79f0ce4449bcb04ab4b56a57a2 Originally from: `Attention Augm

Rotary position embedding NOTE: This is my initial attempt at impl rotary embedding for spatial use, it has not been well tested, and will l

Original CBAM spatial attention module

Transformer block with 2D downsampling '2D' NCHW tensor layout Some gains can be seen on GPU using a 1D / CL block, BUT w/ the need to switc

Transformer block with 2D downsampling '2D' NCHW tensor layout Some gains can be seen on GPU using a 1D / CL block, BUT w/ the need to switc

Layer config context manager that allows setting all layer config flags at once. If a flag arg is None, it will not change the current value.

Attention based 2D feature pooling w/ learned (absolute) pos embedding. This is a multi-head attention based replacement for (spatial) average po

BAT Adapted from: https://github.com/BA-Transform/BAT-Image-Classification

r"""Creates a module that computes blurs and downsample a given feature map. See :cite:`zhang2019shiftinvar` for more details. Corresponds to

Bottleneck Attention Paper: `Bottleneck Transformers for Visual Recognition` - https://arxiv.org/abs/2101.11605 The internal dimensions of t

Constructs a circular ECA module. ECA module where the conv uses circular padding rather than zero padding. Unlike the spatial dimension, the

r""" ConvNeXt A PyTorch impl of : `A ConvNet for the 2020s` - https://arxiv.org/pdf/2201.03545.pdf Args: in_chans (int): Number

ConvNeXt Block