r"""Allows the model to jointly attend to information from different representation subspaces. See reference: Attention Is All You Need .

InceptionV3 feature extractor

SVG tokenizer - supports both 8B and 4B models via config.yaml

Autoregressive generative model

r"""TransformerDecoder is a stack of N decoder layers Args: decoder_layer: an instance of the TransformerDecoderLayer() class (required).

r"""TransformerEncoder is a stack of N encoder layers Args: encoder_layer: an instance of the TransformerEncoderLayer() class (required).

Gradually warm-up(increasing) learning rate in optimizer. Proposed in 'Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour'. Args:

Wrapper around SVGTensorDataset intended to finetune a model on a list of additional SVGs. Randomly samples fraction `frac` of SVGs to be fin

r"""TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network. This standard decoder layer is based on the paper "A

r"""TransformerEncoderLayer is made up of self-attn and feedforward network. This standard encoder layer is based on the paper "Attention Is All Y

Reference: https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Basic_Shapes