Distributed Sampler that subsamples indicies sequentially, making it easier to collate all results at the end. Even though we only use t

Distributed Sampler that subsamples indicies sequentially, making it easier to collate all results at the end. Even though we only use t

Multi-headed attention from 'Attention Is All You Need' paper

Multi-headed attention from 'Attention Is All You Need' paper

Implements Adam algorithm. It has been proposed in `Adam: A Method for Stochastic Optimization`_. Arguments: params (iterable): iterab

Core encoder is a stack of N layers

Encoder is made up of self-attn and feed forward (defined below)

Evaluation output (always contains labels), to be used to compute metrics.

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

This module learns positional embeddings up to a fixed maximum size. Padding ids are ignored by either offsetting based on padding_idx or

Implements FFN equation.

output: tuple: (loss, ) in training

This module produces sinusoidal positional embeddings of any length.

This module produces sinusoidal positional embeddings of any length.

Trainer is a simple but feature-complete training and eval loop for PyTorch, optimized for Transformers.

Used by ASTWrapper to collect information. - put constructors in one place. - checks that all fields have names. - get all optional field

Head for sentence-level classification tasks.

Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a :class:`DecoderLayer`. Args: config: BartConfig

Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a :class:`DecoderLayer`. Args: config: BartConfig

Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a :class:`EncoderLayer`. Args: co

Data collator used for query + schema -> sql modeling.

Dataset for training task: SQL (+ schema) -> text

A simple evaluator

output: tuple: (loss, ) in training

Dataset for pretraining task: query + schema -> SQL There is not masking for query and schema.

Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a :class:`EncoderLayer`. Args: co

Dataset for training task: SQL (+ schema) -> text

Simple schema which maps table&column to a unique identifier

Simple schema which maps table&column to a unique identifier

A residual connection followed by a layer norm. Note for code simplicity the norm is first as opposed to last.

A residual connection followed by a layer norm. Note for code simplicity the norm is first as opposed to last.

This dataset is used for pretraining task on generation-based or retrieval-based text-schema pair examples. The fields that will be used is `qu

Trainer is a simple but feature-complete training and eval loop for PyTorch, optimized for Transformers.

Provides helper methods on the ASDL AST.

Used for preprocessing data according to the model's liking. Some tasks normally performed here: - Constructing a vocabulary from the trainin

output: tuple: (loss, ) in training

Head for sentence-level classification tasks.

Given a model and its bert module, create parameter groups with different lr

Set the lr of bert to be zero when the other param group is warming-up