Inference parameters that are passed to the main model in order to efficienly calculate and store the context during inference.

Inference parameters that are passed to the main model in order to efficienly calculate and store the context during inference.

A class for reordering and batching elements of an array. This class allows for sorting an array based on a provided sorting function, group

TaskManager indexes all tasks from the default `lm_eval/tasks/` and an optional directory if provided.

Feature map to compute 2nd-order Taylor approx. of exp(q^T k / sqrt(d))

Code from https://github.com/OpenNLPLab/cosFormer/blob/main/cosformer.py

Maintains (exponential) moving average of a set of parameters. Args: parameters: Iterable of `torch.nn.Parameter` (typically from

Parent feature map; default is identity function

Loader for IndexedDataset

Inference parameters that are passed to the main model in order to efficienly calculate and store the context during inference.

The attention (mixer) and MLP blocks are done in parallel, similar to GPT-J, GPT-NeoX, and PaLM.

Code from https://github.com/teddykoker/performer/blob/main/performer.py

Feature map to compute 2nd-order Taylor approx. of exp(q^T k / sqrt(d))

Simple torch dataset that returns batches instead of individual examples. This is needed to support data that contains different data segments no

[Copied from https://github.com/huggingface/transformers/blob/main/examples/flax/language-modeling/run_t5_mlm_flax.py] Data collator used for

FilterEnsemble creates a pipeline applying multiple filters. Its intended usage is to stack multiple post-processing steps in order. `tas

Project positive features to upper and lower "halfspaces"

Criteria to stop on the specified multi-token sequence.

Simple wrapper around nn.Conv1d that accepts dimension last.

Simple wrapper around nn.Conv1d that accepts dimension last.

Feature map to compute 2nd-order Taylor approx. of exp(q^T k / sqrt(d))

Wrap torchmetrics.Accuracy to take argmax of y in case of Mixup.

BaseConv with implicit filter parameterized by an MLP. Args: d_model (int): The number of expected features in the input and output.