Returns the number of neuron cores on the current instance.

Process results from OpenAI API response. :param response: dict OpenAI API Response :param ctxlen: int Length of context (so

- context_len allows for a rolling window context, allowing each prediction window to potentially condition on some context :param tok

Creates a dictionary of task objects from either a name of task, config, or prepared Task object. :param task_name_list: List[Union[str, Dict, Ta

Import an object from a string. Parameters: - name (str): The name of the object to import. Returns: - object: The imported

Creates an dictionary of tasks index. :param include_path: str = None An additional path to be searched for tasks :retur

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://huggingface.co

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://arxiv.org/abs/

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://arxiv.org/abs/

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://arxiv.org/abs/

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://arxiv.org/abs/

Load a config from a wandb run ID. Parameters: run_id (str): A full wandb run id like "hazy-research/attention/159o6asi"

Load a model from a wandb run ID. Parameters: run_id (str): A full wandb run id like "hazy-research/attention/159o6asi"

Loads a dictionary of task objects from a list :param task_list: Union[str, list] = None Single string or list of string of task

Unpads query, key, and values tensors, using a single dimension for all tokens even though they belong to different batches. This function i

Override the default metrics used for evaluation with custom metrics. Parameters: - metric_name (str): The name of the custo

Load, tokenize, and save the processed dataset. Args: dataset: Path or name of the dataset. Default: 'HuggingFaceFW/fine

Process a single batch of data

Create a pruned version of a SCROLLS task dataset containing only inputs that are less than `max_tokens` when tokenized by each tokenizer

Reconstruct and mix transformed outputs back into the original input sequence shape. Key operations: 1. Reshapes and transposes `transfo

Find the package root and run the tests for the given tasks

Draw `n` samples from our fewshot docs. This method should be overridden by subclasses.

Parses something like args1=val1,arg2=val2 Into a dictionary

Splits a string on whitespaces and records the indices of each in the original string. @:return generator((word, (start_idx, end_idx)), ...)

dumps the current config as a dictionary object, as a printable format. null fields will not be printed. Used for dumping results alon

Tokenize the input text and split into chunks of specified context length. Args: examples: Dictionary containing the inp

Transform input sequences into memory-organized chunks with capacity constraints. Processes input sequences by routing tokens to designated

Transform input sequences into memory-organized chunks with capacity constraints. Processes input sequences by routing tokens to designa

Transform input sequences into memory-organized chunks with capacity constraints. Processes input sequences by routing tokens to designa

Transform input sequences into memory-organized chunks with capacity constraints. Processes input sequences by routing tokens to designa

Transform input sequences into memory-organized chunks with capacity constraints. Processes input sequences by routing tokens to designa

Custom Code for the Just Read Twice Project

Takes a flat dictionary with '/' separated keys, and returns it as a nested dictionary. Parameters: d (dict): The flat dictionary t

Splits a string into pairs of (ngram words, their start/end indices)