Creates an instance of the LM class using the given argument string and additional config. Parameters: - arg_string: A strin

Decoding, either greedy or with top-k or top-p sampling. If top-k = 0, don't limit the number of candidates (pure sampling). Top-k and top-p c

Decoding, either greedy or with top-k or top-p sampling. If top-k = 0, don't limit the number of candidates (pure sampling). Top-k and top-p c

Decoding, either greedy or with top-k or top-p sampling. If top-k = 0, don't limit the number of candidates (pure sampling). Top-k and top-p c

Decoding, either greedy or with top-k or top-p sampling. If top-k = 0, don't limit the number of candidates (pure sampling). Top-k and top-p c

Only keep keys where fn(key) is True. Support nested DictConfig.

Divide the elements from *iterable* into *n* parts, maintaining order. >>> group_1, group_2 = divide([1, 2, 3, 4, 5, 6], 2) >>> l

residual_in_fp32 only has an effect if residual is None. Otherwise residual dtype is residual.dtype.

residual_in_fp32 only has an effect if residual is None. Otherwise residual dtype is residual.dtype.

Returns the config as a dictionary.

Instantiate and evaluate a model on a list of tasks. :param lm: obj Language Model :param task_dict: dict[str, Task] Dictiona

Example of inference with trained model. It loads trained image classification model from checkpoint. Then it loads example image and predicts

:return: Iterable[obj] A iterable of any object, that doc_to_text can handle

Search upward in the directory tree to a maximum of three layers to find and return the package root (containing the 'tests' folder)

Arguments: x: (batch, seqlen, hidden_dim) (where hidden_dim = num heads * head dim) if cu_seqlens is None and max

x: (batch, length, dim) input tensor state: additional state from the model backbone *args, **kwargs: additional info from th

Gaussian Error Linear Unit (GELU)

GeLU_ activation - Gaussian error linear unit, with tanh approximation .. _GeLU: https://arxiv.org/pdf/1606.08415.pdf

Gets the git commit hash of your current repo (if it exists). Source: https://github.com/EleutherAI/gpt-neox/blob/b608043be541602170bfcfb8ec9

Initializes multi-GPU-friendly python logger.

Returns the number of neuron cores on the current instance.

Gets distributed rank or returns zero if distributed is not initialized.

Gets distributed rank or returns zero if distributed is not initialized.

Process results from OpenAI API response. :param response: dict OpenAI API Response :param ctxlen: int Length of context (so

- context_len allows for a rolling window context, allowing each prediction window to potentially condition on some context :param tok

Creates a dictionary of task objects from either a name of task, config, or prepared Task object. :param task_name_list: List[Union[str, Dict, Ta

Gets total number of distributed workers or returns one if distributed is not initialized.

Gets total number of distributed workers or returns one if distributed is not initialized.

Adjust to get softmax (from RBF)

Generate IID Gaussian random features

Import an object from a string. Parameters: - name (str): The name of the object to import. Returns: - object: The imported

Initialize query and key mapping for linear attention

Initialize query and key mapping for linear attention

Creates an dictionary of tasks index. :param include_path: str = None An additional path to be searched for tasks :retur