Return a sequence_length + 1 token sequence consisting of the following: - (1) S, the RNG length-sentinel in the range [0, sequence_length

Abstract method implementation Args: idx (int): The index into the dataset Returns: Dict[str, torch.Tensor]:

Return the pointer, length, and mode at the index Args: idx (int): The index into the dataset Returns: Tuple

Return from the dataset Args: idx (Union[int, numpy.integer, slice]): The index or index slice into the dataset Raises:

Return from the dataset Args: idx (int): The index into the dataset Returns: Dict[str, Union[torch.Tensor, n

Get dataset sample. Args: idx (int): Index of sample. Returns: A dict containing attribute 'text' of type st

Get block path from index. Args: idx (int): Index of sample. Returns: The path to the block file containing

Get dataset sample. Args: idx (int): The index into the dataset. Returns: Dict[str, numpy.ndarray]: The text

Get dataset sample. Args: sample_idx (int): Index of sample in dataset. Returns: A dict consisting of GPT sa

Get sample, including represented document IDs. Args: idx (int): Sample index. Returns: A sample, which cont

DB dataset sample. Args: chunk_id (int): Index of chunk within dataset. Returns: A dict containing:

Get the state during pickling Returns: Tuple[str, bool, bool]: The state tuple

RMS Normaliation module Args: dim (int): The width of input, i.e. hidden size eps (float): epsilon to use for the nor