Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

move the batch to device Args: device (torch.device, str): torch device Returns: DataProto: the current Data

move the batch to device Args: device (torch.device, str): torch device Returns: DataProto: the current Data

If distributed is initialized, print only on rank 0.

If distributed is initialized, print only on rank 0.

Pop a subset of the DataProto via `batch_keys` and `meta_info_keys` Args: batch_keys (list, optional): a list of strings indicati

Pop a subset of the DataProto via `batch_keys` and `meta_info_keys` Args: batch_keys (list, optional): a list of strings indicati

Context manager for timing code execution. This utility function measures the execution time of code within its context and accumulates the t

Context manager for timing code execution. This utility function measures the execution time of code within its context and accumulates the t

Split the batch among dim=0 into chunks. The meta_info is passed to each DataProto after split. Args: chunks (int): the number of

Create a DataProto from a dict of tensors. This assumes that 1. All the tensor in tensors have the same dim0 2. Only dim0 is the batch

Split the batch among dim=0 into chunks. The meta_info is passed to each DataProto after split. Args: chunks (int): the number of

Return the corresponding torch attribute based on the device type string. Returns: module: The corresponding torch device namespace, or to

Return the corresponding torch attribute based on the device type string. Returns: module: The corresponding torch device namespace, or to

Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special tokens and clean up tokeniz

Sleep engine to offload model weights and discard kv cache.

Load a dataset from a JSON file based on the dataset enum. This function takes a dataset enum value and loads the corresponding JSON file fro

Copy files/directories from HDFS to local cache with validation. Args: src (str): Source path - HDFS path (hdfs://...) or local filesyste

Copy files/directories from HDFS to local cache with validation. Args: src (str): Source path - HDFS path (hdfs://...) or local filesyste

Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special tokens and clean up tokeniz

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

Create a tool instance. Args: instance_id: The instance id of the tool. Returns: The instance id of the tool

Create a huggingface pretrained tokenizer which correctness handles eos and pad tokens. Args: name (str): The name of the tokenizer.

Concat a list of DataProto. The batch is concatenated among dim=0. The meta_info is assumed to be identical and will use the first one.

Concat a list of DataProto. The batch is concatenated among dim=0. The meta_info is assumed to be identical and will use the first one.

Sleep engine to offload model weights and discard kv cache.

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Create a huggingface pretrained tokenizer which correctness handles eos and pad tokens. Args: name (str): The name of the tokenizer.

Create a tool instance. Args: instance_id: The instance id of the tool. Returns: The instance id of the tool

Create a DataProto from a dict of tensors. This assumes that 1. All the tensor in tensors have the same dim0 2. Only dim0 is the batch

A func to sync embedding input with alltoall in sequence parallel gather sequence dimension and scatter head dim: e.g. seq_dim: 1, head_d

A func to sync embedding input with alltoall in sequence parallel gather sequence dimension and scatter head dim: e.g. seq_dim: 1, head_d

Repeat the batch data a specified number of times. Args: repeat_times (int): Number of times to repeat the data.

Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Args: text (`str`, `List[str]` or

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Args: text (`str`, `List[str]` or

Split a batch into micro-batches by total token count, with optional DP sync and padding. Args: batch (TensorDict): must include "at

Repeat the batch data a specified number of times. Args: repeat_times (int): Number of times to repeat the data.

Execute a method on all workers synchronously. Args: method_name: Name of the method to execute *args: Positional arg

r"""Works like os.makedirs() but supports hdfs. Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except tha

Generate sequences

Generate sequences

r"""Works like os.makedirs() but supports hdfs. Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except tha

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Split a batch into micro-batches by total token count, with optional DP sync and padding. Args: batch (TensorDict): must include "at

Check correctness of code generation with a global timeout. The global timeout is to catch some extreme/rare cases not handled by the timeouts