Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

move the batch to device Args: device (torch.device, str): torch device Returns: DataProto: the current Data

Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Args: text (`str`, `List[str]` or

Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special tokens and clean up tokeniz

Copy src from hdfs to local if src is on hdfs or directly return src. If cache_dir is None, we will use the default cache dir of the system. Note

Concat a list of DataProto. The batch is concatenated among dim=0. The meta_info is assumed to be identical and will use the first one.

Create a huggingface pretrained tokenizer. Args: name (str): The name of the tokenizer. correct_pad_token (bool): Whether to corr

Generates the completions for the input prompts. NOTE: This class automatically batches the given prompts, considering the memory con

Performs one decoding iteration and returns newly generated results. This function performs one decoding iteration of the engine. It first

Split the batch among dim=0 into chunks. The meta_info is passed to each DataProto after split. Args: chunks (int): the number of

Create a DataProto from a dict of tensors. This assumes that 1. All the tensor in tensors have the same dim0 2. Only dim0 is the batch

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

Exact match of math if and only if: 1. numerical equal: both can convert to float and are equal 2. symbolic equal: both can convert to sy

Exact match of math if and only if: 1. numerical equal: both can convert to float and are equal 2. symbolic equal: both can convert to sy

Pop a subset of the DataProto via `batch_keys` and `meta_info_keys` Args: batch_keys (list, optional): a list of strings indicati

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

Get the resource pool of the worker_cls

Compute mean of tensor with a masked values.

given TP,DP,PP rank to get the global rank.

Exact match of math if and only if: 1. numerical equal: both can convert to float and are equal 2. symbolic equal: both can convert to sy

broadcast tensor from rank0 across mp_group

Estimates pass@k of each problem and returns them in an array.

See: https://github.com/pytorch/pytorch/issues/563#issuecomment-330103591

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l