Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

move the batch to device Args: device (torch.device, str): torch device Returns: DataProto: the current Data

Create a DataProto from a dict of tensors. This assumes that 1. All the tensor in tensors have the same dim0 2. Only dim0 is the batch

Copy src from hdfs to local if src is on hdfs or directly return src. If cache_dir is None, we will use the default cache dir of the system. Note

Create a huggingface pretrained tokenizer. Args: name (str): The name of the tokenizer. correct_pad_token (bool): Whether to corr

Split the batch among dim=0 into chunks. The meta_info is passed to each DataProto after split. Args: chunks (int): the number of

Pop a subset of the DataProto via `batch_keys` and `meta_info_keys` Args: batch_keys (list, optional): a list of strings indicati

Concat a list of DataProto. The batch is concatenated among dim=0. The meta_info is assumed to be identical and will use the first one.

r"""Works like shutil.copy() for file, and shutil.copytree for dir, and supports hdfs. Copy data and mode bits ("cp src dst"). Return the file's

r"""Works like os.makedirs() but supports hdfs. Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except tha

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

Compute mean of tensor with a masked values.

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Get ulysses sequence parallel process group.

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they are

given TP,DP,PP rank to get the global rank.

Get ulysses sequence parallel world size.

broadcast tensor from rank0 across mp_group

Create a random mask given input_ids. Support left padding and right padding. Process: - Sample valid token length - Sample left_padding l

Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special tokens and clean up tokeniz

A func to sync embedding input with alltoall in sequence parallel gather sequence dimension and scatter head dim: e.g. seq_dim: 1, head_d

Return a tensor with the input `shape` as a view into the 1-D data starting at `start_index`.

Build the model.

See: https://github.com/pytorch/pytorch/issues/563#issuecomment-330103591

Split the batch into a list of micro_batches, where the max_token_len is smaller than max_token_len and the number of valid tokens in each micro b

Repeat the batch data a specified number of times. Args: repeat_times (int): Number of times to repeat the data.

Pad and slice input_ids to be divisible by sp_size Pad position_ids to be divisible by sp_size. Note both input_ids_rmpad and position_i

TODO: optimize this. Technically, we only need one broadcast

broadcast tensor across mp_group

broadcast tensor in tp shards across mp_group

TODO: optimize this. - We can use async ops - We can use only one allgather Args: tensors: size: group:

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

We compute dense reward here so that we can directly train RL without SFT

Generate sequences

Get the resource pool of the worker_cls

Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/tenso

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Rotates half the hidden dims of the input.

Set ulysses sequence parallel process group.

broadcast tensor in tp shards across mp_group