Partially borrowed from Keras # Arguments sequences: list of lists where each element is a sequence maxlen: int, maximum leng

Generate an action by maximum q values.

Disable the redundant torch default initialization to accelerate model creation.

end of sentence token can be int or list: 1 or [1, 2] e.g. eos_token=1 response_id: [0, 0, 2, 42, 3, 5, 1, 0, 0] eos_mask: [1, 1,

Log a value of some diagnostic Call this once for each diagnostic quantity, each iteration If called many times, last value will be used.

Compute mean of tensor with a masked values.

pad a 2D tensors (e.g. responses, logprobs) in the last dim to max_seq_length. input shape: [bs, seq_length] output shape: [bs, max_seq_l

Pad and slice input_ids to be divisible by sp_size Pad position_ids to be divisible by sp_size. Note both input_ids_rmpad and position_i

Pad and slice input_ids to be divisible by sp_size Pad position_ids to be divisible by sp_size. Note both input_ids_rmpad and position_i

given TP,DP,PP rank to get the global rank.

given TP,DP,PP rank to get the global rank.

given TP,DP,PP rank to get the global rank.

given TP,DP,PP rank to get the global rank.

Disable the redundant torch default initialization to accelerate model creation.

Generate sequences

Get all possible actions in the current environment state.

Get a list of all observable objects

returns a dictionary of all targets for the task

Get the description of the current task.

Split the batch into a list of micro_batches, where the max_token_len is smaller than max_token_len and the number of valid tokens in each micro b

Reset internal states

state transition function; here, takes in an action whose * logical and numerical preconds are fulfilled, and returns TRUE, * putting the result int

Select a subset of the DataProto via batch_keys and meta_info_keys Args: batch_keys (list, optional): a list of strings indicatin

broadcast tensor from rank0 across mp_group

broadcast tensor from rank0 across mp_group

broadcast tensor from rank0 across mp_group

broadcast tensor from rank0 across mp_group

fetch tensor

fetch tensor

TODO: optimize this. Technically, we only need one broadcast