Log a table of validation samples to the configured logger (wandb or swanlab)

bind the method with actor_prefix to its original name

添加答案节点

`List[int]`: List the ids of the special tokens(`'<unk>'`, `'<cls>'`, etc.) mapped to class attributes.

`List[str]`: A list of the unique special tokens (`'<unk>'`, `'<cls>'`, ..., etc.). Convert tokens of `tokenizers.AddedToken` type t

Check correctness of code generation with a global timeout. The global timeout is to catch some extreme/rare cases not handled by the timeouts

Tensor extenstion to torch.clamp https://github.com/pytorch/pytorch/issues/2793#issuecomment-428784713

Each output must be a DataProto. We concat the dim=0 of output

Only collect output of megatron pp. This is useful when examine weight names as they are identical in tp/dp

Compute Categorical entropy loss Args: logits: `(torch.Tensor)` shape: (bs, response_length, vocab_size) eos_mask: `(

Adapted from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py Args: token_level_rewards: `(torch.Tensor)`

Compute advantage for GRPO, operating only on Outcome reward (with only one scalar reward for each response). Args: token_level_

Compute the log probability of the responses given input_ids, attention_mask and position_ids Args: data (DataProto): a DataProto

Compute the log probability of the responses given input_ids, attention_mask and position_ids Args: data (DataProto): a DataProto

Adapted from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1122 Args: old_log_prob: `(torch.Tensor)`

Compute advantage for REINFORCE++. This implementation is based on the paper: https://arxiv.org/abs/2501.03262 Args: token_level

Compute advantage for ReMax, operating only on Outcome reward This implementation is based on the paper: https://arxiv.org/abs/2310.10505

Compute advantage for RLOO based on https://arxiv.org/abs/2402.14740 Args: token_level_rewards: `(torch.Tensor)` shape: (

The scoring function for GSM8k. Reference: Trung, Luong, et al. "Reft: Reasoning with reinforced fine-tuning." Proceedings of the 62nd Annual Mee

Compute the value loss. Copied from https://github.com/huggingface/trl/blob/main/trl/trainer/ppo_trainer.py#L1151 Args: vpreds (`torch.Fl

Compute values

Converts a single index or a sequence of indices in a token or a sequence of tokens, using the vocabulary and added tokens.

Converts a sequence of tokens in a single string. The most simple way to do it is `" ".join(tokens)` but we often want to remove sub-

Args: model_name: override_config_kwargs: Returns:

Create a random mask given input_ids. Support left padding and right padding. Process: - Sample valid token length - Sample left_padding l

This method forwards all its arguments to Qwen2TokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this

All the args and kwargs must be DataProto. The batch will be chunked by dp_size and passed to each rank

Calculate entropy from logits.

`Optional[int]`: Id of the end of sentence token in the vocabulary. Returns `None` if the token has not been set.