broadcast tensor across mp_group

broadcast tensor across mp_group

Compute logits given a batch of data. Args: data (DataProto): a batch of data represented by DataProto. It must contain key ```in

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

Get FSDP wrap policy for the module. Args: module: The module to get wrap policy for config: Configuration for wrap policy

Get the resource pool of the worker_cls

end of sentence token can be int or list: 1 or [1, 2] e.g. response_id = torch.tensor([[20, 10, 34, 1, 0, 0, 0],

Pad a DataProto to size divisible by size_divisor Args: size_divisor (int): size divisor Returns: data: (DataProto): the pad

spawn to a dictionary of worker groups, each with a subset of method with prefix.

Reorder the data on single controller such that each dp rank gets similar total tokens

broadcast tensor across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

Currently, we only support chatml format.

model: AutoModelForCausalLM

TODO: optimize this. Technically, we only need one broadcast

We compute dense reward here so that we can directly train RL without SFT

Create a dataset. Arguments: data_config: The data config. tokenizer (Tokenizer): The tokenizer. processor (Processor): T

Create a dataset.

Create a schedule with a learning rate that decreases following the values of the cosine function between the initial lr set in the optimizer

Initialize a Mcore model. Args: tfconfig: The transformer config. hf_config: The HuggingFace config. pre_process: Op

Load weights for mcore GPT model.

r"""Make an iterator from the DataProto. This is built upon that TensorDict can be used as a normal Pytorch dataset. See https://pytorch.org/t

Rotates half the hidden dims of the input.

Rotates half the hidden dims of the input.

Set ulysses sequence parallel process group.

Update the module config with the override_config_kwargs. Args: module_config: The module config from Huggingface Transformers. ov

Bind the worker method to the WorkerGroup

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

fetch tensor in tp shards

fetch tensor in tp shards

Create a base TransformerConfig with common parameters across different model architectures. TODO: (ycl) use dataclass or converter config?

Get current memory usage.