Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Build the parameter buffer in each pp rank

Compile the code on the instance. Args: instance: The MorphCloud instance Returns: InstanceExecResp

Compute the KTO loss using the Liger-Kernel's LigerFusedLinearKTOLoss. Args: model: The policy model used for generating

This function should only be called inside by WorkerGroup

Creates a model from a path or model identifier.

Creates a model from a path or model identifier.

Determine if the model should be saved based on the evaluation metrics. If args.metric_for_best_model is not set, the loss is used.

Enables gradient checkpointing for the model.

Enables gradient checkpointing for the model.

Internal implementation of execute with no retry logic. Args: data: Dictionary containing execution data Return

Execute code using a prepared instance. Args: instance: Ready MorphCloud instance data: Execution data te

hdfs capable to check whether a file_path is exists

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Determines the status code based on the score and feedback message. Args: score: The numeric score (0.0 to 1.0) feedback: Th

Generate samples from the model Args: batch_size (int): Batch size to use for sampling with_grad (bool): Whe

Calculates the probability if the given prompt embedding is from desirable dataset. This function calculates the probability in the p

Get the compile script content.

Extract embeddings from frozen embedding model

Get the run script content.

r""" Initializes the weights of the value head. The default initialization strategy is random. Users can pass a different initializati

r""" We initialize the weights of the value head.