Restore model/optimizer state and return ``(next_step, losses)``. Older checkpoints did not have ``last_completed_step``. For those, ``steps

Generate one completion per (possibly variable-length) tokenized prompt and merge the results into padded tensors. Length-buckets prompts (th

Set all optimizer parameter groups to the same learning rate.

Masked next-token cross-entropy. Args: logits: (B, T, V) model logits for ``tokens``. tokens: (B, T) packed token ids.

Initializes the attention head. Args: head_size (int): The dimensionality of the key, query, and value projections.

Recursively merge ``src`` into ``dst`` (nested-dict aware; future-proof).

Remove a leading ``module.`` from keys saved by DistributedDataParallel.

Mean approximate KL(old || new) over response tokens (a PPO health metric).

Mean ``-log sigmoid(chosen - rejected)`` over a batch of preference pairs.

Construct the exact command (torchrun for multi-GPU, else python).

Read the learning rate from the first optimizer parameter group.

Return a checkpoint directory tied to the configured final model path.

Manages downloading of the dataset, including both validation and training files. Args: val_url (str): URL for the validation da

Standard DPO loss. Inputs are summed response log-probs (B,). Returns ``(loss, chosen_reward, rejected_reward)`` where the implicit rewards

Print a rough training VRAM budget so users can predict OOM before launching. AdamW keeps fp32 weights + grads + two moment buffers (~16 byt

Apply temperature, then optional top-k and nucleus (top-p) filtering to a batch of next-token logits ``(B, vocab)``. Returns logits with filt

Applies the hidden linear layer followed by ReLU activation. Args: x (torch.Tensor): Input tensor. Returns:

Generates text using a pre-trained Transformer model. Args: model_path (str): Path to the saved model checkpoint. input_text

Build a short report describing the runtime environment: PyTorch/CUDA versions and, when running on a GPU, its name, capability, and total VR

Reformat a GSM8K (question, answer) into chat messages whose assistant turn uses the <think>...</think><answer>N</answer> structure.

KTO from paired data: chosen = desirable, rejected = undesirable, with a reference-KL baseline estimated (detached) from the batch's mean log

Load a checkpoint while supporting both newer and older PyTorch versions.

Reconstruct a trained :class:`RewardModel` (backbone + reward head) from a reward checkpoint saved by ``scripts/train_reward.py``.

Return the learning rate that should be active at a given step.

Main function to parse arguments and orchestrate the dataset download process.

Main function to parse arguments, validate directories, and process files.

ORPO (reference-free). Uses per-token MEAN log-probs. ``L = NLL(chosen) + lambda * -log sigmoid(log_odds_chosen - log_odds_rejected)`` w

Pack a list of ``(ids, loss_mask)`` examples into fixed-length ``context_length`` rows by concatenating them and slicing. Examples are alread

Applies the projection linear layer. Args: x (torch.Tensor): Input tensor. Returns: torch.Tensor: O

Render the diagram (if any) + the prose of a docs markdown file.

Mean reward gap (chosen - rejected); a useful training diagnostic.

Real token count per row = last response position + 1 (or prompt_len if no response).

Sequence-level summed log-prob over response tokens (used by DPO/KTO/ORPO). Returns ``(sum_logprob, n_tokens)`` each shape (B,). The per-tok

Return an instance of ``cfg_cls`` shrunk to the tiny SMOKE model dims.

Per-token scalar reward (B, T) -- useful for diagnostics / dense shaping.