Root config for soup.yaml.

SQLite-backed store for registry entries, artifacts, lineage, tags. Usable as a context manager: ``with RegistryStore() as store: ...`` guara

SQLite-backed experiment tracker for training runs and evaluations.

Deterministic char-level fake tokenizer for doctor/lint unit tests. Renders ``<role>:content\\n`` per turn; each char maps to ``(ord(c) % 200

Immutable snapshot of a `soup loop` configuration + counters. The persisted file is JSON-formatted (despite the ``.yaml`` extension) so we ca

High-level wrapper that sets up model + tokenizer + trainer from SoupConfig.

Bridges HF Trainer events to Soup's Rich live display and experiment tracker.

Frozen MiniLLM configuration. Cross-validation: - ``pretrain_anchor_weight > 0`` requires ``pretrain_anchor_path`` to be set (otherwise

Frozen rollout policy: stable vs canary + traffic split + verdict.

Frozen config for the dynamic re-weighting policy. Attributes: num_buckets: Number of difficulty buckets (must match ``traini

SQLite wrapper. Single-connection, single-thread.

Per-mode score with an OK / MINOR / MAJOR verdict + evidence line.

A candidate base model for a tunability probe. `params_b` is the parameter count in billions (e.g. ``0.6`` for 600M).

One CheckList test definition.

Thread-safe append-only JSONL writer with size-based rotation.

Daemon configuration knobs.

Top-level metadata for a ``.can`` file.

Thread-safe append-only JSONL log for the reward-hack controller. One JSON object per :meth:`record` call with shape ``{"ts": ..., "step": ..

Yield batches of indices packed via FFD into bins of ``batch_max_len``. Two modes: * ``real_batches=False`` — each yielded value is a flat l

Frozen ULD configuration. - ``strategy``: one of :data:`SUPPORTED_ULD_STRATEGIES`. - ``student_vocab_size`` / ``teacher_vocab_size``: positiv

Caller-supplied bundle inputs.

One probe entry in a pack.

Frozen controller state; evolve via :func:`dataclasses.replace`. - ``beta`` / ``kl_coef``: current ABSOLUTE coefficient applied (0.0 is the

Inputs needed to predict peak VRAM.

A tool-level failure with a pre-sanitized, path-free message. The MCP SDK stringifies a raised exception verbatim into an ``isError`` result,

Per-token gating network over N task LoRAs. Frozen dataclass — post-construction mutation raises ``FrozenInstanceError``.

One SSE-streamed training event.

An ordered history of checkpoint identifiers to bisect over. The order is "oldest first" — index 0 is the earliest checkpoint and the last en

Stateful per-base-model edit governor. Tracks ``edit_count`` and the last observed verdict so subsequent edits can be refused once the model

High-level wrapper for GRPO training from SoupConfig. GRPO generates multiple completions per prompt, scores them with a reward function, and

Immutable LISA schedule.

Synthetic length-preferring Soup evaluator: prefers the LONGER response. Provides BOTH shapes so it drives either trl API: ``compare_pair`` (pair

Per-run Annex XI/XII input.

Configurable LLM-as-a-judge evaluator.

Parameters for an mSPRT pass.

High-level wrapper for PPO training from SoupConfig. PPO generates completions for each prompt, scores them with a reward model or reward fun

Frozen RL-checkpoint config. - ``save_every_steps``: int >= 1; ``[1, 10_000_000]`` cap. - ``include_optimizer_state``: include AdamW / Lion s

Thread-safe, capped ring of `ToolCallRecord` entries. Uses `collections.deque(maxlen=_MAX_RECORDS)` so the buffer drops the oldest record on

Outcome of one candidate's probe run.

A multi-tenant LoRA storage bank. ``projection_seed`` is the deterministic seed for the shared random projection P. Each ``BankEntry`` carrie

Mock judge that returns scripted weighted_score values.

A full CheckList suite (one or more tests, unique names).

One check's verdict + human-readable message/evidence.

HF TrainerCallback emitting dynamic curriculum bucket-weight history. BETA: the callback is live in v0.53.5 but the sampler-side consumer (li

One SCOPE/CJE bidirectional pairwise judgement. The same (model_a, model_b) pair is judged twice with positions swapped: ``first_winner`` is

Frozen policy for the ReLoRA callback. Attributes: steps: fire every N global steps (must be > 0). warmup_ratio: fraction of tota

Character-level fake tokenizer. Renders messages as ``<role>:<content>\\n`` and tokenizes each char to ``ord(c) % 256``. Two modes for ``appl

One candidate quant configuration + its measured score/latency.

High-level wrapper for DPO training from SoupConfig. DPO requires preference data with three fields: - prompt: the input prompt - chosen:

Per-metric regression tolerance for the pre-push gate. Each tolerance is the *minimum acceptable* delta vs the baseline. Negative means a reg

Frozen per-item Rasch fit.

Live tournament tracker with Elo + win/loss bookkeeping.

Per-run BOM input. The shape mirrors what we can read from a v0.26.0 ``RegistryEntry``: ``base_model`` + ``base_sha`` + ``config_sha`` + ``da

SQLite-backed persistence for per-base-model edit-governor state. Single-connection. The DB path must stay under ``$HOME`` / ``$CWD`` / ``$TM

One package -> version row in the lockfile.

Captured snapshot of the running environment.

Parsed ``evals/gate.yaml``.

Calibration verdict for a pairwise judge. ``position_bias`` ∈ [-1, 1] (0 = no bias). ``conformal_threshold`` ∈ [0, 1] (production gate thresh

One-line decision the user came for, plus its supporting evidence.

Fake RLSignalBuffer returning a scripted sequence of snapshots.

An adapter PR — the triple {base SHA, dataset diff, adapter path} plus eval deltas + sample diffs for human review.

One audit record. JSON-serialised one-per-line.

Single-battery score with OK/MINOR/MAJOR verdict.

The frozen result of running a compilation.

HF TrainerCallback that wires v0.50.0 Part D stability knobs. Lazy-inherits ``transformers.TrainerCallback`` so the module is importable with

Tracks VRAM pressure and recommends batch / accum adjustments.

HF ``TrainerCallback`` implementing LISA layer sampling. Subclasses the lazily-resolved ``TrainerCallback`` so it inherits the no-op default

One pairwise interference measurement (target A, co-loaded B).

Frozen per-round descriptor. - ``round_index``: 0-based, non-negative int (bool rejected). - ``prompts_path``: source prompts JSONL (sampled

HF ``TrainerCallback`` that magnitude-prunes LoRA weights every N steps. Subclasses the lazily-resolved ``TrainerCallback`` so it inherits the no

One node in the data-recipe DAG.

Closed-loop reward-hacking mitigation HF TrainerCallback (v0.71.26). Reads the shared :class:`~soup_cli.utils.rl_signal_buffer.RLSignalBuffer`

Defection probe verdict for one model evaluation.

The on-disk lockfile shape. Stored as JSON at ``soup.lock`` (or operator-named path); committed to git so the team coordinates on "reproducib

Thread-safe ring buffer of `TrainEvent` objects. `push` appends one event (dropping the oldest on overflow); `drain` returns all events queue

Word-level fake tokenizer with offset-mapping support. Tokenises on non-whitespace runs; each token gets a deterministic id and its ``(start,

One metric: baseline -> candidate. The ``delta`` field is a computed property (post-init).

Real HF ``TrainerCallback`` auto-pusher. Subclasses the lazily-resolved ``TrainerCallback`` so it inherits the no-op default for every Traine

High-level wrapper for KTO training from SoupConfig. KTO uses unpaired preference data with three fields: - prompt: the input prompt - co

One proxy-run candidate: mixture weights + observed eval loss. Attributes: weights: Per-dataset weights summing to 1.0 ± 1e-6. ev

One repo's trust-on-first-use fingerprint.

High-level wrapper for continued pre-training from SoupConfig. Pre-training uses raw text data (no instruction/response structure). Each samp

A per-base bundle of probes.

Thread-safe ring of the most-recently observed GRPO step signal. Stores the latest completions (as strings) and the per-reward-function rewar

One per-user scaling vector in the bank.

Tracks checkpoint quality evaluations and determines the best one.

High-level wrapper for embedding model fine-tuning from SoupConfig. Supports contrastive, triplet, and cosine loss for training sentence embe

Canonical endpoint representation across all three spec kinds.

Run a mini benchmark periodically and report accuracy drops.

Discrete latent-space (key, value) store for sequential knowledge edits. Each edit appends a ``(key, value, label)`` triple. At decode time the

Aggregated results from human evaluation.