Lossless codec for valid binary sequences with continuity constraints. A binary sequence is *valid* when every middle run (neither first nor

Template segment descriptor supporting static text and dynamic placeholders.

Unified inference wrapper: obs_dicts in → action dicts out.

Dict-based KV cache for hybrid architectures (e.g., Qwen3.5). Only full_attention layers produce KV cache; linear_attention layers (GatedDelt

RMSNorm with (1 + weight) scaling. Weight initialized to zeros.

Base class for action filters that mark operational dimensions. Action filters are NOT invertible - they modify action_op_mask which can

During training, randomly choose an annotated candidate builder by weight. During inference, use eval_builder. If no candidate builder has annota

Single-field linear normalizer with forward/backward transforms. Forward: normalize data according to mode (z-score, min/max, quantile, etc.

Exception raised when trying to look ahead in the future of a Backtrackable object.

Dict that returns None for missing keys without inserting them. Used by SparseKVCache for GatedDeltaNet conv/recurrent states: - Prefill: key

Block-wise AR training data formatter, independent of VQActionTokenizer.

Backend self-description for VQActionTokenizer decode decisions. VQActionTokenizer reads only this dataclass — it never probes backend intern

Use high_level_instruction as command, then CoT outputs atomic_task before action. Data source (r1lite/r1pro _merged_final_v30): data["hi

Data input preprocessor for tokenizing text, images, proprio, and actions. Template placeholder syntax: <sampleKey_processorKey>

Multi-field linear normalizer for action/state data. Forward: normalize each field according to its mode Backward: denormalize each fiel

Exception raised when trying to look back in the history of a Backtrackable object.

Clamp or soft-remap the model-frame state before sending to the server. Prevents out-of-distribution proprio inputs from collapsing the model's

Robot scheduler — sends obs to server, receives single-step action, publishes. The server (serve_policy) decides when to recompute vs serve from

Dict subclass that allows attribute access (like HF BatchEncoding).

Autoregressive algorithm helper. Args: cfg: model_arch.ar config, containing: - ce_weight: CE loss weight - vocab

Compute token-level accuracy and masked L1 loss for AR action predictions. Args: action_tokenizer: Action tokenizer instance. tok

Configuration for ActionCodecV2Model. ActionCodecV2 is a universal action tokenizer that accepts an arbitrary dict of action components

Action tokenizer model based on FasterV2 architecture with generic dict input. All keys share the same encoder/decoder/RVQ weights because a

Mirror of ActionEncoder: 1×1 proj from latent_dim, then UpBlock2D blocks (in reverse stride order).

Stack of DownBlock2D blocks followed by a 1×1 conv projection to latent_dim. Channel dims: encoder_channels → encoder_channels*c_mults[0]

Manages key routing, merge_spec, and dimension chunking. Pipeline (encode direction):: component_dict → partition → merge → chunk Pi

Match BaseSamplesBuilder template but inject atomic_task into the command slot. This builder has no CoT and goes directly to action. Data so

Multi-head self-attention with: - RoPE positional encoding (partial) - Per-head QK-LayerNorm for numerical stability - Flash Attentio

Proxy to satisfy ALL_ATTENTION_FUNCTIONS interface signature.

BBox CoT: the model outputs object bboxes before action. Data source (lerobot_dataset_v3.py): data["bbox"] <- bbox_index looks up the tas

Joint bbox + subtask CoT: output bbox, then subtask, then action. Template (embodiment=r1, 2 cameras, discrete): <image0_image_!><image1_

Block DCT-II with orthonormal normalization. Applies DCT independently on non-overlapping blocks of ``block_size`` frames along the time

Affordance CoT processor: format pixel coordinates as text with COT_TOKEN_INDEX.

MLP proprio processor with sample-level null dropout.

Placeholder processor for the MLP proprio encoder. Emits only N special `<state>` tokens as placeholders, where N=num_obs_steps is inferred d

Drop proprio placeholders from the token stream.

Qwen3.5 image processor: <|vision_start|> + <|image_pad|>*N + <|vision_end|>.

Default text processor: encode strings into text tokens.

Background-thread OpenCV camera capture.

Wraps model inference with chunk caching, step-by-step serving, and frame buffers. Maintains per-camera frame buffers (deque) for multi-frame mod

Wraps model inference with chunk caching and step-by-step serving. Decides when to call the model (recompute) vs serve from cache. One instan

Wraps batch inference with chunk caching and step-by-step serving. Decides when to call the model (recompute) vs serve from cache. One instan

Generates positive (augmented) views of action sequences for consistency training. All parameters vary **linearly** from begin→end over the

Accumulate check results with PASS/FAIL/WARN/INFO.

Encoder building block: Conv2d(stride_h, stride_a) → N × TransformerBlock The Conv2d downsamples the H (time-patch) axis; stride_a=1

DROID dataset on the unified explicit schema. Gripper flip (`1 - x`) is handled in `_slice_meta_feature`, so sample path and fast-stats

Collects inference requests and dispatches them as micro-batches. A single asyncio task (_worker_loop) consumes from an asyncio.Queue. Reques

Single-level EMA vector quantizer with: - K-means initialisation (first forward pass) - EMA codebook updates (no gradient through codeboo

Chunked serving with temporal ensemble of overlapping predictions. Instead of discarding predictions beyond action_steps, stores all overlapp

Record one episode at a time; call stop_and_save() between episodes.

Record one episode at a time; call stop_and_save() between episodes.

Flow Matching algorithm helper. Args: cfg: model_arch.fm config, containing: - time_convention: "pi_convention" | "galaxea_co

Finite Scalar Quantization (FSQ) drop-in replacement for EMAVectorQuantize. Instead of learning a codebook via EMA, FSQ quantizes each dimen

Single-part backend: codes → flat token sequence.

Single background thread owns the serial bus. Ported from molmoact2-so101/molmoact_so101/setup/robot.py. All external callers use set_target(

Feed-forward network using GEGLU activation with full 4× expansion. GEGLU: ``GEGLU(x) = x1 * GELU(x2)`` where ``(x1, x2) = split(W_up(x))``.

Hack for inner model layers to pass some monitoring info to the trainer.

Layout information for one group.

Extends FlatSerializer with per-group marker tokens. The flat VQ code sequence has a specific structure: - NN portion: level-first serializ

Layout information for one alternative name in a two-level merge_spec.

Async loop at action_fps Hz: send obs/cache request → set_target(). Follows the server's need_obs flag: - need_obs=True → send full obs (ima

Unified state for three-stage inference. prefill() creates the initial state, generate_text() updates it and appends generated results, and g

A namespace object that supports both dictionary-like iteration and dot notation access. Automatically converts nested dictionaries into Iter

Model FLOPS Utilization (MFU) Tracker Tracks and calculates the hardware utilization during training by comparing actual FLOPS achieved

G05 attention mask + position ID builder (weight-free pure computation). Attributes: position_ids_type: one of "lyc" | "pi0fast" | "gauss

One cv2 window per camera. cot → logger; bbox → head window.

Memory input + memory_update CoT output. Data source (lerobot_dataset_v3.py): data["memory"] <- prev_memory_index looks up the tas

Memory input + subtask CoT + memory_update CoT output. Data fields: data["memory"] <- prev_memory_index (input, masked) da

Qwen3.5 full attention with: - q_proj outputs 2x dim (query + gate) - QK norm (Qwen3_5RMSNorm on head_dim) - partial_rotary_factor=0.25 vi

Qwen3.5 decoder layer with 2 norms (standard pre-norm). For full_attention layers: uses MixtureAttentionQwen35 For linear_attention layers: u

A dataset consisting of multiple underlying `LeRobotDataset`s. The underlying `LeRobotDataset`s are effectively concatenated, and this class adop

State produced by encode_partition, consumed by decode_partition.

Runs one eval batch periodically during training and returns log metrics.

Plan input (masked) + plan step CoT output. Data source (lerobot_dataset_v3.py): data["plan"] <- plan_index looks up the tasks table

Preprocessed sample plus its sub_processor reference.