Select the positions belonging to one meta from a shared query tensor. Args: data: Tensor returned by `MultiLeRobotDatas

Aggregate stats from multiple compute_stats outputs into a single set of stats. The final stats will have the union of all data keys from each of

Common build flow; subclasses do not need to override it.

Build the processor.preprocess input from client raw observations. Data format is aligned with base_lerobot_dataset.py:304-323 __getitem__ output

Input projection. VLM: input_ids [B,S] → [B,S,d] (Embedding, NO sqrt scaling for Qwen3.5) AE: psi_t [B,H,D] → [B,H,d] (Linear, flo

Compute dataset statistics for normalization. Args: processor: MixtureProcessor containing per-embodiment processors

Given a task in natural language, returns its task_index if the task already exists in the dataset, otherwise return None.

Instantiate dataset from cfg.data, stripping processors first. cfg.data may contain a ``processors`` key that is consumed by :func:`build_pro

Return True iff *sequence* is a valid binary sequence under this tokenizer's constraints.

Convert rotations given as rotation matrices to quaternions. Args: matrix: Rotation matrices as tensor of shape (..., 3, 3). Re

Postprocess the data for the policy model. Args: data: Dict[str, Any], lerobot sample in raw mcap Returns:

Preprocess input data. Args: training: whether this is training mode; affects action corruption and alignment.

Resolve a local directory or HF model repo id to a local directory path. Existing local directories are returned unchanged. HF model repo ids are

Set processor instance from external initialization.

Override to silently ignore train(True) after configure_eval().

Truncate dataset_dirs to at most *max_datasets* entries (debug mode). Supports both mixture configs (embodiment_datasets → dataset_groups → datas

Split flat tensor back to per-embodiment raw key dict.

Text progress bar.

Precompute per-frame episode end index for sliding window operations. Args: ep_indices: Episode index per frame, shape (N,)

Concatenate aligned dict into a single tensor following max_shape_meta key order. Args: aligned_dict: Aligned data dict

Flatten nested dict: {"a": {"b": tensor}} -> {"a.b": tensor}.

Format threshold value for display. Uses scientific notation for values < 0.1, otherwise decimal.

Align raw keys, apply merge_spec grouping, concat to flat tensor.

Extract the pad mask corresponding to one state/action query. Args: meta: State or action meta with resolved query infor

Restore to original keys and dimensions, discard padding and virtual keys

Split concatenated tensor back to aligned dict following max_shape_meta key order. Args: concatenated_tensor: Concatenat

Append text to the log file if one has been configured.

Cast parameters matching fp32_param_patterns to float32.

Encode a binary sequence to a list of ``num_tokens`` token integers. Parameters ---------- sequence: A b

Estimate FLOPs for one sample through this Mixture (forward + backward). Uses the standard 6*N*S formula for linear projections (fwd+bwd), pl

Build a complete manager for a given VLM backbone. Registers bos/eos/pad/image from *tokenizer* and fills chat format tokens accordin

Get file size on disk in megabytes. Args: file_path (Path): Path to the file.

AR inference: last_hidden -> sample -> decode loop. The caller must run vlm_prefill() first to provide past_key_values and last_hidde

Load a model and apply checkpoint weights, optionally using meta-device acceleration. Compared with the old three-step path (instantiate -> torch

Single AR training step. Args: model: G05Model vlm_hidden: [B, S, d_vlm] labels: [B, S] Returns:

Align dict keys and dimensions to max_shape_meta. Processing logic: 1. key exists + insufficient dimension → pad to target_d

Map generated token IDs -> TOKEN_INDEX for attention_mask growth. Aligned with legacy GalaxeaARMixin.assign_position_ids_by_ranges. W

Remove block-wise AR markers and control tokens from decoded string.

Concatenate tensors in dict according to meta order. Supports both 2D (T, D) for action and 1D (D,) for action_op_mask.

Crop padding portion, keep dimensions defined by meta. Supports 3D (B, T, D) for action and 2D (B, D) for action_op_mask.

One-line description of a value.

Encode a single padded tensor through the full pipeline up to RVQ. Args: x: (B, T, max_component_dim) —

Write all buffered episode metadata to parquet file.

Return standard quantile specification list.

Extract 'samples' sub-dict from preprocess output.

Register flat action tokens (no group markers).

Create a semi-transparent merged-chunk trace for a single action dimension. Args: chunks_1d: ``(episode_size, chunk_size)`` — one dim sli

Pad right to specified dimension, returns padded tensor and padding mask. Supports both 2D (T, D) for action and 1D (D,) for action_op_mask.

Pad or clip the last dimension to ``max_d``.