Merges the loaded parameters with the reference parameters. Args: loaded_params: The parameters to merge. params: The reference p

解析LeRobot格式图像为模型输入格式,严格对应图像转换逻辑

解析LeRobot格式图像为模型输入格式,严格对应图像转换逻辑

Helper method to preprocess observation.

Connect to server and start background receiver.

create random image as a fallback

Create a random image as fallback.

Disconnect and cleanup pending requests.

Embed images with SigLIP and language tokens with embedding layer to prepare for PaliGemma transformer processing.

Embed state, noisy_actions, timestep to prepare for Expert Gemma processing.

Generate subtask

Get a config by name.

Obtains image last hidden states from the vision tower and apply multimodal projection. Args: pixel_values (`torch.Float

初始化推理引擎

Asynchronously initialize model.

Returns the input specification for the model. Values are jax.ShapeDtypeStruct.

Adapted from big_vision. Tokens can attend to valid inputs tokens which have a cumulative mask_ar smaller or equal to theirs. This way `mask_

This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, he

Rotates half the hidden dims of the input.

Start websocket server.

Convert the Observation to a nested dict.

Unflatten a flattened dictionary. Assumes that '/' was used as a separator.

Adjust histograms when min or max changes.

Compute quantiles based on histograms.

Download a file from a remote filesystem to the local cache, and return the local path.

Get bins per dimension according to the Lookup-Free Quantization paper (2 bins per dimension)

Initializes and returns the LIBERO environment, along with the task description.

Initializes and returns the LIBERO environment, along with the task description.

Initializes and returns the LIBERO environment, along with the task description.

Periodic subtask refresh loop for a connected client.

Initialize KV cache with padding to cache_size. Args: k: Key tensor of shape (batch, num_heads, seq_len, head_dim) v:

Initialize KV cache

Initialize KV cache

Loads and validates the weights. Returns a loaded subset of the weights.

Log normalization values used at inference time.

Log normalization values used at inference time.

Action generation callback function, can be overridden by subclasses

Action generation callback function, can be overridden by subclasses.

Subtask refresh callback function, can be overridden by subclasses

Subtask refresh callback function, can be overridden by subclasses.

Background task for periodic subtask refresh.

Copied from robosuite: https://github.com/ARISE-Initiative/robosuite/blob/eafb81f54ffc104f905ee48a16bb15f059176ad3/robosuite/utils/transform_util

Copied from robosuite: https://github.com/ARISE-Initiative/robosuite/blob/eafb81f54ffc104f905ee48a16bb15f059176ad3/robosuite/utils/transform_util

Copied from robosuite: https://github.com/ARISE-Initiative/robosuite/blob/eafb81f54ffc104f905ee48a16bb15f059176ad3/robosuite/utils/transform_util

Background receiver that routes messages by request_id and refresh type.

Route websocket messages to request futures or refresh queue.

Set to position mode and do position resets: first close then open. Then change back to PWM mode NOTE: This diverges from the original Aloha

Replicates tf.image.resize_with_pad for one image using PIL. Resizes an image to a target height and width without distortion by padding with zero

Runs a single episode.

Save raw + model-input images to verify preprocessing alignment.

Save raw + model-input images to verify preprocessing alignment.