Keys to access image and video stream from cameras.

Keys to access image and video stream from cameras.

The returned observations do not have a batch dimension.

The returned observations do not have a batch dimension.

Open a on-disk zarr directly (for dataset larger than memory). Slower.

Run the batch through the model and compute the loss for training or validation. Returns a dictionary with "loss" and potentially other infor

Run the batch through the model and compute the loss for training or validation.

Args: features: (B, C, H, W) input feature maps. Returns: (B, K, 2) image-space coordinates of keypoints.

Args: x: (B, C, H, W) image tensor with pixel values in [0, 1]. Returns: (B, D) image feature.

Args: x: (B, T, input_dim) tensor for input to the Unet. timestep: (B,) tensor of (timestep_we_are_denoising_from - 1

Args: x: (B, in_channels, T) cond: (B, cond_dim) Returns: (B, out_channels, T)

Run the batch through the model and compute the loss for training or validation.

Args: features: (B, C, H, W) input feature maps. Returns: (B, K, 2) image-space coordinates of keypoints.

Args: x: (B, C, H, W) image tensor with pixel values in [0, 1]. Returns: (B, D) image feature.

For given input tensor x, this function will return the quantized output, the indices of the quantized output, and the loss. First, t

Run the batch through the model and compute the loss. Returns a dictionary with loss as a tensor, and other information as native floats.

Encode the image and/or state vector. Each modality is encoded into a feature vector of size (latent_dim,) and then a uniform mean is taken

Run the batch through the model and compute the loss for training or validation.

A forward pass through the Action Chunking Transformer (with optional VAE encoder). `batch` should have the following structure: {

Args: x: (Decoder Sequence, Batch, Channel) tensor of input tokens. encoder_out: (Encoder Sequence, B, C) output feat

Args: x: A (B, C, H, W) batch of 2D feature map to generate the embeddings for. Returns: A (1, C, H, W) batch

Frames per second used during data collection.

Frames per second used during data collection. NOTE: Fow now, this relies on a check in __init__ to make sure all sub-datasets have the same

Check if the robot is in freedrive mode. Returns: bool: True if the robot is in freedrive mode, False otherwise.

Get a transform function that convert items from Hugging Face dataset (pyarrow) to torch tensors. Importantly, images are converted from PIL, whic