GR00T-WholeBodyControl

This is the codebase for the GR00T Whole-Body Control (WBC) projects. It hosts model checkpoints and scripts for training, evaluating, and deploying advanced whole-body controllers for humanoid robots. We currently support:

• Decoupled WBC: the decoupled controller (RL for lower body, and IK for upper body) used in NVIDIA GR00T N1.5 and N1.6 models;
• GEAR-SONIC Series: our latest iteration of generalist humanoid whole-body controllers (see our whitepaper);
• MotionBricks: a real-time latent generative model for interactive motion control in animation and robotics (see the project page).

News

• [2026-06-16] Low-latency SONIC release — added a low-latency G1 controller variant on Hugging Face under low_latency/. See the Download Models and VLA Inference docs for usage.
• [2026-05-07] 🤖 End-to-end VLA workflow on G1 — collect teleop data, fine-tune Isaac-GR00T N1.7, and deploy with SONIC whole-body control. See Data Collection, VLA Workflow, and VLA Inference.
• [2026-04-27] 🧩 MotionBricks preview — interactive G1 demo, pretrained checkpoints (VQVAE · pose · root), synthetic training code, and motion-representation docs. See motionbricks/ and the project page.
• [2026-04-14] 🌐 Live web demo — try SONIC interactively in your browser. Features Kimodo text-to-motion generation.
• [2026-04-10] 🚀 Released SONIC training code and checkpoint on HuggingFace. Train from scratch or finetune. Additional embodiment support and VLA data collection pipeline. See Training Guide.
• [2026-03-24] 🔧 C++ inference stack update: motor error monitoring, TTS alerts, ZMQ protocol v4, idle-mode readaptation. ZMQ header size changed to 1280 bytes.
• [2026-03-16] 📦 BONES-SEED open-sourced — 142K+ human motions (~288 hours) with G1 MuJoCo trajectories.
• [2026-02-19] 🎉 Released GEAR-SONIC: pretrained checkpoints, C++ inference, VR teleoperation, and documentation.
• [2025-11-12] 🏁 Initial release with Decoupled WBC for GR00T N1.5 and N1.6.

Table of Contents

• News
• GEAR-SONIC
• VR Whole-Body Teleoperation
• Kinematic Planner
• SONIC Training
• TODOs
• What's Included
• Setup
• Documentation
• Citation
• License
• Support
• MotionBricks
• Decoupled WBC

GEAR-SONIC

<a href="https://nvlabs.github.io/GEAR-SONIC/"><strong>Website</strong></a> | 
<a href="https://huggingface.co/nvidia/GEAR-SONIC"><strong>Model</strong></a> | 
<a href="https://arxiv.org/abs/2511.07820"><strong>Paper</strong></a> | 
<a href="https://nvlabs.github.io/GR00T-WholeBodyControl/"><strong>Docs</strong></a>

SONIC is a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data. Rather than building separate controllers for predefined motions, SONIC uses motion tracking as a scalable training task, enabling a single unified policy to produce natural, whole-body movement and support a wide range of behaviors — from walking and crawling to teleoperation and multi-modal control. It is designed to generalize beyond the motions it has seen during training and to serve as a foundation for higher-level planning and interaction.

In this repo, we release SONIC's training code, deployment framework, model checkpoints, and teleoperation stack for data collection.

The low-latency SONIC variant is available on Hugging Face under low_latency/. It keeps the default top-level deployment policy unchanged. Download it with python download_from_hf.py --low-latency.

For C++ deployment:

cd gear_sonic_deploy
./deploy.sh \
    --cp policy/low_latency/model \
    --obs-config policy/low_latency/observation_config.yaml \
    --input-type zmq_manager \
    real

For the Python VLA launcher:

python gear_sonic/scripts/launch_inference.py \
    --deploy-checkpoint policy/low_latency/model \
    --deploy-obs-config policy/low_latency/observation_config.yaml \
    --camera-host 192.168.123.164 \
    --prompt "pick up the cup"

VR Whole-Body Teleoperation

SONIC supports real-time whole-body teleoperation via PICO VR headset, enabling natural human-to-robot motion transfer for data collection and interactive control.

This repo can also drive the headset over Isaac Teleop / CloudXR by launching gear_sonic/scripts/pico_manager_thread_server.py --input-source isaac-teleop. The streamer hosts the CloudXR runtime in-process via isaacteleop[cloudxr] — no separate publisher container required. That path is currently documented and supported only for G1 with a Thor backpack. The Isaac Teleop bring-up steps are documented in docs/source/tutorials/isaac_teleop_publisher_setup.md.

Walking	Running
Sideways Movement	Kneeling
Getting Up	Jumping
Bimanual Manipulation	Object Hand-off

Kinematic Planner

SONIC includes a kinematic planner for real-time locomotion generation — choose a movement style, steer with keyboard/gamepad, and adjust speed and height on the fly.

In-the-Wild Navigation
Run	Happy
Stealth	Injured
Kneeling	Hand Crawling
Elbow Crawling	Boxing

SONIC Training

SONIC can be trained from scratch on the Bones-SEED motion capture dataset (142K+ motions, ~288 hours, Unitree G1 retargeted), or finetuned from the released checkpoint on Hugging Face.

Quick start

# Install training dependencies (Isaac Lab must be installed separately — see docs)
pip install -e "gear_sonic/[training]"

# Download checkpoint + SMPL data from Hugging Face
pip install huggingface_hub
python download_from_hf.py --training

# Download Bones-SEED G1 CSVs from huggingface.co/datasets/bones-studio/seed, then convert and filter
python gear_sonic/data_process/convert_soma_csv_to_motion_lib.py \
    --input /path/to/bones_seed/g1/csv/ \
    --output data/motion_lib_bones_seed/robot --fps 30 --fps_source 120 --individual --num_workers 16
python gear_sonic/data_process/filter_and_copy_bones_data.py \
    --source data/motion_lib_bones_seed/robot --dest data/motion_lib_bones_seed/robot_filtered

# Finetune from released checkpoint (64+ GPUs recommended)
accelerate launch --num_processes=8 gear_sonic/train_agent_trl.py \
    +exp=manager/universal_token/all_modes/sonic_release \
    +checkpoint=sonic_release/last.pt \
    num_envs=4096 headless=True \
    ++manager_env.commands.motion.motion_lib_cfg.motion_file=data/motion_lib_bones_seed/robot_filtered \
    ++manager_env.commands.motion.motion_lib_cfg.smpl_motion_file=data/smpl_filtered

For the full guide including multi-node training, evaluation, ONNX export, and SOMA encoder setup: 📖 Installation (Training) | Training Guide

TODOs

• [x] Release pretrained SONIC policy checkpoints
• [x] Open source C++ inference stack
• [x] Setup documentation
• [x] Open source teleoperation stack and demonstration scripts
• [x] Release training scripts and recipes for motion imitation and fine-tuning
• [x] Open source large-scale data collection workflows and fine-tuning VLA scripts.
• [x] Publish additional preprocessed large-scale human motion datasets

What's Included

This release includes:

• gear_sonic_deploy: C++ inference stack for deploying SONIC policies on real hardware
• gear_sonic: Full SONIC training stack — PPO training, data processing pipeline, and configuration system for training on Bones-SEED and custom motion datasets
• motionbricks: Preview release of the MotionBricks real-time latent generative stack — interactive G1 demo, pretrained checkpoints, synthetic training code, and motion-representation docs

Setup

Git LFS required. This repo contains large binary assets (meshes, ONNX models). Without Git LFS, you will get small pointer files instead of actual data, causing silent failures. Install Git LFS first if you don't have it: sudo apt install git-lfs && git lfs install

MotionBricks pretrained checkpoints are skipped by default to avoid an extra ~2.2 GiB download during normal monorepo setup. MotionBricks GIFs and meshes still download normally. Fetch the checkpoints explicitly if you plan to run the MotionBricks demo.

git clone https://github.com/NVlabs/GR00T-WholeBodyControl.git
cd GR00T-WholeBodyControl
git lfs pull

# Optional: fetch MotionBricks pretrained checkpoints.
git lfs pull --include="motionbricks/out/**" --exclude=""

# Verify your environment
python check_environment.py

Which environment do I need?

Each use case has its own lightweight environment. The install scripts use uv and create isolated venvs automatically — you don't need to manage them manually. Training is the only one that requires