GR00T-WholeBodyControl

This is the codebase for the GR00T Whole-Body Control (WBC) projects. It hosts model checkpoints and scripts for training, evaluating, and deploying advanced whole-body controllers for humanoid robots. We currently support:

• Decoupled WBC: the decoupled controller (RL for lower body, and IK for upper body) used in NVIDIA GR00T N1.5 and N1.6 models;
• GEAR-SONIC Series: our latest iteration of generalist humanoid whole-body controllers (see our whitepaper);
• MotionBricks: a real-time latent generative model for interactive motion control in animation and robotics (see the project page).

News

• [2026-07-23] SONIC v1.1 checkpoint — released a robot-heading-normalized SONIC controller trained with wrist-pose augmentation for whole-body teleoperation and SONIC-backed VLA execution. See the Model Card and Download Models.
• [06/16] Isaac Teleop Setup (CloudXR / DeviceIO, in-process) — added bring-up docs for the in-process CloudXR path via isaacteleop[cloudxr], with no separate publisher container. See Isaac Teleop Setup.
• [2026-06-16] Low-latency teleoperation checkpoint — released a SONIC checkpoint with 4-frame SMPL reference lookahead for more responsive whole-body teleoperation. See the Model Card, Download Models, and VLA Inference.
• [2026-05-07] 🤖 End-to-end VLA workflow on G1 — collect teleop data, fine-tune Isaac-GR00T N1.7, and deploy with SONIC whole-body control. See Data Collection, VLA Workflow, and VLA Inference.
• [2026-04-27] 🧩 MotionBricks preview — interactive G1 demo, pretrained checkpoints (VQVAE · pose · root), synthetic training code, and motion-representation docs. See motionbricks/ and the project page.
• [2026-04-14] 🌐 Live web demo — try SONIC interactively in your browser. Features Kimodo text-to-motion generation.
• [2026-04-10] 🚀 Released SONIC training code and checkpoint on HuggingFace. Train from scratch or finetune. Additional embodiment support and VLA data collection pipeline. See Training Guide.
• [2026-03-24] 🔧 C++ inference stack update: motor error monitoring, TTS alerts, ZMQ protocol v4, idle-mode readaptation. ZMQ header size changed to 1280 bytes.
• [2026-03-16] 📦 BONES-SEED open-sourced — 142K+ human motions (~288 hours) with G1 MuJoCo trajectories.
• [2026-02-19] 🎉 Released GEAR-SONIC: pretrained checkpoints, C++ inference, VR teleoperation, and documentation.
• [2025-11-12] 🏁 Initial release with Decoupled WBC for GR00T N1.5 and N1.6.

Table of Contents

• News
• GEAR-SONIC
• Model Card
• VR Whole-Body Teleoperation
• Kinematic Planner
• SONIC Training
• TODOs
• What's Included
• Setup
• Documentation
• Citation
• License
• Support
• MotionBricks
• Decoupled WBC

GEAR-SONIC

<a href="https://nvlabs.github.io/GEAR-SONIC/"><strong>Website</strong></a> | 
<a href="https://huggingface.co/nvidia/GEAR-SONIC"><strong>Model</strong></a> | 
<a href="https://arxiv.org/abs/2511.07820"><strong>Paper</strong></a> | 
<a href="https://nvlabs.github.io/GR00T-WholeBodyControl/"><strong>Docs</strong></a>

SONIC is a humanoid behavior foundation model that gives robots a core set of motor skills learned from large-scale human motion data. Rather than building separate controllers for predefined motions, SONIC uses motion tracking as a scalable training task, enabling a single unified policy to produce natural, whole-body movement and support a wide range of behaviors — from walking and crawling to teleoperation and multi-modal control. It is designed to generalize beyond the motions it has seen during training and to serve as a foundation for higher-level planning and interaction.

In this repo, we release SONIC's training code, deployment framework, model checkpoints, and teleoperation stack for data collection.

Model Card

SONIC provides three released Unitree G1 checkpoints. Choose the model based on its reference representation and intended deployment.

Available Models

All three models use the SONIC universal-token controller, produce 64-dimensional latent motion tokens, run the controller at 50 Hz, and support SMPL pose, G1 motion reference, and teleoperation inputs. Deployment uses C++ and TensorRT; the PyTorch checkpoints support Isaac Lab evaluation and continued training.

The lookahead values describe the reference horizon presented to the controller. They are not measurements of total end-to-end teleoperation latency, which also includes sensing, networking, preprocessing, and inference. Model weights are covered by the NVIDIA Open Model License.

Released Files

Usage

Download the default model and planner:

python download_from_hf.py

Download the low-latency teleoperation model and planner:

python download_from_hf.py --low-latency

Download SONIC v1.1 and the planner:

python download_from_hf.py --sonic-v1-1

Run the default C++ deployment stack:

cd gear_sonic_deploy
./deploy.sh --input-type zmq_manager real

Run the low-latency C++ deployment stack:

cd gear_sonic_deploy
./deploy.sh \
    --cp policy/low_latency/model \
    --obs-config policy/low_latency/observation_config.yaml \
    --input-type zmq_manager \
    real

Run the SONIC v1.1 C++ deployment stack:

cd gear_sonic_deploy
./deploy.sh \
    --cp policy/sonic_v1_1/model \
    --obs-config policy/sonic_v1_1/observation_config.yaml \
    --input-type zmq_manager \
    real

Run the default Python VLA launcher, which orchestrates the C++ controller and Python inference client:

python gear_sonic/scripts/launch_inference.py \
    --camera-host 192.168.123.164 \
    --prompt "pick up the cup"

For the low-latency model, add the matching deployment files:

python gear_sonic/scripts/launch_inference.py \
    --deploy-checkpoint policy/low_latency/model \
    --deploy-obs-config policy/low_latency/observation_config.yaml \
    --camera-host 192.168.123.164 \
    --prompt "pick up the cup"

For SONIC v1.1, use policy/sonic_v1_1/model and its matching policy/sonic_v1_1/observation_config.yaml in the same launcher flags.

See Downloading Model Checkpoints for Python checkpoint evaluation and additional deployment options. Test in simulation before using the checkpoint on a physical robot.

VR Whole-Body Teleoperation

SONIC supports real-time whole-body teleoperation via PICO VR headset, enabling natural human-to-robot motion transfer for data collection and interactive control.

SONIC v1.1: Whole-body teleoperation mode with expressive wrist motion and dynamic movement.

SONIC Low Latency: Whole-body teleoperation mode with a successful ground pickup.

This repo can also drive the headset over Isaac Teleop / CloudXR by launching gear_sonic/scripts/pico_manager_thread_server.py --input-source isaac-teleop. The streamer hosts the CloudXR runtime in-process via isaacteleop[cloudxr] — no separate publisher container required. That path is currently documented and supported only for G1 with a Thor backpack. The Isaac Teleop bring-up steps are documented in docs/source/tutorials/isaac_teleop_publisher_setup.md.

Walking	Running
Sideways Movement	Kneeling
Getting Up	Jumping
Bimanual Manipulation	Object Hand-off

Kinematic Planner

SONIC includes a kinematic planner for real-time locomotion generation — choose a movement style, steer with keyboard/gamepad, and adjust speed and height on the fly.

In-the-Wild Navigation
Run	Happy
Stealth	Injured
<