Mimic Intent, Not Just Trajectories

An intent-to-execution policy for precise and transferable robotic manipulation.

<a href="https://arxiv.org/abs/2602.08602"><img alt="arXiv" src="https://img.shields.io/badge/arXiv-2602.08602-b31b1b?style=for-the-badge&logo=arxiv&logoColor=white"></a>
<a href="https://huggingface.co/huangrm/MINT-libero-130"><img alt="Hugging Face Policy" src="https://img.shields.io/badge/HuggingFace-Policy-ffca28?style=for-the-badge&logo=huggingface&logoColor=black"></a>
<a href="https://huggingface.co/huangrm/MINT-tokenizer-libero-130"><img alt="Hugging Face Tokenizer" src="https://img.shields.io/badge/HuggingFace-Tokenizer-ffca28?style=for-the-badge&logo=huggingface&logoColor=black"></a>







<img alt="Python" src="https://img.shields.io/badge/Python-3.12-3776AB?style=for-the-badge&logo=python&logoColor=white" />
<img alt="Status" src="https://img.shields.io/badge/Status-Released-16a34a?style=for-the-badge" />
<img alt="License" src="https://img.shields.io/badge/License-MIT-16a34a?style=for-the-badge" />

SDAT maps each action chunk into multi-scale tokens: coarse tokens capture intent, and fine tokens capture execution details. The S1 token space forms behavior-level clusters.

MINT predicts tokens from intent to execution with next-scale autoregression, then decodes them into actions. Intent-based ensemble improves long-horizon stability.

Overview

We introduce MINT (Mimic Intent, Not just Trajectories), a framework for end-to-end imitation learning in dexterous manipulation. MINT explicitly disentangles behavior intent from execution details by learning a hierarchical, multi-scale token representation of actions. Coarse tokens capture global, low-frequency intent, while finer tokens encode high-frequency execution details. Our policy generates trajectories via next-scale autoregression, performing progressive intent-to-execution reasoning. This structure enables efficient learning, robust adaptation to environmental dynamics, and one-shot skill transfer by reusing the intent token from a demonstration. Experiments on simulation and real robots demonstrate strong performance, high generalization, and effective skill transfer.

Open-Source Roadmap

Installation

LeRobot compatibility

The current main branch supports LeRobot 0.5.1. Use the legacy branch only when an existing environment must remain on LeRobot 0.4.3; the two integrations should not be mixed.

conda create -y -n mint python=3.12 cmake=3.11
conda activate mint

pip install "lerobot[pi]==0.5.1"
# Install LIBERO dependencies via LeRobot:
pip install "lerobot[libero]==0.5.1"

# Install all MINT policy runtime dependencies:
pip install -r requirements.txt

conda install -y ffmpeg -c conda-forge

# install policy
pip install -e ./policy/lerobot_policy_mint
pip install -e ./policy/lerobot_policy_mint_light

Note: If you encounter build errors on Linux, you may also need system packages such as cmake, build-essential, python3-dev, pkg-config, and FFmpeg development libraries.

apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev

Model Zoo

The policy directory contains two independent packages with matching pyproject.toml plus src/<package>/ layouts. Install the package you need, then choose --policy.type=mint or --policy.type=mint_light; both use the standard LeRobot commands.

MINT-4B (LeRobot implementation) 🤗

MINT-Light / MINT-30M ⚡

• MINT-4B (mint) uses the PaliGemma backbone and action expert for maximum capacity.
• MINT-Light (mint_light) keeps MINT's coarse-to-fine action-token prediction but replaces the large VLM with a fixed DINOv3 ViT-L/16 visual encoder, a SigLIP2 text encoder.

Training Example

First, download the required tokenizer:

hf download huangrm/MINT-tokenizer-libero-130 --local-dir <path/to/tokenizer>

Or, train your own tokenizer:

# install tokenizer training dependencies
pip install -r requirements.txt
python -m SDAT.train --config-name train

Start MINT-4B training:

accelerate launch \
    --multi_gpu \
    --num_processes=2 \
    $(which lerobot-train) \
    --dataset.repo_id=HuggingFaceVLA/libero \
    --policy.type=mint \
    --output_dir=<path/to/output> \
    --job_name=mint_training \
    --policy.repo_id=mint \
    --policy.pretrained_path=huangrm/pi05_base \
    --policy.vqvae_name_or_path=<path/to/tokenizer-checkpoint> \
    --policy.compile_model=false \
    --policy.gradient_checkpointing=true \
    --policy.dtype=float32 \
    --steps=100000 \
    --save_freq=20000 \
    --policy.device=cuda \
    --batch_size=16

For MINT-Light, use the same command and change the policy type. you can reuse the same tokenizer checkpoint from above, or train a new one with the SDAT training script.

lerobot-train \
    --dataset.repo_id=<dataset-repo-id> \
    --policy.type=mint_light \
    --policy.use_language=true \
    --policy.vqvae_name_or_path=<path/to/tokenizer-checkpoint> \
    --policy.device=cuda \
    --output_dir=<path/to/output>

use_language=false trains the visual-only MINT-Light policy used by one-shot transfer.

Evaluation

lerobot-eval \
    --policy.path=huangrm/MINT-libero-130 \
    --policy.vqvae_name_or_path=<path/to/tokenizer-checkpoint> \
    --env.type=libero \
    --env.task=libero_object,libero_10,libero_goal,libero_spatial \
    --eval.batch_size=1 \
    --eval.n_episodes=50 \
    --seed=1000 \
    --policy.n_action_steps=4

Evaluation is identical for MINT-Light: point --policy.path at a local pretrained_model directory or a Hugging Face model ID. LeRobot reads the saved policy type automatically.

One-Shot Transfer

Set the policy and VQ-VAE references to their Hugging Face model IDs. Both are downloaded and cached automatically by from_pretrained:

CHECKPOINT=huangrm/MINT-light-zero
VQVAE=huangrm/MINT-tokenizer-libero-130

You can also use existing local paths:

CHECKPOINT=/path/to/pretrained_model
VQVAE=/path/to/tokenizer/ms_vqvae.pth

The three evaluation demonstrations are already included in transfer/demo_traj. Run any of the transfer tests with the selected model:

# New task
bash transfer/scripts/transfer_new_task.sh "$CHECKPOINT" "$VQVAE"

# New layout
bash transfer/scripts/transfer_new_layout.sh "$CHECKPOINT" "$VQVAE"

# Extended horizon
bash transfer/scripts/transfer_extend_horizon.sh "$CHECKPOINT" "$VQVAE"

The scripts save results and trajectory videos under outputs/transfer_*. Set options such as SEED, NUM_EPISODES, or SAVE_VIDEO before the command to override their defaults:

SEED=42 NUM_EPISODES=5 SAVE_VIDEO=1 \
bash transfer/scripts/transfer_new_task.sh "$CHECKPOINT" "$VQVAE"

See transfer/README.md for the concise transfer usage guide.

Citation

If you find this project useful, please cite:

@article{huang2026mimic,
  title={Mimic Intent, Not Just Trajectories},
  author={Huang, Renming and Zeng, Chendong and Tang, Wenjing and Cai, Jintian and Lu, Cewu and Cai, Panpan},
  journal={arXiv preprint arXiv:2602.08602},
  year={2026}
}

Acknowledgement

This project is built on top of excellent open-source ecosystems. We sincerely thank the teams behind LeRobot and OpenPI for their impactful contributions.