ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization

<a href='https://shenwenhao01.github.io/' target='_blank'>Wenhao Shen</a>&emsp;
<a href='https://scholar.google.com/citations?user=zlIJwBEAAAAJ&hl=en' target='_blank'>Wanqi Yin</a>&emsp;
<a href='https://xfyang.net/' target='_blank'>Xiaofeng Yang</a>&emsp;
<a href='https://scholar.google.com/citations?user=nNQU71kAAAAJ&hl=zh-CN' target='_blank'>Cheng Chen</a>&emsp;
<a href='https://chaoyuesong.github.io/' target='_blank'>Chaoyue Song</a>&emsp;
<a href='https://caizhongang.github.io/' target='_blank'>Zhongang Cai</a>&emsp;
<a href='https://yanglei.me/' target='_blank'>Lei Yang</a>&emsp;
<a href='https://wanghao.tech/' target='_blank'>Hao Wang</a>&emsp;
<a href='https://guosheng.github.io/' target='_blank'>Guosheng Lin</a>&emsp;

ICML 2025

🛠️ Install

Set up the environment

conda create -n adhmr python=3.8 -y
conda activate adhmr
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.3 -c pytorch -y
pip install mmcv-full==1.7.1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html
pip install -r requirements.txt

# install pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

# install mmpose
cd HMR-Scorer/main/transformer_utils
pip install -v -e .
cd -

HMR-Scorer

• cd HMR-Scorer/
• download all datasets
• 3DPW
• Human3.6M
• HI4D
• BEDLAM
• RenBody (DNA-Rendering)
• GTA-Human
• SPEC
• InstaVariety
• process all datasets into HumanData format, except the following:
• Human3.6M.
• follow OSX in preparing this dataset.
• follow OSX in preparing pretrained ViTPose models. Download the ViTPose pretrained weights for ViT-small and ViT-huge from here.
• download SMPL-X and SMPL body models.
• download HMR-Scorer test datasets from OneDrive (DNA-Rendering and GTA-Human) and put them under dataset/cache_scorer_eval.

The file structure should be like:

HMR-Scorer/
├── common/
│   └── utils/
│       └── human_model_files/  # body model
│           ├── smpl/
│           │   ├──SMPL_NEUTRAL.pkl
│           │   ├──SMPL_MALE.pkl
│           │   └──SMPL_FEMALE.pkl
│           └── smplx/
│               ├──MANO_SMPLX_vertex_ids.pkl
│               ├──SMPL-X__FLAME_vertex_ids.npy
│               ├──SMPLX_NEUTRAL.pkl
│               ├──SMPLX_to_J14.pkl
│               ├──SMPLX_NEUTRAL.npz
│               ├──SMPLX_MALE.npz
│               └──SMPLX_FEMALE.npz
├── data/
├── main/
├── output/
├── pretrained_models/  # pretrained ViT-Pose, SMPLer_X and mmdet models
│   ├── smpler_x_s32.pth.tar
│   ├── smpler_x_b32.pth.tar
│   ├── smpler_x_l32.pth.tar
│   ├── smpler_x_h32.pth.tar
│   ├── vitpose_small.pth
│   ├── vitpose_base.pth
│   ├── vitpose_large.pth
│   └── vitpose_huge.pth
└── dataset/  
    ├── 3DPW/       
    ├── Human36M/             
    ├── HI4D
    ├── BEDLAM/      
    ├── RenBody/      
    ├── GTA_Human2/           
    ├── CHI3D/       
    ├── InstaVariety/         
    ├── SPEC/   
    ├── cache_scorer_eval/      # HMR-Scorer test datasets
    └── preprocessed_datasets/  # HumanData files

ADHMR

• cd ADHMR/
• follow ScoreHypo to prepare data.

🚀 Pretrained Models

• download ckeckpoint of HMR-Scorer from OneDrive
• put it under HMR-Scorer/output/
• download checkpoint of ADHMR from OneDrive
• put them under ADHMR/experiment/hyponet

📝 Evaluation

HMR-Scorer

# To eval the model HMR-Scorer/output/{TRAIN_OUTPUT_DIR}
export PYTHONPATH=$PYTHONPATH:/path/to/ADHMR/HMR-Scorer
cd ./HMR-Scorer/main/
JOB_NAME=GTA_Human2 / RenBody_HiRes
torchrun test_scorer.py --num_gpus 1 --exp_name output/scorer_test_${JOB_NAME} --result_path train_scorer_b5_2d_1118_all_loss_20241120_144943 --ckpt_idx 20 --testset ${JOB_NAME}

ADHMR

# To eval on 3DPW
torchrun --nproc_per_node=2 --master_port=23452 main/main.py --config config/test/test-3dpw-custom.yaml --exp experiment/scorenet --doc 3dpw --validate --multihypo_n 100 --batch_size 80
# To eval on Human3.6M
torchrun --nproc_per_node=2 --master_port=23452 main/main.py --config config/test/test-h36m-custom.yaml --exp experiment/scorenet --doc h36m --validate --multihypo_n 100 --batch_size 80

🚄 Training

ADHMR

# To train on 3DPW
torchrun main/main.py --config config/train/hyponet/instavariety-dpo-scorer.yaml --exp experiment/hyponet --doc instavariety-dpo
# To train on Human3.6M
torchrun main/main.py --config config/train/hyponet/h36m-dpo.yaml --exp experiment/hyponet --doc h36m-dpo

📚 Citation

If you find our work useful for your research, please consider citing the paper:

@inproceedings{shen2025adhmr,
  title={ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization},
  author={Shen, Wenhao and Yin, Wanqi and Yang, Xiaofeng and Chen, Cheng and Song, Chaoyue and Cai, Zhongang and Yang, Lei and Wang, Hao and Lin, Guosheng},
  booktitle={International Conference on Machine Learning},
  year={2025},
  organization={PMLR}
}

👏 Acknowledgement

This repo is built on the excellent work SMPLer-X, ScoreHypo. Thanks for their great projects.

Explore More Motrix Projects

Motion Capture

• [SMPL-X] [TPAMI'25] SMPLest-X: An extended version of SMPLer-X with stronger foundation models.
• [SMPL-X] [NeurIPS'23] SMPLer-X: Scaling up EHPS towards a family of generalist foundation models.
• [SMPL-X] [ECCV'24] WHAC: World-grounded human pose and camera estimation from monocular videos.
• [SMPL-X] [CVPR'24] AiOS: An all-in-one-stage pipeline combining detection and 3D human reconstruction.
• [SMPL-X] [NeurIPS'23] RoboSMPLX: A framework to enhance the robustness of whole-body pose and shape estimation.
• [SMPL-X] [ICML'25] ADHMR: A framework to align diffusion-based human mesh recovery methods via direct preference optimization.
• [SMPL-X] MKA: Full-body 3D mesh reconstruction from single- or multi-view RGB videos.
• [SMPL] [ICCV'23] Zolly: 3D human mesh reconstruction from perspective-distorted images.
• [SMPL] [IJCV'26] PointHPS: 3D HPS from point clouds captured in real-world settings.
• [SMPL] [NeurIPS'22] HMR-Benchmarks: A comprehensive benchmark of HPS datasets, backbones, and training strategies.

Motion Generation

• [SMPL-X] [ICLR'26] ViMoGen: A comprehensive framework that transfers knowledge from ViGen to MoGen across data, modeling, and evaluation.
• [SMPL-X] [ECCV'24] LMM: Large Motion Model for Unified Multi-Modal Motion Generation.
• [SMPL-X] [NeurIPS'23] FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing.
• [SMPL] InfiniteDance: A large-scale 3D dance dataset and an MLLM-based music-to-dance model designed for robust in-the-wild generalization.
• [SMPL] [NeurIPS'23] InsActor: Generating physics-based human motions from language and waypoint conditions via diffusion policies.
• [SMPL] [ICCV'23] ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model.
• [SMPL] [TPAMI'24] MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model.

Motion Dataset

• [SMPL] [ECCV'22] HuMMan: Toolbox for HuMMan, a large-scale multi-modal 4D human dataset.
• [SMPLX] [T-PAMI'24] GTA-Human: Toolbox for GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine.