<a href='https://github.com/mengrang' target='_blank'>Rang Meng</a><sup>1</sup> 
<a href='https://github.com/' target='_blank'>Xingyu Zhang</a> 
<a href='https://lymhust.github.io/' target='_blank'>Yuming Li</a><sup>2</sup> 
<a href='https://openreview.net/profile?id=~Chenguang_Ma3' target='_blank'>Chenguang Ma</a><sup>2</sup>
Terminal Technology Department, Alipay, Ant Group.
<sup>1</sup>Core Contributor 
<sup>2</sup>Corresponding Authors
<a href='https://antgroup.github.io/ai/echomimic_v2/'><img src='https://img.shields.io/badge/Project-Page-blue'></a>
<a href='https://huggingface.co/BadToBest/EchoMimicV2'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Model-yellow'></a>
<a href='https://modelscope.cn/models/BadToBest/EchoMimicV2'><img src='https://img.shields.io/badge/ModelScope-Model-purple'></a>
<a href='https://arxiv.org/abs/2411.10061'><img src='https://img.shields.io/badge/Paper-Arxiv-red'></a>
<a href='https://openaccess.thecvf.com/content/CVPR2025/papers/Meng_EchoMimicV2_Towards_Striking_Simplified_and_Semi-Body_Human_Animation_CVPR_2025_paper.pdf'><img src='https://img.shields.io/badge/Paper-CVPR2025-blue'></a>
<a href='https://github.com/antgroup/echomimic_v2/blob/main/assets/halfbody_demo/wechat_group.png'><img src='https://badges.aleen42.com/src/wechat.svg'></a>
<a href='https://github.com/antgroup/echomimic_v2/discussions/53'><img src='https://img.shields.io/badge/English-Common Problems-orange'></a>
<a href='https://github.com/antgroup/echomimic_v2/discussions/40'><img src='https://img.shields.io/badge/中文版-常见问题汇总-orange'></a>
git clone https://github.com/antgroup/echomimic_v2
cd echomimic_v2
sh linux_setup.sh
git clone https://github.com/antgroup/echomimic_v2
cd echomimic_v2
Create conda environment (Recommended):
conda create -n echomimic python=3.10
conda activate echomimic
Install packages with pip
pip install pip -U
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 xformers==0.0.28.post3 --index-url https://download.pytorch.org/whl/cu124
pip install torchao --index-url https://download.pytorch.org/whl/nightly/cu124
pip install -r requirements.txt
pip install --no-deps facenet_pytorch==2.6.0
Download and decompress ffmpeg-static, then
export FFMPEG_PATH=/path/to/ffmpeg-4.4-amd64-static
git lfs install
git clone https://huggingface.co/BadToBest/EchoMimicV2 pretrained_weights
The pretrained_weights is organized as follows.
./pretrained_weights/
├── denoising_unet.pth
├── reference_unet.pth
├── motion_module.pth
├── pose_encoder.pth
├── sd-vae-ft-mse
│ └── ...
└── audio_processor
└── tiny.pt
In which denoising_unet.pth / reference_unet.pth / motion_module.pth / pose_encoder.pth are the main checkpoints of EchoMimic. Other models in this hub can be also downloaded from it's original hub, thanks to their brilliant works: - sd-vae-ft-mse - audio_processor(whisper)
Run the gradio:
python app.py
Run the python inference script:
python infer.py --config='./configs/prompts/infer.yaml'
Run the python inference script for accelerated version. Make sure to check out the configuration for accelerated inference:
python infer_acc.py --config='./configs/prompts/infer_acc.yaml'
Download dataset:
python ./EMTD_dataset/download.py
Slice dataset:
bash ./EMTD_dataset/slice.sh
Process dataset:
python ./EMTD_dataset/preprocess.py
Make sure to check out the discussions to learn how to start the inference.
| Status | Milestone | ETA |
|---|---|---|
| ✅ | The inference source code of EchoMimicV2 meet everyone on GitHub | 21st Nov, 2024 |
| ✅ | Pretrained models trained on English and Mandarin Chinese on HuggingFace | 21st Nov, 2024 |
| ✅ | Pretrained models trained on English and Mandarin Chinese on ModelScope | 21st Nov, 2024 |
| ✅ | EMTD dataset list and processing scripts | 21st Nov, 2024 |
| ✅ | Jupyter demo with pose and reference image alignmnet | 16st Dec, 2024 |
| ✅ | Accelerated models | 3st Jan, 2025 |
| 🚀 | Online Demo on ModelScope to be released | TBD |
| 🚀 | Online Demo on HuggingFace to be released | TBD |
This project is intended for academic research, and we explicitly disclaim any responsibility for user-generated content. Users are solely liable for their actions while using the generative model. The project contributors have no legal affiliation with, nor accountability for, users' behaviors. It is imperative to use the generative model responsibly, adhering to both ethical and legal standards.
We would like to thank the contributors to the MimicMotion and Moore-AnimateAnyone repositories, for their open research and exploration.
We are also grateful to CyberHost and Vlogger for their outstanding work in the area of audio-driven human animation.
If we missed any open-source projects or related articles, we would like to complement the acknowledgement of this specific work immediately.
If you find our work useful for your research, please consider citing the paper :
``` @article{meng2024echomimicv2, title={EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation}, author={Meng, Rang and Zhang, Xingyu and Li, Yuming and Ma, Chenguang}, journal={arXiv preprint arXiv:2411.10061}, year={2024} } @article{meng2025echomimicv3, title={Echomimicv3: 1
$ claude mcp add echomimic_v2 \
-- python -m otcore.mcp_server <graph>