Two by Two✌️ : Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation

Project Page | arXiv | Twitter | Dataset

Yu Qi, Yuanchen Ju, Tianming Wei, Chi Chu, Lawson L.S. Wong, Huazhe Xu

CVPR, 2025

🛠️ Installation

This project is tested on Ubuntu 22.04 with CUDA 11.8.

• Install Anaconda or Miniconda
• Clone the repository and create the environment. The environment should be installed correctly within minutes.

git clone git@github.com:TEA-Lab/TwoByTwo.git
conda env create -f environment.yml
conda activate twobytwo

• (Optional) If you would like to calculate Chamfer Distance, clone the CUDA-accelerated Chamfer Distance library:

cd src/shape_assembly/models
git clone https://github.com/ThibaultGROUEIX/ChamferDistancePytorch.git

🧩 Dataset

2BY2 Dataset has been released. To obtain our dataset, please fill out this form.

[News] 🆕Sep-29-2025: We have fixed some mis-match problems in the dataset and update the new mesh in the original link.

🍰 Dataset Utility Support

It is recommended to use our pre-generated point cloud. In the meantime, you can also generate your own point cloud, add your own data, or generate URDF(Unified Robot Description Format) file for robot simulation purpose, please see data_util folder for more detailed instructions.

🐰Training and Inference

In src/config modify the path of log_dir data root_dir. We support Distributed Data Parallel Training.

• Train Network B

cd src
python script/our_train_B.py --cfg_file train_B.yml

• Train Network A

cd src
python script/our_train_A.py --cfg_file train_A.yml

• Inference

cd src
python script/our_eval.py

🎟️ Licence

This repository is released under the MIT license. Refer to LICENSE for more information.

🎨 Acknowledgement & Contact

Our codebase is developed based on SE3-part-assembly, and we express our gratitude to all the authors for their generously open-sourced code, as well as the open-source contributions of all baseline projects Puzzlefusion++, Jigsaw, Neural Shape Mating. for their valuable impact on the community.

For inquiries about this project, please reach out to Yu Qi: qi.yu2@northeastern.edu and Yuanchen Ju: juuycc0213@gmail.com. You’re also welcome to open an issue or submit a pull request!😄

🎸 BibTeX

We would appreciate it if you find this work useful and consider citing it.

@inproceedings{qi2025two,
  title={Two by two: Learning multi-task pairwise objects assembly for generalizable robot manipulation},
  author={Qi, Yu and Ju, Yuanchen and Wei, Tianming and Chu, Chi and Wong, Lawson LS and Xu, Huazhe},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={17383--17393},
  year={2025}
}