AwesomeSim2Real

🚀 Contributions Welcome! If you’ve published or come across any Sim2Real RL papers not listed here, please feel free to open an issue — we’d love to add them to this list and update our survey! 🚀

The Research Trend in Sim2Real RL

The image only shows the paper included in this page, maybe not covering all papers, please let us know if your work is relevant to this community!

The sim-to-real research work has been an interest in well-known venues, such as: ICRA, IROS, CoRL, ICLR, AAAI, CVPR, NeurIPS, SISY, RAAD, SSCI, ITSC, CIKM, CDC, CoDIT, IV, ICANN, Aerospace, etc.

Sim-to-Real-Paper-Collection

In the paper collection, we collected RL-based sim-to-real papers published in the recent years (201x-202x) on top conferences and journals, and categorize based on the research domains, namely simulators, robotics, transportations and others.

• Sim-to-Real Paper Collection
• Surveys and Simulators
  • Survey Papers
  • Environments and Benchmarks
  • Robotics Environments
  • Robotics Benchmarks
  • Transportation Environments
  • Transportation Benchmarks
  • Recommender System Environments
  • Recommender System Benchmarks
  • Other Environments
  • Other Benchmarks
• Technique Papers
  • Obeservation
  • Action
  • Transistion
  • Reward
• The Research Trend in Sim2Real RL

Surveys and Simulators

Survey Papers

1. A Survey on Sim-to-Real Transfer Methods for Robotic Manipulation. Pitkevich, Andrei and Makarov, Ilya. IEEE 22nd Jubilee International Symposium on Intelligent Systems and Informatics (SISY). 2024. link
2. How simulation helps autonomous driving: A survey of sim2real, digital twins, and parallel intelligence. Hu, Xuemin and Li, Shen and Huang, Tingyu and Tang, Bo and Huai, Rouxing and Chen, Long. IEEE Transactions on Intelligent Vehicles. 2023. link
3. Parallel learning: Overview and perspective for computational learning across Syn2Real and Sim2Real. HMiao, Qinghai and Lv, Yisheng and Huang, Min and Wang, Xiao and Wang, Fei-Yue. IEEE/CAA Journal of Automatica Sinica. 2023. link
4. A Brief Survey of Sim2Real Methods for Robot Learning. Dimitropoulos, Konstantinos and Hatzilygeroudis, Ioannis and Chatzilygeroudis, Konstantinos. International Conference on Robotics in Alpe-Adria Danube Region. 2022 link
5. A survey of sim-to-real transfer techniques applied to reinforcement learning for bioinspired robots. Zhu, Wei and Guo, Xian and Owaki, Dai and Kutsuzawa, Kyo and Hayashibe, Mitsuhiro. IEEE Transactions on Neural Networks and Learning Systems. 2021 link
6. Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning. Salvato, Erica and Fenu, Gianfranco and Medvet, Eric and Pellegrino, Felice Andrea. IEEE Access. 2021 link
7. Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. Wenshuai Zhao, Jorge Peña Queralta, Tomi Westerlund. IEEE Symposium Series on Computational Intelligence. 2020. link

Environments and benchmarks

Robotics Environments

1. Delving Deeper into Out-of-Distribution Detection in Deep Neural Networks. Alok Sahu, Harshit Pandey, et, al. . IEEE Robotics and Automation. 2023. link

2. CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks. Oier Mees, Lukas Hermann, Erick Rosete-Beas, Wolfram Burgard. IEEE Robotics and Automation. 2021. link

3. robosuite: A Modular Simulation Framework and Benchmark for Robot Learning. Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, Yifeng Zhu. IEEE Robotics and Automation. 2020. link

4. dm_control: Software and tasks for continuous control. Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess and Yuval Tassa. Software Impacts. 2020. link
5. SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation. Xingyu Lin, Yufei Wang, Jake Olkin, David Held CoRL. 2020. link
6. Assistive Gym: A Physics Simulation Framework for Assistive Robotics. Zackory Erickson, Vamsee Gangaram, Ariel Kapusta, C. Karen Liu, Charles C. Kemp. International Conference on Robotics and Automation (ICRA). 2020. link
7. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning. Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, Sergey Levine CoRL. 2019. link
8. Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments. Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, Ilya Sutskever, Igor Mordatch, Pieter Abbeel. International Conference on Learning Representations. 2017. link
9. PyBullet: Real-Time Physics Simulation. Erwin Coumans and Yunfei Bai. 2017. link
10. OpenAI Gym. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba. 2016. link
11. MuJoCo: A physics engine for model-based control. Emanuel Todorov, Tom Erez, Yuval Tassa. 2012. link
12. Design and use paradigms for Gazebo, an open-source multi-robot simulator. Koenig, Nathan and Howard, Andrew. 2004. link

Robotics Benchmarks

1. Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning. u, Shangding and Shi, Laixi and Wen, Muning and Jin, Ming and Mazumdar, Eric and Chi, Yuejie and Wierman, Adam and Spanos, Costas. 2024. link
2. Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer. Xinyang Gu, Yen-Jen Wang, Jianyu Chen. .ICRA 2024 Workshop on Agile Robotics. 2024 link

3. DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environments. XYufei Jia, Guangyu Wang, Yuhang Dong, Junzhe Wu, Yupei Zeng, Haizhou Ge, Kairui Ding, Zike Yan,Weibin Gu, Chuxuan Li, Ziming Wang, Yunjie Cheng, Wei Sui, Ruqi Huang, Guyue Zhou. . 2024 link

4. NeuronsGym: A Hybrid Framework and Benchmark for Robot Tasks with Sim2Real Policy Learning. Haoran Li, Shasha Liu, Mingjun Ma, Guangzheng Hu, Yaran Chen, Dongbin Zhao. IEEE Transactions on Emerging Topics in Computational Intelligence .2024. link

5. RRLS : Robust Reinforcement Learning Suite. Adil Zouitine, David Bertoin, Pierre Clavier, Matthieu Geist, Emmanuel Rachelson. 2023. link
6. ManipulaTHOR: A Framework for Visual Object Manipulation. Ehsani, Kiana and Han, Winson and Herrasti, Alvaro and VanderBilt, Eli and Weihs, Luca and Kolve, Eric and Kembhavi, Aniruddha and Mottaghi, Roozbeh. CVPR. 2021. link
7. RLBench: The Robot Learning Benchmark & Learning Environment. Stephen James, Zicong Ma, David Rovick Arrojo, Andrew J. Davison. 2019. link

Transportation Environments

1. TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characters. Jonathan Wilder Lavington, Ke Zhang, Vasileios Lioutas. 2024. link
2. AutoVRL: A High Fidelity Autonomous Ground Vehicle Simulator for Sim-to-Real Deep Reinforcement Learning. Shathushan Sivashangaran, Apoorva Khairnar, Azim Eskandarian et, al. 2023. link
3. Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research. SCole Gulino, Justin Fu, Wenjie Luo, George Tucker, Eli Bronstein et, al. 2023. link

4. MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning. Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang, Zhenghai Xue, Bolei Zhou. 2023. link

5. InterSim: Interactive Traffic Simulation via Explicit Relation Modeling. Qiao Sun and Xin Huang and Brian C. Williams and Hang Zhao. 2022. link

6. TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors. Simon Suo and Sebastian Regalado and Sergio Casas and Raquel Urtasun. 2021. link
7. SUMMIT: A Simulator for Urban Driving in Massive Mixed Traffic. Panpan Cai and Yiyuan Lee and Yuanfu Luo and David Hsu. 2020. link
8. SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving. Ming Zhou, Jun Luo, Julian Villella, Yaodong Yang, David Rusu, Jiayu Miao et, al. 2020. link
9. Deepdrive Zero. Craig Quiter. 2020. link
10. CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario. Zhang, Huichu and Feng, Siyuan and Liu, Chang and Ding, Yaoyao and Zhu et, al. 2019. link
11. Highway-Env: An Environment for Autonomous Driving Decision-Making. Leurent, Edouard. 2018. link
12. Microscopic Traffic Simulation using SUMO. Lopez, Pablo Alvarez and Behrisch, Michael and Bieker-Walz, Laura and Erdmann et, al. International Conference on Intelligent Transportation Systems (ITSC) 2018. link
13. CARLA: An open urban driving simulator. Dosovitskiy, Alexey and Ros, German and Codevilla, Felipe and Lopez, Antonio and Koltun, Vladlen. CoRL 2017. link
14. Duckietown: An open, inexpensive and flexible platform for autonomy education and research. Paull, Liam and Tani, Jacopo and Ahn, Heejin and Alonso-Mora, Javier and Carlone, Luca and Cap, Michal et, al. 2017. link

Recommender System Environments

1. Recsim: A configurable simulation platform for recommender systems. Eugene Ie, Chih-wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, Craig Boutilier. 2019. link
2. Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology. Eugene Ie, Vihan Jain, Jing Wang, Sanmit Narvekar, Ritesh Agarwal, Rui Wu, Heng-Tze Cheng, Morgane Lustman, Vince Gatto, Paul Covington, Jim McFadden, Tushar Chandra, Craig Boutilier. 2019. link
3. RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising. David Rohde and Stephen Bonner and Travis Dunlop and Flavian Vasile and Alexandros Karatzoglou. 2018. link
4. Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learning. Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, An-Xiang Zeng. 2018. link

Recommender System Benchmarks

1. KuaiSim: A Comprehensive Simulator for Recommender Systems. Kesen Zhao, Shuchang Liu, Qingpeng Cai, Xiangyu Zhao, Ziru Liu, Dong Zheng, Peng Jiang, Kun Gai. 2023. link
2. Sim-to-Real Interactive Recommendation via Off-Dynamics Reinforcement Learning. *Wu, Junda and Xie, Zhihui and Yu, Tong and Li, Qizhi and Li, Shuai.