“Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly” by Tian, Xu, Li, Luo, Sueda, et al. …
Conference:
Type(s):
Title:
- Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly
Session/Category Title: Computer-Aided Design
Presenter(s)/Author(s):
Abstract:
Assembly planning is the core of automating product assembly, maintenance, and recycling for modern industrial manufacturing. Despite its importance and long history of research, planning for mechanical assemblies when given the final assembled state remains a challenging problem. This is due to the complexity of dealing with arbitrary 3D shapes and the highly constrained motion required for real-world assemblies. In this work, we propose a novel method to efficiently plan physically plausible assembly motion and sequences for real-world assemblies. Our method leverages the assembly-by-disassembly principle and physics-based simulation to efficiently explore a reduced search space. To evaluate the generality of our method, we define a large-scale dataset consisting of thousands of physically valid industrial assemblies with a variety of assembly motions required. Our experiments on this new benchmark demonstrate we achieve a state-of-the-art success rate and the highest computational efficiency compared to other baseline algorithms. Our method also generalizes to rotational assemblies (e.g., screws and puzzles) and solves 80-part assemblies within several minutes.
References:
1. Maneesh Agrawala, Doantam Phan, Julie Heiser, John Haymaker, Jeff Klingner, Pat Hanrahan, and Barbara Tversky. 2003. Designing effective step-by-step assembly instructions. ACM Transactions on Graphics (TOG) 22, 3 (2003), 828–837.
2. Iker Aguinaga, Diego Borro, and Luis Matey. 2008. Parallel RRT-based path planning for selective disassembly planning. The International Journal of Advanced Manufacturing Technology 36, 11 (2008), 1221–1233.
3. Jacopo Aleotti and Stefano Caselli. 2009. Efficient planning of disassembly sequences in physics-based animation. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 87–92.
4. OpenAI : Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Józefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, Jonas Schneider, Szymon Sidor, Josh Tobin, Peter Welinder, Lilian Weng, and Wojciech Zaremba. 2020. Learning dexterous in-hand manipulation. The International Journal of Robotics Research 39, 1 (2020), 3–20.
5. Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:arXiv:1606.01540
6. Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv:1512.03012 (2015).
7. Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. 2019. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 8973–8979.
8. Guanlong Chen, Jiangqi Zhou, Wayne Cai, Xinmin Lai, Zhongqin Lin, and Roland Menassa. 2006. A framework for an automotive body assembly process design system. Computer-Aided Design 38, 5 (2006), 531–539.
9. Hao Chen, Weiwei Wan, and Kensuke Harada. 2021a. Planning to Build Soma Blocks Using a Dual-arm Robot. In 2021 IEEE International Conference on Development and Learning (ICDL). IEEE, 1–7.
10. Tao Chen, Jie Xu, and Pulkit Agrawal. 2021b. A Simple Method for Complex In-hand Manipulation. Conference on Robot Learning (2021).
11. Wen-Chin Chen, Pei-Hao Tai, Wei-Jaw Deng, and Ling-Feng Hsieh. 2008. A three-stage integrated approach for assembly sequence planning using neural networks. Expert Systems with Applications 34, 3 (2008), 1777–1786.
12. Rémi Coulom. 2006. Efficient selectivity and backup operators in Monte-Carlo tree search. In International conference on computers and games. Springer, 72–83.
13. Erwin Coumans and Yunfei Bai. 2016. Pybullet, a python module for physics simulation for games, robotics and machine learning. (2016).
14. LS Homem De Mello and Arthur C Sanderson. 1990. AND/OR graph representation of assembly plans. IEEE Transactions on robotics and automation 6, 2 (1990), 188–199.
15. Joris De Winter, Ilias EI Makrini, Greet Van de Perre, Ann Nowé, Tom Verstraten, and Bram Vanderborght. 2021. Autonomous assembly planning of demonstrated skills with reinforcement learning in simulation. Autonomous Robots 45, 8 (2021), 1097–1110.
16. Timothy Ebinger, Sascha Kaden, Shawna Thomas, Robert Andre, Nancy M Amato, and Ulrike Thomas. 2018. A general and flexible search framework for disassembly planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3548–3555.
17. Yongxiang Fan, Jieliang Luo, and Masayoshi Tomizuka. 2019. A learning framework for high precision industrial assembly. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 811–817.
18. Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski, and Stelian Coros. 2020. ADD: Analytically Differentiable Dynamics for Multi-Body Systems with Frictional Contact. ACM Trans. Graph. 39, 6, Article 190 (Nov. 2020), 15 pages.
19. Somaye Ghandi and Ellips Masehian. 2015. Review and taxonomies of assembly and disassembly path planning problems and approaches. Computer-Aided Design 67 (2015), 58–86.
20. Dan Halperin, J-C Latombe, and Randall H Wilson. 2000. A general framework for assembly planning: The motion space approach. Algorithmica 26, 3 (2000), 577–601.
21. Valentin Noah Hartmann, Andreas Orthey, Danny Driess, Ozgur S Oguz, and Marc Toussaint. 2021. Long-horizon multi-robot rearrangement planning for construction assembly. arXiv preprint arXiv:2106.02489 (2021).
22. L.S. Homem de Mello and A.C. Sanderson. 1991. A correct and complete algorithm for the generation of mechanical assembly sequences. IEEE Transactions on Robotics and Automation 7, 2 (1991), 228–240.
23. Zhimin Hou, Jiajun Fei, Yuelin Deng, and Jing Xu. 2020. Data-efficient hierarchical reinforcement learning for robotic assembly control applications. IEEE Transactions on Industrial Electronics 68, 11 (2020), 11565–11575.
24. David Hsu, Jean-Claude Latombe, Rajeev Motwani, and Lydia E Kavraki. 1999. Capturing the connectivity of high-dimensional geometric spaces by parallelizable random sampling techniques. In Advances in randomized parallel computing. Springer, 159–182.
25. Ruizhen Hu, Wenchao Li, Oliver Van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang. 2017. Learning to predict part mobility from a single static snapshot. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1–13.
26. Yijiang Huang, Caelan R Garrett, Ian Ting, Stefana Parascho, and Caitlin T Mueller. 2021. Robotic additive construction of bar structures: Unified sequence and motion planning. Construction Robotics 5, 2 (2021), 115–130.
27. Benjamin Jones, Dalton Hildreth, Duowen Chen, Ilya Baran, Vladimir G. Kim, and Adriana Schulz. 2021. AutoMate: A Dataset and Learning Approach for Automatic Mating of CAD Assemblies. ACM Transactions on Graphics (TOG) 40, 6 (2021).
28. Lydia E Kavraki, Petr Svestka, J-C Latombe, and Mark H Overmars. 1996. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE transactions on Robotics and Automation 12, 4 (1996), 566–580.
29. Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A big CAD model dataset for geometric deep learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 9601–9611.
30. James J Kuffner and Steven M LaValle. 2000. RRT-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), Vol. 2. IEEE, 995–1001.
31. Steven M LaValle et al. 1998. Rapidly-exploring random trees: A new tool for path planning. (1998).
32. Duc Thanh Le, Juan Cortés, and Thierry Siméon. 2009. A path planning approach to (dis) assembly sequencing. In 2009 IEEE International Conference on Automation Science and Engineering. IEEE, 286–291.
33. Jieliang Luo and Hui Li. 2021. A Learning Approach to Robot-Agnostic Force-Guided High Precision Assembly. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2151–2157.
34. Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. 2021. Isaac Gym: High Performance GPU Based Physics Simulation For Robot Learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
35. Ellips Masehian and Somayé Ghandi. 2021. Assembly sequence and path planning for monotone and nonmonotone assemblies with rigid and flexible parts. Robotics and Computer-Integrated Manufacturing 72 (2021), 102180.
36. Ine Melckenbeeck, Sofie Burggraeve, Bart Van Doninck, Jeroen Vancraen, and Albert Rosich. 2020. Optimal assembly sequence based on design for assembly (DFA) rules. Procedia CIRP 91 (2020), 646–652.
37. Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. 2019. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 909–918.
38. Mark Moll, Lydia Kavraki, Jan Rosell, et al. 2017. Randomized physics-based motion planning for grasping in cluttered and uncertain environments. IEEE Robotics and Automation Letters 3, 2 (2017), 712–719.
39. Yashraj Narang, Kier Storey, Iretiayo Akinola, Miles Macklin, Philipp Reist, Lukasz Wawrzyniak, Yunrong Guo, Adam Moravanszky, Gavriel State, Michelle Lu, et al. 2022. Factory: Fast Contact for Robotic Assembly. arXiv preprint arXiv:2205.03532 (2022).
40. Xinwen Niu, Han Ding, and Youlun Xiong. 2003. A hierarchical approach to generating precedence graphs for assembly planning. International Journal of Machine Tools and Manufacture 43, 14 (2003), 1473–1486.
41. SK Ong, MML Chang, and AYC Nee. 2021. Product disassembly sequence planning: state-of-the-art, challenges, opportunities and future directions. International Journal of Production Research 59, 11 (2021), 3493–3508.
42. YF Qin and ZG Xu. 2007. Assembly process planning using a multi-objective optimization method. In Proceedings of the 2007 IEEE international conference on mechatronics and automation, ICMA, Vol. 4303610. 593–598.
43. Carlos Ramos, Joao Rocha, and Zita Vale. 1998. On the complexity of precedence graphs for assembly and task planning. Computers in industry 36, 1–2 (1998), 101–111.
44. Mohd Fadzil Faisae Rashid, Windo Hutabarat, and Ashutosh Tiwari. 2012. A review on assembly sequence planning and assembly line balancing optimisation using soft computing approaches. The International Journal of Advanced Manufacturing Technology 59, 1 (2012), 335–349.
45. Juan Jesús Roldán, Elena Crespo, Andrés Martín-Barrio, Elena Peña-Tapia, and Antonio Barrientos. 2019. A training system for Industry 4.0 operators in complex assemblies based on virtual reality and process mining. Robotics and computer-integrated manufacturing 59 (2019), 305–316.
46. Marco Santochi, Gino Dini, and Franco Failli. 2002. Computer aided disassembly planning: state of the art and perspectives. CIRP Annals 51, 2 (2002), 507–529.
47. Cem Sinanoğlu and H Rıza Börklü. 2005. An assembly sequence-planning system for mechanical parts using neural network. Assembly Automation (2005).
48. Qiang Su. 2009. A hierarchical approach on assembly sequence planning and optimal sequences analyzing. Robotics and Computer-Integrated Manufacturing 25, 1 (2009), 224–234.
49. Ioan A Sucan and Lydia E Kavraki. 2011. A sampling-based tree planner for systems with complex dynamics. IEEE Transactions on Robotics 28, 1 (2011), 116–131.
50. Sujay Sundaram, Ian Remmler, and Nancy M Amato. 2001. Disassembly sequencing using a motion planning approach. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), Vol. 2. IEEE, 1475–1480.
51. Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
52. Garrett Thomas, Melissa Chien, Aviv Tamar, Juan Aparicio Ojea, and Pieter Abbeel. 2018. Learning robotic assembly from cad. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3524–3531.
53. Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 5026–5033.
54. Weiwei Wan, Kensuke Harada, and Kazuyuki Nagata. 2017. Assembly sequence planning for motion planning. Assembly Automation (2017).
55. Hui Wang, Yiming Rong, and Dong Xiang. 2014. Mechanical assembly planning using ant colony optimization. Computer-Aided Design 47 (2014), 59–71.
56. Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao, and Kai Xu. 2019b. Shape2motion: Joint analysis of motion parts and attributes from 3d shapes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8876–8884.
57. Ying Wang, Nicholas J Weidner, Margaret A Baxter, Yura Hwang, Danny M Kaufman, and Shinjiro Sueda. 2019a. REDMAX: Efficient & flexible approach for articulated dynamics. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1–10.
58. Karl DD Willis, Pradeep Kumar Jayaraman, Hang Chu, Yunsheng Tian, Yifei Li, Daniele Grandi, Aditya Sanghi, Linh Tran, Joseph G Lambourne, Armando Solar-Lezama, and Wojciech Matusik. 2022. JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
59. Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik. 2021. Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences. ACM Transactions on Graphics (TOG) 40, 4 (2021).
60. Randall H Wilson and Jean-Claude Latombe. 1994. Geometric reasoning about mechanical assembly. Artificial Intelligence 71, 2 (1994), 371–396.
61. Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. 2020. SAPIEN: A SimulAted Part-based Interactive ENvironment. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
62. Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, and Pulkit Agrawal. 2021. An End-to-End Differentiable Framework for Contact-Aware Robot Design. In Proceedings of Robotics: Science and Systems. Virtual.
63. Zihao Yan, Ruizhen Hu, Xingguang Yan, Luanmin Chen, Oliver van Kaick, Hao Zhang, and Hui Huang. 2019. RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud. Annual Conference on Computer Graphics and Interactive Techniques Asia (SIGGRAPH Asia) 38, 6 (2019), 240:1–240:15.
64. Mingxin Yu, Lin Shao, Zhehuan Chen, Tianhao Wu, Qingnan Fan, Kaichun Mo, and Hao Dong. 2021. RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment. arXiv preprint arXiv:2112.10143 (2021).
65. Kevin Zakka, Andy Zeng, Johnny Lee, and Shuran Song. 2020. Form2fit: Learning shape priors for generalizable assembly from disassembly. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 9404–9410.
66. Xinya Zhang, Robert Belfer, Paul G Kry, and Etienne Vouga. 2020. C-Space tunnel discovery for puzzle path planning. ACM Transactions on Graphics (TOG) 39, 4 (2020), 104–1.
67. Hongkai Zhao. 2005. A fast sweeping method for eikonal equations. Mathematics of computation 74, 250 (2005), 603–627.
68. Yahan Zhou, Shinjiro Sueda, Wojciech Matusik, and Ariel Shamir. 2014. Boxelization: Folding 3D Objects into Boxes. ACM Trans. Graph. 33, 4, Article 71 (Jul 2014), 8 pages.
69. Zuyuan Zhu and Huosheng Hu. 2018. Robot learning from demonstration in robotic assembly: A survey. Robotics 7, 2 (2018), 17.
70. Stefan Zickler and Manuela M Veloso. 2009. Efficient physics-based planning: sampling search via non-deterministic tactics and skills.. In AAMAS (1). Citeseer, 27–33.
