“Learning to use chopsticks in diverse gripping styles” by Yang, Yin and Liu

Learning dexterous manipulation skills is a long-standing challenge in computer graphics and robotics, especially when the task involves complex and delicate interactions between the hands, tools and objects. In this paper, we focus on chopsticks-based object relocation tasks, which are common yet demanding. The key to successful chopsticks skills is steady gripping of the sticks that also supports delicate maneuvers. We automatically discover physically valid chopsticks holding poses by Bayesian Optimization (BO) and Deep Reinforcement Learning (DRL), which works for multiple gripping styles and hand morphologies without the need of example data. Given as input the discovered gripping poses and desired objects to be moved, we build physics-based hand controllers to accomplish relocation tasks in two stages. First, kinematic trajectories are synthesized for the chopsticks and hand in a motion planning stage. The key components of our motion planner include a grasping model to select suitable chopsticks configurations for grasping the object, and a trajectory optimization module to generate collision-free chopsticks trajectories. Then we train physics-based hand controllers through DRL again to track the desired kinematic trajectories produced by the motion planner. We demonstrate the capabilities of our framework by relocating objects of various shapes and sizes, in diverse gripping styles and holding positions for multiple hand morphologies. Our system achieves faster learning speed and better control robustness, when compared to vanilla systems that attempt to learn chopstick-based skills without a gripping pose optimization module and/or without a kinematic motion planner. Our code and models are available at this link.1

1. Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. 2019. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113 (2019).Google Scholar
2. Sheldon Andrews and Paul G Kry. 2013. Goal directed multi-finger manipulation: Control policies and analysis. Computers & Graphics 37, 7 (2013), 830–839.Google ScholarDigital Library
3. Yahya Aydin and Masayuki Nakajima. 1999. Database guided computer animation of human grasping using forward and inverse kinematics. Computers & Graphics 23, 1 (1999), 145–154.Google ScholarCross Ref
4. Kevin Bergamin, Simon Clavet, Daniel Holden, and James Richard Forbes. 2019. DReCon: Data-driven responsive control of physics-based characters. ACM Transactions on Graphics (TOG) 38, 6, Article 206 (2019).Google ScholarDigital Library
5. Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 103–112.Google ScholarDigital Library
6. Eric Brochu, Abhijeet Ghosh, and Nando de Freitas. 2007. Preference galleries for material design. SIGGRAPH Posters 105, 10.1145 (2007), 1280720–1280834.Google Scholar
7. Bao-Chi Chang, Biing-Shiun Huang, Ching-Kong Chen, and Shyh-Jen Wang. 2007. The pincer chopsticks: The investigation of a new utensil in pinching function. Applied ergonomics 38, 3 (2007), 385–390.Google Scholar
8. Tao Chen, Jie Xu, and Pulkit Agrawal. 2021. A system for general in-hand object re-orientation. In Conference on Robot Learning. PMLR.Google Scholar
9. George ElKoura and Karan Singh. 2003. Handrix: animating the human hand. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 110–119.Google Scholar
10. Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. 2020a. Graspnet-1billion: A large-scale benchmark for general object grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). 11444–11453.Google ScholarCross Ref
11. Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, and Silvio Savarese. 2020b. Learning task-oriented grasping for tool manipulation from simulated self-supervision. The International Journal of Robotics Research 39, 2–3 (2020), 202–216.Google ScholarDigital Library
12. Guillermo Garcia-Hernando, Edward Johns, and Tae-Kyun Kim. 2020. Physics-based dexterous manipulations with estimated hand poses and residual reinforcement learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 9561–9568.Google ScholarDigital Library
13. GPy. since 2012. GPy: A Gaussian process framework in python. http://github.com/SheffieldML/GPy.Google Scholar
14. Philipp Hennig and Christian J Schuler. 2012. Entropy search for information-efficient global optimization. The Journal of Machine Learning Research 13, 1 (2012), 1809–1837.Google ScholarDigital Library
15. Daniel Holden, Jun Saito, Taku Komura, and Thomas Joyce. 2015. Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 Technical Briefs. 1–4.Google Scholar
16. Sha Hu, Zeshi Yang, and Greg Mori. 2021. Neural fidelity warping for efficient robot morphology design. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 7079–7086.Google ScholarDigital Library
17. Zhiyong Huang, Ronan Boulic, Nadia Magnenat Thalmann, and Daniel Thalmann. 1995. A multi-sensor approach for grasping and 3D interaction. In Computer graphics. Elsevier, 235–253.Google Scholar
18. Sumit Jain and C Karen Liu. 2011. Controlling physics-based characters using soft contacts. ACM Transactions on Graphics (TOG) 30, 6 (2011), 1–10.Google ScholarDigital Library
19. Rohan A Joseph, Alvin C Goh, Sebastian P Cuevas, Michael A Donovan, Matthew G Kauffman, Nilson A Salas, Brian Miles, Barbara L Bass, and Brian J Dunkin. 2010. “Chopstick” surgery: A novel technique improves surgeon performance and eliminates arm collision in robotic single-incision laparoscopic surgery. Surgical endoscopy 24, 6 (2010), 1331–1335.Google Scholar
20. Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael J Black, Krikamol Muandet, and Siyu Tang. 2020. Grasping field: Learning implicit representations for human grasps. In 2020 International Conference on 3D Vision (3DV). IEEE, 333–344.Google ScholarCross Ref
21. Liyiming Ke, Ajinkya Kamat, Jingqiang Wang, Tapomayukh Bhattacharjee, Christoforos Mavrogiannis, and Siddhartha S Srinivasa. 2020. Telemanipulation with chopsticks: Analyzing human factors in user demonstrations. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 11539–11546.Google ScholarDigital Library
22. Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattacharjee, Byron Boots, and Siddhartha Srinivasa. 2021. Grasping with chopsticks: Combating covariate shift in model-free imitation learning for fine manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6185–6191.Google ScholarDigital Library
23. Junhwan Kim, Frederic Cordier, and Nadia Magnenat-Thalmann. 2000. Neural network-based violinist’s hand animation. In Proceedings Computer Graphics International 2000. IEEE, 37–41.Google ScholarCross Ref
24. Seungsu Kim, Ashwini Shukla, and Aude Billard. 2014. Catching objects in flight. IEEE Transactions on Robotics 30, 5 (2014), 1049–1065.Google ScholarCross Ref
25. Uikyum Kim, Dawoon Jung, Heeyoen Jeong, Jongwoo Park, Hyun-Mok Jung, Joono Cheong, Hyouk Ryeol Choi, Hyunmin Do, and Chanhun Park. 2021. Integrated linkage-driven dexterous anthropomorphic robotic hand. Nature Communications 12, 1 (2021), 1–13.Google ScholarCross Ref
26. Yoshihito Koga, Koichi Kondo, James Kuffner, and Jean-Claude Latombe. 1994. Planning motions with intentions. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 395–408.Google ScholarDigital Library
27. Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential gallery for interactive visual design optimization. ACM Transactions on Graphics (TOG) 39, 4, Article 88 (2020).Google ScholarDigital Library
28. Paul G Kry and Dinesh K Pai. 2006. Interaction capture and synthesis. ACM Transactions on Graphics (TOG) 25, 3 (2006), 872–880.Google ScholarDigital Library
29. Shutao Li, Mingkui Tan, Ivor W Tsang, and James Tin-Yau Kwok. 2011. A hybrid PSO-BFGS strategy for global optimization of multimodal functions. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 41, 4 (2011), 1003–1014.Google ScholarDigital Library
30. C Karen Liu. 2008. Synthesis of interactive hand manipulation. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 163–171.Google ScholarDigital Library
31. C Karen Liu. 2009. Dextrous manipulation from a grasping pose. ACM Transactions on Graphics (TOG) 28, 3, Article 59 (2009).Google ScholarDigital Library
32. Libin Liu and Jessica Hodgins. 2017. Learning to schedule control fragments for physics-based characters using deep Q-learning. ACM Transactions on Graphics (TOG) 36, 3, Article 42a (2017).Google ScholarDigital Library
33. Macro. since 2021. Ten thousand ways to use chopsticks. https://marcosticks.org/poster-ten-thousand-ways-to-use-chopsticks/.Google Scholar
34. Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. 2020. Catch & Carry: Reusable neural controllers for vision-guided whole-body tasks. ACM Transactions on Graphics (TOG) 39, 4, Article 39 (2020).Google ScholarDigital Library
35. Lester James V. Miranda. 2018. PySwarms, a research-toolkit for particle swarm optimization in Python. Journal of Open Source Software 3 (2018).Google Scholar
36. Igor Mordatch, Zoran Popović, and Emanuel Todorov. 2012. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 137–144.Google Scholar
37. Yukiko Mukai and Keiko Hashimoto. 1978. A study on ways of holding chopsticks. Journal of Home Economics of Japan 29, 7 (1978), 467–473.Google Scholar
38. Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. 2020. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning. PMLR, 1101–1112.Google Scholar
39. Tomoko Osera, Chihiro Yamamoto, Rika Senke, Misako Kobayashi, Setsuko Tsutie, and Nobutaka Kurihara. 2018. Relationship between mothers and children on how to hold chopsticks and concerns about chopsticks in japanese kindergarten. Journal of Japanese Society of Shokuiku 12, 1 (2018), 19–25.Google Scholar
40. Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee. 2019. Learning predict-and-simulate policies from unorganized human motion data. ACM Transactions on Graphics (TOG) 38, 6, Article 205 (2019).Google ScholarDigital Library
41. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).Google Scholar
42. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018a. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics (TOG) 37, 4, Article 143 (2018).Google ScholarDigital Library
43. Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. 2018b. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 3803–3810.Google ScholarDigital Library
44. Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics (TOG) 36, 4, Article 41 (2017).Google ScholarDigital Library
45. Xue Bin Peng, Michael Chang, Grace Zhang, Pieter Abbeel, and Sergey Levine. 2019. MCP: Learning composable hierarchical control with multiplicative compositional policies. Advances in Neural Information Processing Systems 32 (2019).Google Scholar
46. Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, and Sergey Levine. 2020. Learning Agile Robotic Locomotion Skills by Imitating Animals. In Proceedings of Robotics: Science and Systems.Google ScholarCross Ref
47. Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. 2021. AMP: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (TOG) 40, 4, Article 144 (2021).Google ScholarDigital Library
48. Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017).Google Scholar
49. Madhu Ragupathi, Diego I Ramos-Valadez, Rodrigo Pedraza, and Eric M Haas. 2010. Robotic-assisted single-incision laparoscopic partial cecectomy. The International Journal of Medical Robotics and Computer Assisted Surgery 6, 3 (2010), 362–367.Google ScholarCross Ref
50. Akshara Rai, Rika Antonova, Seungmoon Song, William Martin, Hartmut Geyer, and Christopher Atkeson. 2018. Bayesian optimization using domain knowledge on the ATRIAS biped. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1771–1778.Google ScholarDigital Library
51. Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. 2018. Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. In Proceedings of Robotics: Science and Systems.Google ScholarCross Ref
52. Ahmed A Ramadan, Tomohito Takubo, Yasushi Mae, Kenichi Oohara, and Tatsuo Arai. 2009. Developmental process of a chopstick-like hybrid-structure two-fingered micromanipulator hand for 3-D manipulation of microscopic objects. IEEE Transactions on Industrial Electronics 56, 4 (2009), 1121–1135.Google ScholarCross Ref
53. Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer School on Machine Learning. Springer, 63–71.Google Scholar
54. Haruka Sakurai, Takahiro Kanno, and Kenji Kawashima. 2016. Thin-diameter chopsticks robot for laparoscopic surgery. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4122–4127.Google ScholarDigital Library
55. John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In International conference on machine learning. PMLR, 1889–1897.Google Scholar
56. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
57. Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. 2010. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th annual international conference on machine learning. 1015–1022.Google Scholar
58. Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. 2020. GRAB: A dataset of whole-body human grasping of objects. In European conference on computer vision (ECCV). Springer, 581–600.Google ScholarDigital Library
59. Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. 2018. Sim-to-Real: Learning Agile Locomotion For Quadruped Robots. In Proceedings of Robotics: Science and Systems.Google ScholarCross Ref
60. Emanuel Todorov. 2014. Convex and analytically-invertible dynamics with contacts and constraints: Theory and implementation in MuJoCo. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6054–6061.Google ScholarCross Ref
61. Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 5026–5033.Google ScholarCross Ref
62. Deepak Tolani and Norman I. Badler. 1996. Real-time inverse kinematics of the human arm. Presence: Teleoperators and Virtual Environments 5, 4 (1996), 393–401.Google ScholarDigital Library
63. Marc A Toussaint, Kelsey Rebecca Allen, Kevin A Smith, and Joshua B Tenenbaum. 2018. Differentiable physics and stable modes for tool-use and manipulation planning. (2018).Google Scholar
64. Yangang Wang, Jianyuan Min, Jianjie Zhang, Yebin Liu, Feng Xu, Qionghai Dai, and Jinxiang Chai. 2013. Video-based hand manipulation capture through composite motion control. ACM Transactions on Graphics (TOG) 32, 4, Article 43 (2013).Google ScholarDigital Library
65. Nkenge Wheatland, Yingying Wang, Huaguang Song, Michael Neff, Victor Zordan, and Sophie Jörg. 2015. State of the art in hand and finger modeling and animation. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 735–760.Google Scholar
66. Jungdam Won and Jehee Lee. 2019. Learning body shape variation in physics-based characters. ACM Transactions on Graphics (TOG) 38, 6, Article 207 (2019).Google ScholarDigital Library
67. Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, and Pieter Abbeel. 2019. Learning to manipulate deformable objects without demonstrations. arXiv preprint arXiv:1910.13439 (2019).Google Scholar
68. Kazuki Yamakawa, Yuko Tashiro, Mizuki Nakajima, and Tsuyoshi Saitoh. 2018. Development of a support system for holding chopsticks correctly. In 2018 International Workshop on Advanced Image Technology (IWAIT). IEEE, 1–2.Google ScholarCross Ref
69. Tomoko Yamauchi, Atsumi Koide, Atsuko Yamamoto, and Kazuko Oba. 2010. Effect of parental training on table manners and the way of holding chopsticks. Journal of Cookery Science of Japan 43, 4 (2010), 260–264.Google Scholar
70. Akira Yamazaki and Ryosuke Masuda. 2012. Autonomous foods handling by chopsticks for meal assistant robot. In ROBOTIK 2012; 7th German Conference on Robotics. VDE, 1–6.Google Scholar
71. Zeshi Yang and Zhiqi Yin. 2021. Efficient hyperparameter optimization for physics-based character animation. Proceedings of the ACM on Computer Graphics and Interactive Techniques 4, 1 (2021), 1–19.Google ScholarDigital Library
72. Yuting Ye and C Karen Liu. 2012. Synthesis of detailed hand manipulations using contact sampling. ACM Transactions on Graphics (TOG) 31, 4, Article 41 (2012).Google ScholarDigital Library
73. Zhiqi Yin, Zeshi Yang, Michel Van de Panne, and Kangkang Yin. 2021. Discovering Diverse Athletic Jumping Strategies. ACM Transactions on Graphics (TOG) 40, 4, Article 91.Google ScholarDigital Library
74. Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. 2020. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics 36, 4 (2020), 1307–1319.Google ScholarDigital Library
75. Wenyuan Zeng, Wenjie Luo, Simon Suo, Abbas Sadat, Bin Yang, Sergio Casas, and Raquel Urtasun. 2019. End-to-end interpretable neural motion planner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8660–8669.Google ScholarCross Ref
76. He Zhang, Yuting Ye, Takaaki Shiratori, and Taku Komura. 2021. ManipNet: Neural manipulation synthesis with a hand-object spatial representation. ACM Transactions on Graphics (TOG) 40, 4, Article 121 (2021).Google ScholarDigital Library
77. Yunbo Zhang, Wenhao Yu, C Karen Liu, Charlie Kemp, and Greg Turk. 2020. Learning to manipulate amorphous materials. ACM Transactions on Graphics (TOG) 39, 6, Article 189 (2020).Google ScholarDigital Library
78. Wenping Zhao, Jianjie Zhang, Jianyuan Min, and Jinxiang Chai. 2013. Robust realtime physics-based motion control for human grasping. ACM Transactions on Graphics (TOG) 32, 6, Article 207 (2013).Google ScholarDigital Library