“SoftCon: simulation and control of soft-bodied animals with biomimetic actuators” by Min, Won, Lee, Park and Lee
Conference:
Type(s):
Title:
- SoftCon: simulation and control of soft-bodied animals with biomimetic actuators
Session/Category Title: Learning to Move
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a novel and general framework for the design and control of underwater soft-bodied animals. The whole body of an animal consisting of soft tissues is modeled by tetrahedral and triangular FEM meshes. The contraction of muscles embedded in the soft tissues actuates the body and limbs to move. We present a novel muscle excitation model that mimics the anatomy of muscular hydrostats and their muscle excitation patterns. Our deep reinforcement learning algorithm equipped with the muscle excitation model successfully learned the control policy of soft-bodied animals, which can be physically simulated in real-time, controlled interactively, and resilient to external perturbations. We demonstrate the effectiveness of our approach with various simulated animals including octopuses, lampreys, starfishes, stingrays and cuttlefishes. They learn diverse behaviors such as swimming, grasping, and escaping from a bottle. We also implemented a simple user interface system that allows the user to easily create their creatures.
References:
1. David Baraff and Andrew Witkin. 1998. Large Steps in Cloth Simulation. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). 43–54.Google ScholarDigital Library
2. Jernej Barbič, Marco da Silva, and Jovan Popović. 2009. Deformable Object Animation Using Reduced Optimal Control. ACM Trans. Graph. 28, 3, Article 53 (2009).Google Scholar
3. Jernej Barbič and Jovan Popović. 2008. Real-time Control of Physically Based Simulations Using Gentle Forces. ACM Trans. Graph. 27, 5, Article 163 (2008).Google Scholar
4. James M. Bern, Kai-Hung Chang, and Stelian Coros. 2017. Interactive Design of Animated Plushies. ACM Trans. Graph. 36, 4, Article 80 (2017).Google Scholar
5. Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective Dynamics: Fusing Constraint Projections for Fast Simulation. ACM Trans. Graph. 32, 6, Article 154 (2014).Google ScholarDigital Library
6. Christopher Brandt, Elmar Eisemann, and Klaus Hildebrandt. 2018. Hyper-reduced Projective Dynamics. ACM Trans. Graph. 37, 4, Article 80 (2018).Google Scholar
7. Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized Biped Walking Control. ACM Trans. Graph. 29, 4, Article 130 (2010).Google ScholarDigital Library
8. Stelian Coros, Andrej Karpathy, Ben Jones, Lionel Reveret, and Michiel van de Panne. 2011. Locomotion Skills for Simulated Quadrupeds. ACM Trans. Graph. 30, 4, Article 59 (2011).Google ScholarDigital Library
9. Stelian Coros, Sebastian Martin, Bernhard Thomaszewski, Christian Schumacher, Robert Sumner, and Markus Gross. 2012. Deformable Objects Alive! ACM Trans. Graph. 31, 4, Article 69 (2012).Google Scholar
10. Marco da Silva, Yeuhi Abe, and Jovan Popović. 2008. Interactive Simulation of Stylized Human Locomotion. ACM Trans. Graph. 27, 3, Article 82 (2008).Google Scholar
11. Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu, and Peter Zhokhov. 2017. OpenAI Baselines. https://github.com/openai/baselines. (2017).Google Scholar
12. Ye Fan, Joshua Litven, and Dinesh K. Pai. 2014. Active Volumetric Musculoskeletal Systems. ACM Trans. Graph. 33, 4, Article 152 (2014).Google Scholar
13. Jingyi Fang, Chenfanfu Jiang, and Demetri Terzopoulos. 2013. Modeling and Animating Myriapoda: A Real-time Kinematic/Dynamic Approach. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’13). 203–212.Google ScholarDigital Library
14. Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible Muscle-based Locomotion for Bipedal Creatures. ACM Trans. Graph. 32, 6, Article 206 (2013).Google ScholarDigital Library
15. Radek Grzeszczuk, Demetri Terzopoulos, and Geoffrey Hinton. 1998. NeuroAnimator: Fast Neural Network Emulation and Control of Physics-based Models. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). 9–20.Google ScholarDigital Library
16. Sehoon Ha, Stelian Coros, Alexander Alspach, Joohyung Kim, and Katsu Yamane. 2017. Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem. In Robotics: Science and Systems.Google Scholar
17. Binyamin Hochner. 2012. An Embodied View of Octopus Neurobiology. Current Biology 22, 20 (2012), R887 — R892.Google Scholar
18. Jessica K. Hodgins, Wayne L. Wooten, David C. Brogan, and James F. O’Brien. 1995. Animating Human Athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. 71–78.Google Scholar
19. Yixin Hu, Qingnan Zhou, Xifeng Gao, Alec Jacobson, Denis Zorin, and Daniele Panozzo. 2018. Tetrahedral Meshing in the Wild. ACM Trans. Graph. 37, 4, Article 60 (2018).Google Scholar
20. Takashi Ijiri, Kenshi Takayama, Hideo Yokota, and Takeo Igarashi. 2009. ProcDef: Local-to-global Deformation for Skeleton-free Character Animation. Computer Graphics Forum (proceedings of Pacific Graphics) 28, 7 (2009), 1821–1828.Google Scholar
21. Eunjung Ju, Jungdam Won, Jehee Lee, Byungkuk Choi, Junyong Noh, and Min Gyu Choi. 2013. Data-driven Control of Flapping Flight. ACM Trans. Graph. 32, 5, Article 151 (2013).Google Scholar
22. Junggon Kim and Nancy S. Pollard. 2011. Fast Simulation of Skeleton-driven Deformable Body Characters. ACM Trans. Graph. 30, 5, Article 121 (2011).Google Scholar
23. Cecilia Laschi, Matteo Cianchetti, Barbara Mazzolai, Laura Margheri, Maurizio Follador, and Paolo Dario. 2012. Soft Robot Arm Inspired by the Octopus. Advanced Robotics 26, 7 (2012), 709–727.Google Scholar
24. Joseph Laszlo, Michiel van de Panne, and Eugene Fiume. 1996. Limit Cycle Control and Its Application to the Animation of Balancing and Walking. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. 155–162.Google Scholar
25. Seunghwan Lee, Ri Yu, Jungnam Park, Mridul Aanjaneya, Eftychios Sifakis, and Jehee Lee. 2018. Dexterous Manipulation and Control with Volumetric Muscles. ACM Trans. Graph. 37, 4, Article 57 (2018).Google Scholar
26. Yoonsang Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. 2014. Locomotion Control for Many-muscle Humanoids. ACM Trans. Graph. 33, 6, Article 218 (2014).Google Scholar
27. Guy Levy and Binyamin Hochner. 2017. Embodied organization of Octopus vulgaris morphology, vision, and locomotion. Frontiers in physiology 8 (2017), 164.Google Scholar
28. Libin Liu and Jessica Hodgins. 2018. Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning. ACM Trans. Graph. 37, 4, Article 142 (2018).Google Scholar
29. Libin Liu, Michiel Van De Panne, and Kangkang Yin. 2016. Guided Learning of Control Graphs for Physics-Based Characters. ACM Trans. Graph. 35, 3, Article 29 (2016).Google Scholar
30. Tiantian Liu, Adam W. Bargteil, James F. O’Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-spring Systems. ACM Trans. Graph., Article 214 (2013).Google Scholar
31. Sebastian Martin, Bernhard Thomaszewski, Eitan Grinspun, and Markus Gross. 2011. Example-based Elastic Materials. ACM Trans. Graph. 30, 4, Article 72 (2011).Google Scholar
32. Matthias Müller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. 2007. Position Based Dynamics. Journal of Visual Communication and Image Representation 18, 2 (2007), 109–118.Google Scholar
33. Andrew Nealen, Matthias Müller, Richard Keiser, Eddy Boxerman, and Mark Carlson. 2006. Physically based deformable models in computer graphics. Computer Graphics Forum 25, 4 (2006), 809–836.Google Scholar
34. Zherong Pan and Dinesh Manocha. 2018. Active Animations of Reduced Deformable Models with Environment Interactions. ACM Trans. Graph. 37, 3, Article 36 (2018).Google Scholar
35. Sung-Jin Park, Mattia Gazzola, Kyung Soo Park, Shirley Park, Valentina Di Santo, Erin L. Blevins, Johan U. Lind, Patrick H. Campbell, Stephanie Dauth, Andrew K. Capulli, Francesco S. Pasqualini, Seungkuk Ahn, Alexander Cho, Hongyan Yuan, Ben M. Maoz, Ragu Vijaykumar, Jeong-Woo Choi, Karl Deisseroth, George V. Lauder, L. Mahadevan, and Kevin Kit Parker. 2016. Phototactic guidance of a tissue-engineered soft-robotic ray. Science 353, 6295 (2016), 158–162.Google Scholar
36. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018a. Deep-Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Trans. Graph. 37, 4, Article 143 (2018).Google ScholarDigital Library
37. Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2016. Terrain-adaptive Locomotion Skills Using Deep Reinforcement Learning. ACM Trans. Graph. 35, 4, Article 81 (2016).Google Scholar
38. Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel van de Panne. 2017. DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning. ACM Trans. Graph. 36, 4, Article 41 (2017).Google ScholarDigital Library
39. Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, and Sergey Levine. 2018b. SFV: Reinforcement Learning of Physical Skills from Videos. ACM Trans. Graph. 37, 6, Article 178 (2018).Google ScholarDigital Library
40. Jonas N Richter, Binyamin Hochner, and Michael J Kuba. 2015. Octopus arm movements under constrained conditions: adaptation, modification and plasticity of motor primitives. Journal of Experimental Biology 218, 7 (2015), 1069–1076.Google Scholar
41. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
42. Christian Schulz, Christoph von Tycowicz, Hans-Peter Seidel, and Klaus Hildebrandt. 2014. Animating Deformable Objects Using Sparse Spacetime Constraints. ACM Trans. Graph. 33, 4, Article 109 (2014).Google Scholar
43. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic Biomechanical Simulation and Control of Human Swimming. ACM Trans. Graph. 34, 1, Article 10 (2014).Google Scholar
44. Eftychios Sifakis and Jernej Barbic. 2012. FEM Simulation of 3D Deformable Solids: A Practitioner’s Guide to Theory, Discretization and Model Reduction. In ACM SIGGRAPH 2012 Courses (SIGGRAPH ’12). Article 20.Google ScholarDigital Library
45. Breannan Smith, Fernando De Goes, and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37, 2, Article 12 (2018).Google Scholar
46. Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26, 3, Article 107 (2007).Google Scholar
47. Jie Tan, Yuting Gu, Greg Turk, and C. Karen Liu. 2011. Articulated swimming creatures. ACM Trans. Graph. 30, 4, Article 58 (2011).Google Scholar
48. Jie Tan, Greg Turk, and C. Karen Liu. 2012. Soft Body Locomotion. ACM Trans. Graph. 28, 3, Article 26 (2012).Google Scholar
49. TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). http://tensorflow.org/ Software available from tensorflow.org.Google Scholar
50. Juan Tian and Qiang Lu. 2015. Simulation of Octopus Arm Based on Coupled CPGs. J. Robot. 2015, Article 4 (2015).Google Scholar
51. Xiaoyuan Tu and Demetri Terzopoulos. 1994. Artificial Fishes: Physics, Locomotion, Perception, Behavior. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’94). 43–50.Google ScholarDigital Library
52. Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing Locomotion Controllers Using Biologically-based Actuators and Objectives. ACM Trans. Graph. 31, 4, Article 25 (2012).Google Scholar
53. Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to Train Your Dragon: Example-guided Control of Flapping Flight. ACM Trans. Graph. 36, 6, Article 198 (2017).Google Scholar
54. Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics Control of Flying Creatures via Self-regulated Learning. ACM Trans. Graph. 37, 6, Article 181 (2018).Google Scholar
55. Jia-chi Wu and Zoran Popović. 2003. Realistic modeling of bird flight animations. ACM Trans. Graph. 22, 3 (2003), 888–895.Google Scholar
56. Yuting Ye and C. Karen Liu. 2010. Optimal Feedback Control for Character Animation Using an Abstract Model. ACM Trans. Graph. 29, 4, Article 74 (2010).Google Scholar
57. Yoram Yekutieli, German Sumbre, Tamar Flash, and Binyamin Hochner. 2003. How to move with no rigid skeleton? The octopus has the answers. 49 (2003), 250–4.Google Scholar
58. KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: Simple Biped Locomotion Control. ACM Trans. Graph. 26, 3, Article 105 (2007).Google ScholarDigital Library
