“MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters” by Feng, Xu and Liu
Conference:
Type(s):
Title:
- MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters
Session/Category Title: Character and Rigid Body Control
Presenter(s)/Author(s):
Abstract:
In this paper, we present a simulation and control framework for generating biomechanically plausible motion for muscle-actuated characters. We incorporate a fatigue dynamics model, the 3CC-r model, into the widely-adopted Hill-type muscle model to simulate the development and recovery of fatigue in muscles, which creates a natural evolution of motion style caused by the accumulation of fatigue from prolonged activities. To address the challenging problem of controlling a musculoskeletal system with high degrees of freedom, we propose a novel muscle-space control strategy based on PD control. Our simulation and control framework facilitates the training of a generative model for muscle-based motion control, which we refer to as MuscleVAE. By leveraging the variational autoencoders (VAEs), MuscleVAE is capable of learning a rich and flexible latent representation of skills from a large unstructured motion dataset, encoding not only motion features but also muscle control and fatigue properties. We demonstrate that the MuscleVAE model can be efficiently trained using a model-based approach, resulting in the production of high-fidelity motions and enabling a variety of downstream tasks.
References:
[1]
Vittorio La Barbera, Fabio Pardo, Yuval Tassa, Monica Daley, Christopher Richards, Petar Kormushev, and John Hutchinson. 2022. OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion. arxiv:2112.06061 [cs.RO]
[2]
Noshaba Cheema, Laura A Frey-Law, Kourosh Naderi, Jaakko Lehtinen, Philipp Slusallek, and Perttu Hämäläinen. 2020. Predicting mid-air interaction movements and fatigue using deep reinforcement learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[3]
Scott L Delp, Frank C Anderson, Allison S Arnold, Peter Loan, Ayman Habib, Chand T John, Eran Guendelman, and Darryl G Thelen. 2007. OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE transactions on biomedical engineering 54, 11 (2007), 1940–1950.
[4]
Ye Fan, Joshua Litven, and Dinesh K Pai. 2014. Active volumetric musculoskeletal systems. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–9.
[5]
Levi Fussell, Kevin Bergamin, and Daniel Holden. 2021. SuperTrack: Motion Tracking for Physically Simulated Characters Using Supervised Learning. ACM Trans. Graph. 40, 6, Article 197 (dec 2021), 13 pages. https://doi.org/10.1145/3478513.3480527
[6]
Thomas Geijtenbeek. 2021. The Hyfydy Simulation Software. https://hyfydy.com https://hyfydy.com.
[7]
Thomas Geijtenbeek, Michiel Van De Panne, and A Frank Van Der Stappen. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1–11.
[8]
Hartmut Geyer and Hugh Herr. 2010. A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities. IEEE Transactions on neural systems and rehabilitation engineering 18, 3 (2010), 263–273.
[9]
Yohanan Giat, Joseph Mizrahi, and Mark Levy. 1996. A model of fatigue and recovery in paraplegic’s quadriceps muscle subjected to intermittent FES. (1996).
[10]
Yohanan Giat, Joseph Mizrahi, Mark Levy, 1993. A musculotendon model of the fatigue profiles of paralyzed quadriceps muscle under FES. IEEE transactions on biomedical engineering 40, 7 (1993), 664–674.
[11]
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. 2023. Mastering Diverse Domains through World Models. arXiv preprint arXiv:2301.04104 (2023).
[12]
Félix G Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. 2020. Robust motion in-betweening. ACM Transactions on Graphics (TOG) 39, 4 (2020), 60–1.
[13]
Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In ICLR.
[14]
Archibald Vivian Hill. 1938. The heat of shortening and the dynamic constants of muscle. Proceedings of the Royal Society of London. Series B-Biological Sciences 126, 843 (1938), 136–195.
[15]
Jessica K Hodgins, Wayne L Wooten, David C Brogan, and James F O’Brien. 1995. Animating human athletics. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. 71–78.
[16]
Alexandru-Eugen Ichim, Petr Kadleček, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based face modeling and animation. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–14.
[17]
Yifeng Jiang, Tom Van Wouwe, Friedl De Groote, and C Karen Liu. 2019. Synthesis of biologically realistic human motion using joint torque actuation. ACM Transactions On Graphics (TOG) 38, 4 (2019), 1–12.
[18]
Taku Komura, Yoshihisa Shinagawa, and Tosiyasu L Kunii. 2000. Creating and retargetting motion by the musculoskeletal human body model. The visual computer 16, 5 (2000), 254–270.
[19]
Seyoung Lee, Sunmin Lee, Yongwoo Lee, and Jehee Lee. 2021. Learning a family of motor skills from a single motion clip. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–13.
[20]
Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable Muscle-Actuated Human Simulation and Control. ACM Trans. Graph. 38, 4, Article 73 (jul 2019), 13 pages. https://doi.org/10.1145/3306346.3322972
[21]
Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics (TOG) 28, 4 (2009), 1–17.
[22]
Sung-Hee Lee and Demetri Terzopoulos. 2006. Heads up! Biomechanical modeling and neuromuscular control of the neck. In ACM SIGGRAPH 2006 Papers. 1188–1198.
[23]
Yoonsang Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. 2014. Locomotion control for many-muscle humanoids. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–11.
[24]
Yuwei Li, Longwen Zhang, Zesong Qiu, Yingwenqi Jiang, Nianyi Li, Yuexin Ma, Yuyao Zhang, Lan Xu, and Jingyi Yu. 2022. NIMBLE: a non-rigid hand model with bones and muscles. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–16.
[25]
Jing Z Liu, Robert W Brown, and Guang H Yue. 2002. A dynamical model of muscle activation, fatigue, and recovery. Biophysical journal 82, 5 (2002), 2344–2359.
[26]
Libin Liu and Jessica Hodgins. 2017. Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning. ACM Transactions on Graphics 36, 4 (June 2017), 42a:1. https://doi.org/10.1145/3072959.3083723
[27]
Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-Driven Soft Body Characters. ACM Transactions on Graphics 32, 6 (Nov. 2013), 1–8.
[28]
John M Looft, Nicole Herkert, and Laura Frey-Law. 2018. Modification of a three-compartment muscle fatigue model to predict peak torque decline during intermittent tasks. Journal of biomechanics 77 (2018), 16–25.
[29]
Liang Ma, Damien Chablat, Fouad Bennis, and Wei Zhang. 2009. A new simple dynamic muscle fatigue model and its validation. International journal of industrial ergonomics 39, 1 (2009), 211–220.
[30]
Walter Maurel and Daniel Thalmann. 2000. Human shoulder modeling including scapulo-thoracic constraint and joint sinus cones. Computers & Graphics 24, 2 (2000), 203–218.
[31]
Vismay Modi, Lawson Fulton, Alec Jacobson, Shinjiro Sueda, and David IW Levin. 2021. Emu: Efficient muscle simulation in deformation space. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 234–248.
[32]
Uldarico Muico, Jovan Popović, and Zoran Popović. 2011. Composite control of physically simulated characters. ACM Transactions on Graphics (TOG) 30, 3 (2011), 1–11.
[33]
Jungnam Park, Sehee Min, Phil Sik Chang, Jaedong Lee, Moon Seok Park, and Jehee Lee. 2022. Generative gaitnet. In ACM SIGGRAPH 2022 Conference Proceedings. 1–9.
[34]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024–8035.
[35]
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG) 37, 4 (2018), 1–14.
[36]
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. 2022. ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters. arXiv preprint arXiv:2205.01906 (2022).
[37]
Xue Bin Peng and Michiel van de Panne. 2017. Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?. In Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA ’17). ACM, New York, NY, USA, Article 12, 13 pages. https://doi.org/10.1145/3099564.3099567
[38]
Jim R. Potvin and Andrew J. Fuglevand. 2017. A motor unit-based model of muscle fatigue. PLOS Computational Biology 13, 6 (06 2017), 1–30. https://doi.org/10.1371/journal.pcbi.1005581
[39]
Hoseok Ryu, Minseok Kim, Seungwhan Lee, Moon Seok Park, Kyoungmin Lee, and Jehee Lee. 2021. Functionality-Driven Musculature Retargeting. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 341–356.
[40]
Pierre Schumacher, Daniel F.B. Haeufle, Dieter Büchler, Syn Schmitt, and Georg Martius. 2023. DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=C-xa_D3oTj6
[41]
Ajay Seth, Jennifer L Hicks, Thomas K Uchida, Ayman Habib, Christopher L Dembia, James J Dunne, Carmichael F Ong, Matthew S DeMers, Apoorva Rajagopal, Matthew Millard, 2018. OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLoS computational biology 14, 7 (2018), e1006223.
[42]
Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic biomechanical simulation and control of human swimming. ACM Transactions on Graphics (TOG) 34, 1 (2014), 1–15.
[43]
Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. In ACM SIGGRAPH 2007 papers. 107–es.
[44]
Sangeetha Grama Srinivasan, Qisi Wang, Junior Rojas, Gergely Klár, Ladislav Kavan, and Eftychios Sifakis. 2021. Learning active quasistatic physics-based models from data. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–14.
[45]
Shinjiro Sueda, Andrew Kaufman, and Dinesh K Pai. 2008. Musculotendon simulation for hand animation. In ACM SIGGRAPH 2008 papers. 1–8.
[46]
Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5026–5033. https://doi.org/10.1109/IROS.2012.6386109
[47]
Winnie Tsang, Karan Singh, and Eugene Fiume. 2005. Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation. 319–328.
[48]
Frans CT Van der Helm. 1994. A finite element musculoskeletal model of the shoulder mechanism. Journal of biomechanics 27, 5 (1994), 551–569.
[49]
Caggiano Vittorio, Wang Huawei, Durandau Guillaume, Sartori Massimo, and Kumar Vikash. 2022. MyoSuite – A contact-rich simulation suite for musculoskeletal motor control. https://github.com/facebookresearch/myosuite. https://arxiv.org/abs/2205.13600
[50]
Jack M Wang, Samuel R Hamner, Scott L Delp, and Vladlen Koltun. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1–11.
[51]
Ying Wang, Jasper Verheul, Sang-Hoon Yeo, Nima Khademi Kalantari, and Shinjiro Sueda. 2022. Differentiable simulation of inertial musculotendons. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1–11.
[52]
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. 2021. Control strategies for physically simulated characters performing two-player competitive sports. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–11.
[53]
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. 2022. Physics-based character controllers using conditional VAEs. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–12.
[54]
Ting Xia and Laura A Frey Law. 2008. A theoretical approach for modeling peripheral muscle fatigue and recovery. Journal of biomechanics 41, 14 (2008), 3046–3052.
[55]
Lingchen Yang, Byungsoo Kim, Gaspard Zoss, Baran Gözcü, Markus Gross, and Barbara Solenthaler. 2022. Implicit neural representation for physics-driven actuated soft bodies. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–10.
[56]
Heyuan Yao, Zhenhua Song, Baoquan Chen, and Libin Liu. 2022. ControlVAE. ACM Transactions on Graphics 41, 6 (nov 2022), 1–16. https://doi.org/10.1145/3550454.3555434
[57]
KangKang Yin, Kevin Loken, and Michiel Van de Panne. 2007. Simbicon: Simple biped locomotion control. ACM Transactions on Graphics (TOG) 26, 3 (2007), 105–es.
[58]
Zhiqi Yin, Zeshi Yang, Michiel Van De Panne, and KangKang Yin. 2021. Discovering diverse athletic jumping strategies. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–17.
[59]
Goh Jing Ying and KangKang Yin. 2023. SFU Motion Capture Database. https://mocap.cs.sfu.ca/. Accessed: 2023/09/01.
[60]
Felix E Zajac. 1989. Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control.Critical reviews in biomedical engineering 17, 4 (1989), 359–411.
[61]
Qing-hong Zhu, Yan Chen, and Arie Kaufman. 1998. Real-time biomechanically-based muscle volume deformation using FEM. In Computer Graphics Forum, Vol. 17. Wiley Online Library, 275–284.
