“Synthesis of biologically realistic human motion using joint torque actuation” by Jiang, Wouwe, Groote and Liu
Conference:
Type(s):
Title:
- Synthesis of biologically realistic human motion using joint torque actuation
Session/Category Title: Learning to Move
Presenter(s)/Author(s):
Abstract:
Using joint actuators to drive the skeletal movements is a common practice in character animation, but the resultant torque patterns are often unnatural or infeasible for real humans to achieve. On the other hand, physiologically-based models explicitly simulate muscles and tendons and thus produce more human-like movements and torque patterns. This paper introduces a technique to transform an optimal control problem formulated in the muscle-actuation space to an equivalent problem in the joint-actuation space, such that the solutions to both problems have the same optimal value. By solving the equivalent problem in the joint-actuation space, we can generate human-like motions comparable to those generated by musculotendon models, while retaining the benefit of simple modeling and fast computation offered by joint-actuation models. Our method transforms constant bounds on muscle activations to nonlinear, state-dependent torque limits in the joint-actuation space. In addition, the metabolic energy function on muscle activations is transformed to a nonlinear function of joint torques, joint configuration and joint velocity. Our technique can also benefit policy optimization using deep reinforcement learning approach, by providing a more anatomically realistic action space for the agent to explore during the learning process. We take the advantage of the physiologically-based simulator, OpenSim, to provide training data for learning the torque limits and the metabolic energy function. Once trained, the same torque limits and the energy function can be applied to drastically different motor tasks formulated as either trajectory optimization or policy learning.
References:
1. Marko Ackermann and Antonie J. Van den Bogert. 2010. Optimality principles for model-based prediction of human gait. Journal of biomechanics 43, 6 (2010), 1055–1060.Google ScholarCross Ref
2. Andrew A. Amis, Duncan Dowson, and Verna Wright. 1980. Elbow joint force predictions for some strenuous isometric actions. Journal of Biomechanics 13, 9 (1980), 765–775.Google ScholarCross Ref
3. Dennis E. Anderson, Michael L. Madigan, and Maury A. Nussbaum. 2007. Maximum voluntary joint torque as a function of joint angle and angular velocity: model development and application to the lower limb. Journal of biomechanics 40, 14 (2007), 3105–3113.Google ScholarCross Ref
4. Frank C. Anderson and Marcus G. Pandy. 1999. A dynamic optimization solution for vertical jumping in three dimensions. Computer methods in biomechanics and biomedical engineering 2, 3 (1999), 201–231.Google Scholar
5. Frank C. Anderson and Marcus G. Pandy. 2001a. Dynamic optimization of human walking. Journal of biomechanical engineering 123, 5 (2001), 381–390.Google ScholarCross Ref
6. Frank C. Anderson and Marcus G. Pandy. 2001b. Static and dynamic optimization solutions for gait are practically equivalent. Journal of biomechanics 34, 2 (2001), 153–161.Google ScholarCross Ref
7. Lindsay J. Bhargava, Marcus G. Pandy, and Frank C. Anderson. 2004. A phenomenological model for estimating metabolic energy consumption in muscle contraction. Journal of biomechanics 37, 1 (2004), 81–88.Google ScholarCross Ref
8. Alexander Clegg, Wenhao Yu, Jie Tan, C. Karen Liu, and Greg Turk. 2018. Learning to Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning. ACM Trans. Graph. 37, 6, Article 179 (2018), 10 pages. Google ScholarDigital Library
9. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (ELUs). ICLR 2016 (2015).Google Scholar
10. Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust Task-based Control Policies for Physics-based Characters. ACM Trans. Graph. 28, 5, Article 170 (Dec. 2009), 9 pages. Google ScholarDigital Library
11. Friedl De Groote, Allison L. Kinney, Anil V. Rao, and Benjamin J. Fregly. 2016. Evaluation of direct collocation optimal control problem formulations for solving the muscle redundancy problem. Annals of biomedical engineering 44, 10 (2016), 2922–2936.Google ScholarCross Ref
12. Scott L. Delp, J. Peter Loan, Melissa G. Hoy, Felix E. Zajac, Eric L. Topp, and Joseph M. Rosen. 1990. An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Transactions on Biomedical engineering 37, 8 (1990), 757–767.Google ScholarCross Ref
13. Anthony C. Fang and Nancy S. Pollard. 2003. Efficient Synthesis of Physically Valid Human Motion. ACM Trans. Graph. 22, 3 (July 2003), 417–426. Google ScholarDigital Library
14. Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible Muscle-based Locomotion for Bipedal Creatures. ACM Trans. Graph. 32, 6, Article 206 (Nov. 2013), 11 pages. Google ScholarDigital Library
15. Thomas Geijtenbeek, Antonie J. van den Bogert, Ben J.H. van Basten, and Arjan Egges. 2010. Evaluatingthe Physical Realism of Character Animations Using Musculoskeletal Models. In Third International Conference in Motion in Games, 2010. (Lecture Notes in Computer Science). Google ScholarDigital Library
16. Karin G.M. Gerritsen, Antonie J. van den Bogert, Manuel Hulliger, and Ronald F. Zernicke. 1998. Intrinsic muscle properties facilitate locomotor control – a computer simulation study. Motor Control 2, 3 (Aug. 1998), 206–220.Google ScholarCross Ref
17. Michael Gleicher. 1998. Retargetting Motion to New Characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). ACM, New York, NY, USA, 33–42. Google ScholarDigital Library
18. Sampath K. Gollapudi and David C. Lin. 2009. Experimental determination of sarcomere force-length relationship in type-I human skeletal muscle fibers. Journal of biomechanics 42, 13 (2009), 2011–2016.Google ScholarCross Ref
19. Jehee J. Lee and Kang Hoon Lee. 2004. Precomputing Avatar Behavior from Human Motion Data. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’04). 79–87. Google ScholarDigital Library
20. GC Joyce, PMH Rack, and DR Westbury. 1969. The mechanical properties of cat soleus muscle during controlled lengthening and shortening movements. The Journal of physiology 204, 2 (1969), 461–474.Google ScholarCross Ref
21. Michael Kass and John Anderson. 2008. Animating Oscillatory Motion with Overlap: Wiggly Splines. ACM Trans. Graph. 27, 3, Article 28 (Aug. 2008), 8 pages. Google ScholarDigital Library
22. Lukasz Kidzinski, Sharada P. Mohanty, Carmichael F. Ong, Jennifer L. Hicks, Sean F. Carroll, Sergey Levine, Marcel Salathé, and Scott L. Delp. 2018a. Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning. (2018). http://arxiv.org/abs/1804.00198Google Scholar
23. Lukasz Kidzinski, Sharada Prasanna Mohanty, Carmichael F. Ong, Zhewei Huang, Shuchang Zhou, Anton Pechenko, Adam Stelmaszczyk, Piotr Jarosik, Mikhail Pavlov, Sergey Kolesnikov, Sergey M. Plis, Zhibo Chen, Zhizheng Zhang, Jiale Chen, Jun Shi, Zhuobin Zheng, Chun Yuan, Zhihui Lin, Henryk Michalewski, Piotr Milos, Blazej Osinski, Andrew Melnik, Malte Schilling, Helge J. Ritter, Sean F. Carroll, Jennifer L. Hicks, Sergey Levine, Marcel Salathé, and Scott L. Delp. 2018b. Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. (2018). http://arxiv.org/abs/1804.00361Google Scholar
24. Anne D. Koelewijn, Eva Dorschky, and Antonie J. Van den Bogert. 2018. A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Computer methods in biomechanics and biomedical engineering 21, 8 (2018), 521–531.Google Scholar
25. Taku Komura, Yoshihisa Shinagawa, and L. Tosiyasu Kunii. 2000. Creating and retargetting motion by the musculoskeletal human body model. Visual Computer 16 (2000), 254–270. Issue 5.Google ScholarCross Ref
26. Yi-Chung Lin, Tim W. Dorn, Anthony G. Schache, and Marcus G. Pandy. 2012. Comparison of different methods for estimating muscle forces in human movement. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine 226, 2 (2012), 103–112.Google ScholarCross Ref
27. C. Karen Liu, Aaron Hertzmann, and Zoran Popović. 2005. Learning Physics-based Motion Style with Nonlinear Inverse Optimization. ACM Trans. Graph. 24, 3 (July 2005), 1071–1081. Google ScholarDigital Library
28. C. Karen Liu and Zoran Popović. 2002. Synthesis of Complex Dynamic Character Motion from Simple Animations. ACM Trans. Graph. 21, 3 (July 2002), 408–416. Google ScholarDigital Library
29. Wan-Yen Lo and Matthias Zwicker. 2008. Real-time Planning for Parameterized Human Motion. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’08). 29–38. Google ScholarDigital Library
30. MathWorks 2017. MATLAB 2017b. (2017). The MathWorks, Natick, MA, USA.Google Scholar
31. James McCann and Nancy Pollard. 2007. Responsive Characters from Motion Fragments. ACM Trans. Graph. 26, 3 (July 2007). Google ScholarDigital Library
32. Matthew Millard, Thomas Uchida, Ajay Seth, and Scott L. Delp. 2013. Flexing computational muscle: modeling and simulation of musculotendon dynamics. Journal of biomechanical engineering 135, 2 (2013).Google ScholarCross Ref
33. Ross H. Miller. 2014. A comparison of muscle energy models for simulating human walking in three dimensions. Journal of biomechanics 47, 6 (2014), 1373–1381.Google ScholarCross Ref
34. Jianyuan Min, Huajun Liu, and Jinxiang Chai. 2010. Synthesis and Editing of Personalized Stylistic Human Motion. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D ’10). ACM, New York, NY, USA, 39–46. Google ScholarDigital Library
35. Igor Mordatch, Emanuel Todorov, and Zoran Popović. 2012. Discovery of Complex Behaviors Through Contact-invariant Optimization. ACM Trans. Graph. 31, 4, Article 43 (July 2012), 8 pages. Google ScholarDigital Library
36. Igor Mordatch, Jack M. Wang, Emanuel Todorov, and Vladlen Koltun. 2013. Animating Human Lower Limbs Using Contact-invariant Optimization. ACM Trans. Graph. 32, 6, Article 203 (Nov. 2013), 8 pages. Google ScholarDigital Library
37. Masaki Nakada, Tao Zhou, Honglin Chen, Tomer Weiss, and Demetri Terzopoulos. 2018. Deep Learning of Biomimetic Sensorimotor Control for Biomechanical Human Animation. ACM Trans. Graph. 37, 4, Article 56 (July 2018), 15 pages. Google ScholarDigital Library
38. Evert Jan Nijhofa and David A. Gabriel. 2006. Maximum isometric arm forces in the horizontal plane. Journal of Biomechanics 39, 4 (2006), 708–716.Google ScholarCross Ref
39. Marcus G. Pandy, Felix E. Zajac, Eunsup Sim, and William S. Levine. 1990. An optimal control model for maximum-height human jumping. Journal of biomechanics 23, 12 (1990), 1185–1198.Google ScholarCross Ref
40. Antonio Pedotti, V.V. Krishnan, and L. Stark. 1978. Optimization of muscle-force sequencing in human locomotion. Mathematical Biosciences 38, 1–2 (1978), 57–76.Google ScholarCross Ref
41. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michielvan de Panne. 2018. DeepMimic: Example-guided Deep Reinforcement Learning of Physics-based Character Skills. ACM Trans. Graph. 37, 4, Article 143 (July 2018), 14 pages. Google ScholarDigital Library
42. Xue Bin Peng, Glen Berseth, Kangkang Yin, and Michiel Van De Panne. 2017. DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning. ACM Trans. Graph. 36, 4, Article 41 (July 2017), 13 pages. Google ScholarDigital Library
43. Xue Bin Peng and Michiel van de Panne. 2017. Learning locomotion skills using DeepRL: Does the choice of action space matter?. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. ACM, 12. Google ScholarDigital Library
44. Zoran Popović and Andrew Witkin. 1999. Physically Based Motion Transformation. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’99). 11–20. Google ScholarDigital Library
45. Anil V. Rao. 2009. A survey of numerical methods for optimal control. Advances in the Astronautical Sciences 135, 1 (2009), 497–528.Google Scholar
46. Charles Rose, Brian Guenter, Bobby Bodenheimer, and Michael F. Cohen. 1996. Efficient Generation of Motion Transitions Using Spacetime Constraints. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96). 147–154. Google ScholarDigital Library
47. Seunghwan S. Lee, Ri Yu, Jungnam Park, Mridul Aanjaneya, Eftychios Sifakis, and Jehee Lee. 2018. Dexterous Manipulation and Control with Volumetric Muscles. ACM Trans. Graph. 37, 4, Article 57 (July 2018), 13 pages. Google ScholarDigital Library
48. Alla Safonova, Jessica K. Hodgins, and Nancy S. Pollard. 2004. Synthesizing Physically Realistic Human Motion in Low-dimensional, Behavior-specific Spaces. ACM Trans. Graph. 23, 3 (Aug. 2004), 514–521. Google ScholarDigital Library
49. Ajay Seth, Michael Sherman, Jeffrey A. Reinbolt, and Scott L. Delp. 2011. OpenSim: a musculoskeletal modeling and simulation framework for in silico investigations and exchange. Procedia Iutam 2 (2011), 212–232.Google ScholarCross Ref
50. Sung-Hee S.H. Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009), 17 pages. Google ScholarDigital Library
51. Sung-Hee S.H. Lee and Demetri Terzopoulos. 2006. Heads Up!: Biomechanical Modeling and Neuromuscular Control of the Neck. ACM Trans. Graph. 25, 3 (July 2006), 1188–1198. Google ScholarDigital Library
52. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic Biomechanical Simulation and Control of Human Swimming. ACM Trans. Graph. 34, 1, Article 10 (Dec. 2014), 15 pages. Google ScholarDigital Library
53. Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data. ACM Trans. Graph. 24, 3 (July 2005), 417–425. Google ScholarDigital Library
54. Shinjiro Sueda, Andrew Kaufman, and Dinesh K. Pai. 2008. Musculotendon Simulation for Hand Animation. ACM Trans. Graph. 27, 3, Article 83 (Aug. 2008), 8 pages. Google ScholarDigital Library
55. Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal Character Animation with Continuous Control. ACM Trans. Graph. 26, 3 (July 2007). Google ScholarDigital Library
56. Winnie Tsang, Karan Singh, and Eugene Fiume. 2005. Helping Hand: An Anatomically Accurate Inverse Dynamics Solution for Unconstrained Hand Motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’05). ACM, New York, NY, USA, 319–328. Google ScholarDigital Library
57. Thomas K. Uchida, Jennifer L. Hicks, Christopher L. Dembia, and Scott L. Delp. 2016. Stretching your energetic budget: how tendon compliance affects the metabolic cost of running. PloS one 11, 3 (2016).Google Scholar
58. Brian R. Umberger. 2010. Stance and swing phase costs in human walking. Journal of the Royal Society Interface 7, 50 (2010), 1329–1340.Google ScholarCross Ref
59. Brian R. Umberger, Karin GM Gerritsen, and Philip E. Martin. 2003. A model of human muscle energy expenditure. Computer methods in biomechanics and biomedical engineering 6, 2 (2003), 99–111.Google Scholar
60. Marjolein M. van der Krogt, Wendy W. de Graaf, Claire T. Farley, Chet T. Moritz, L. J. Richard Casius, and Maarten F. Bobbert. 2009. Robust passive dynamics of the musculoskeletal system compensate for unexpected surface changes during human hopping. Journal of Applied Physiology 107, 3, Article 112 (Sept. 2009), 7 pages.Google ScholarCross Ref
61. Andreas Wächter and Lorenz T. Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006), 25–57. Google ScholarDigital Library
62. Kevin Wampler, Zoran Popović, and Jovan Popović. 2014. Generalizing Locomotion Style to New Animals with Inverse Optimal Regression. ACM Trans. Graph. 33, 4, Article 49 (July 2014), 11 pages. Google ScholarDigital Library
63. Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing Locomotion Controllers Using Biologically-based Actuators and Objectives. ACM Trans. Graph. 31, 4, Article 25 (July 2012), 11 pages. Google ScholarDigital Library
64. Andrew Witkin and Michael Kass. 1988. Spacetime Constraints. In Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’88). ACM, New York, NY, USA, 159–168. Google ScholarDigital Library
65. Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics Control of Flying Creatures via Self-regulated Learning. In SIGGRAPH Asia 2018 Technical Papers (SIGGRAPH Asia ’18). ACM, New York, NY, USA, Article 181, 10 pages. Google ScholarDigital Library
66. Yuencheng Y.C. Lee, Demetri Terzopoulos, and Keith Waters. 1995. Realistic Modeling for Facial Animation. In Proceedings of the 22Nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’95). ACM, New York, NY, USA, 55–62. Google ScholarDigital Library
67. Yuting Ye and C. Karen Liu. 2008. Animating Responsive Characters with Dynamic Constraints in Near-unactuated Coordinates. ACM Trans. Graph. 27, 5, Article 112 (Dec. 2008), 5 pages. Google ScholarDigital Library
68. Yongjoon Y.J. Lee, Seong Jae Lee, and Zoran Popović. 2009. Compact Character Controllers. ACM Trans. Graph. 28, 5, Article 169 (Dec. 2009), 8 pages. Google ScholarDigital Library
69. Yoonsang Y.S. Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. 2014. Locomotion Control for Many-muscle Humanoids. ACM Trans. Graph. 33, 6, Article 218 (Nov. 2014), 11 pages. Google ScholarDigital Library
70. Wenhao Yu, Greg Turk, and C. Karen Liu. 2018. Learning Symmetric and Low-energy Locomotion. ACM Trans. Graph. 37, 4, Article 144 (July 2018), 12 pages. Google ScholarDigital Library
71. Felix E. Zajac. 1989. Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4 (1989), 359–411.Google Scholar
