“Learning time-critical responses for interactive character control” by Lee, Min, Lee and Lee

Creating agile and responsive characters from a collection of unorganized human motion has been an important problem of constructing interactive virtual environments. Recently, learning-based approaches have successfully been exploited to learn deep network policies for the control of interactive characters. The agility and responsiveness of deep network policies are influenced by many factors, such as the composition of training datasets, the architecture of network models, and learning algorithms that involve many threshold values, weights, and hyper-parameters. In this paper, we present a novel teacher-student framework to learn time-critically responsive policies, which guarantee the time-to-completion between user inputs and their associated responses regardless of the size and composition of the motion databases. We demonstrate the effectiveness of our approach with interactive characters that can respond to the user’s control quickly while performing agile, highly dynamic movements.

1. Okan Arikan, David A. Forsyth, and James F. O’Brien. 2005. Pushing People Around. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2005. 56–66.Google Scholar
2. Kevin Bergamin, Simon Claver, Daniel Holden, and James Richard Forbes. 2019. DReCon: Data-Driven Responsive Control of Physics-Based Characters. ACM Transactions on Graphics 38, 6, Article 206 (2019).Google ScholarDigital Library
3. Myung Geol Choi, Manmyung Kim, Kyunglyul Hyun, and Jehee Lee. 2011. Deformable Motion: Squeezing into Cluttered Environments. Computer Graphics Forum 30, 2 (2011).Google Scholar
4. Simon Clavet. 2016. Motion Matching and The Road to Next-Gen Animation. In GDC 2016.Google Scholar
5. Alexaander Clegg, Wenhao Yu, Jie Tan, Karen C. Liu, and Greg Turk. 2018. Learning To Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning. ACM Transactions on Graphics 37, 6 (2018).Google ScholarDigital Library
6. Epic Games. 2019. Unreal Engine. https://www.unrealengine.comGoogle Scholar
7. Katerina Fragkiadaki, Sergey Levine, Panna Felsen, and Jitendra Malik. 2015. Recurrent network models for human dynamics. In Proceedings of the IEEE International Conference on Computer Vision. 4346–4354.Google ScholarDigital Library
8. Felix Harvey, Mike Yurick, Christopher Pal, and Derek Nowrouzezahrai. 2020. Robust Motion In-Betweening. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
9. Rachel Heck and Michael Gleicher. 2007. Parametric motion graphs. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 129–136.Google ScholarDigital Library
10. Gustav Eje Henter, Simon Alexanderson, and Jonas Beskow. 2020. MoGlow: Probabilistic and Controllable Motion Synthesis Using Normalising Flows. ACM Trans. Graph. 39, 6, Article 236 (2020).Google ScholarDigital Library
11. Daniel Holden, Oussama Kanoun, Maksym Perepichika, and Tiberiu Popa. 2020. Learned Motion Matching. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
12. Daniel Holden, Taku Komura, and Jun Saito. 2017. Phase-functioned neural networks for character control. ACM Transactions on Graphics 36, 4, Article 42 (2017).Google ScholarDigital Library
13. Daniel Holden, Jun Saito, and Taku Komura. 2016. A Deep Learning Framework for Character Motion Synthesis and Editing. ACM Transactions on Graphics 35, 4, Article 138 (2016).Google ScholarDigital Library
14. Kyunglyul Hyun, Kyungho Lee, and Jehee Lee. 2016. Motion grammars for character animation. Computer Graphics Forum 35, 2 (2016), 103–113.Google ScholarCross Ref
15. Leslie Ikemoto, Okan Arikan, and David Forsyth. 2007. Quick transitions with cached multi-way blends. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 145–151.Google ScholarDigital Library
16. Yifeng Jiang, Tom Wouwe, Friedl De Groote, and Karen C. Liu. 2019. Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation. ACM Transactions on Graphics 38, 4 (2019).Google ScholarDigital Library
17. Sophie Jörg, Aline Normoyle, and Alla Safonova. 2012. How responsiveness affects players’ perception in digital games. In Proceedings of the ACM Symposium on Applied Perception. 33–38.Google ScholarDigital Library
18. Mubbasir Kapadia, Xu Xianghao, Maurizio Nitti, Marcelo Kallmann, Stelian Coros, Robert Sumner, and Markus Gross. 2016. Precision: precomputing environment semantics for contact-rich character animation. In Proceedings of the 2016 symposium on Interactive 3D graphics and games. 29–37.Google ScholarDigital Library
19. Manmyung Kim, Kyunglyul Hyun, Jongmin Kim, and Jehee Lee. 2009. Synchronized Multi-character Motion Editing. ACM Transactions on Graphics 28, 3, Article 79 (2009).Google ScholarDigital Library
20. Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3 (2002), 473–482.Google ScholarDigital Library
21. Jehee Lee, Jinxiang Chai, Paul S. A. Reitsma, Jessica K. Hodgins, and Nancy S. Pollard. 2002. Interactive Control of Avatars Animated with Human Motion Data. ACM Transactions on Graphics 21, 3 (2002), 491–500.Google ScholarDigital Library
22. Jehee Lee and Kang Hoon Lee. 2004. Precomputing Avatar Behavior from Human Motion Data. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 79–87.Google ScholarDigital Library
23. Kyungho Lee, Seyoung Lee, and Jehee Lee. 2018. Interactive Character Animation by Learning Multi-Objective Control. ACM Transactions on Graphics 37, 6, Article 180 (2018).Google ScholarDigital Library
24. Kang Hoon Lee, Myung Geol Choi, and Jehee Lee. 2006. Motion Patches: Building blocks for virtual environments annotated with motion data. ACM Transactions on Graphics 25, 3 (2006), 898–906.Google ScholarDigital Library
25. Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable Muscle-actuated Human Simulation and Control. ACM Transactions on Graphics 38, 4, Article 73 (2019).Google ScholarDigital Library
26. Yongjoon Lee, Kevin Wampler, Gilbert Bernstein, Jovan Popović, and Zoran Popović. 2010. Motion Fields for Interactive Character Locomotion. ACM Transactions on Graphics 29, 6, Article 138 (2010).Google ScholarDigital Library
27. Sergey Levine, Yongjoon Lee, Vladren Koltun, and Zoran Popović. 2011. Space-time planning with parameterized locomotion controllers. ACM Transactions on Graphics 30, 3 (2011).Google ScholarDigital Library
28. Sergey Levine, Jack M. Wang, Alexis Haraux, Zoran Popović, and Vladlen Koltun. 2012. Continuous Character Control with Low-Dimensional Embeddings. ACM Transactions on Graphics 31, 4 (2012).Google ScholarDigital Library
29. Hung Yu Ling, Fabio Zinno, George Cheng, and Michiel van de Panne. 2020. Character Controllers Using Motion VAEs. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
30. Libin Liu and Jessica Hodgins. 2017. Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning. ACM Transactions on Graphics 36, 3 (2017).Google ScholarDigital Library
31. Libin Liu and Jessica Hodgins. 2018. Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning. ACM Transactions on Graphics 37, 4 (2018).Google ScholarDigital Library
32. Wan-Yen Lo and Matthias Zwicker. 2008. Real-time planning for parameterized human motion. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2008. 29–38.Google Scholar
33. Ying-Sheng Luo, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen. 2020. CARL: Controllable Agent With Reinforcement Learning for Quadruped Locomotion. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
34. Julieta Martinez, Michael J Black, and Javier Romero. 2017. On human motion prediction using recurrent neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 4674–4683.Google ScholarCross Ref
35. James McCann and Nancy Pollard. 2007. Responsive characters from motion fragments. In ACM Transactions on Graphics, Vol. 26. 6.Google ScholarDigital Library
36. Jianyuan Min and Jinxiang Chai. 2012. Motion graphs++ a compact generative model for semantic motion analysis and synthesis. ACM Transactions on Graphics 31, 6 (2012), 1–12.Google ScholarDigital Library
37. Sehee Min, Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. 2019. SoftCon: Simulation and Control of Soft-Bodied Animals with Biomimetic Actuators. ACM Transactions on Graphics 38, 6, Article 208 (2019).Google ScholarDigital Library
38. Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee. 2019. Learning predict-and-simulate policies from unorganized human motion data. ACM Transactions on Graphics 38, 6, Article 205 (2019).Google ScholarDigital Library
39. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics 37, 4, Article 143 (2018).Google ScholarDigital Library
40. Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Lee, Jie Tan, and Sergey Levine. 2020. Learning Agile Robotic Locomotion Skills by Imitating Animals. arXiv preprint (2020).Google Scholar
41. Paul SA Reitsma and Nancy S Pollard. 2007. Evaluating motion graphs for character animation. ACM Transactions on Graphics 26, 4 (2007), 18.Google ScholarDigital Library
42. Cheng Ren, Liming Zhao, and Alla Safonova. 2010. Human motion synthesis with optimization-based graphs. In Computer Graphics Forum, Vol. 29. 545–554.Google ScholarCross Ref
43. Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. 2016. Policy Distillation. arXiv:1511.06295 [cs.LG]Google Scholar
44. Alla Safonova and Jessica K Hodgins. 2007. Construction and optimal search of interpolated motion graphs. ACM Transactions on Graphics 26, 3, Article 106 (2007).Google ScholarDigital Library
45. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG]Google Scholar
46. Jeremy M Sheppard and Warren B Young. 2006. Agility literature review: Classifications, training and testing. Journal of sports sciences 24, 9 (2006), 919–932.Google ScholarCross Ref
47. Hubert Shum, Taku Komura, and Shuntaro Yamazaki. 2012. Simulating Multiple Character Interactions with Collaborative and Adversarial Goals. IEEE Transactions on Visualization and Computer Graphics 18 (2012), 741–52.Google ScholarDigital Library
48. Hubert PH Shum, Taku Komura, Masashi Shiraishi, and Shuntaro Yamazaki. 2008. Interaction patches for multi-character animation. ACM Transactions on Graphics 27, 5, Article 114 (2008).Google ScholarDigital Library
49. Bernard W Silverman. 1986. Density estimation for statistics and data analysis. Vol. 26. CRC press.Google Scholar
50. Sebastian Starke, He Zhang, Taku Komura, and Jun Saito. 2019. Neural State Machine for Character-Scene Interactions. ACM Transactions on Graphics 38, 6, Article 178 (2019).Google ScholarDigital Library
51. Sebastian Starke, Yiwei Zhao, Taku Komura, and Kazi Zaman. 2020. Local Motion Phases for Learning Multi-Contact Character Movements. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
52. TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/Google Scholar
53. Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal character animation with continuous control. ACM Transactions on Graphics 26, 3, Article 7 (2007).Google ScholarDigital Library
54. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv:1706.03762 [cs.CL]Google Scholar
55. Jungdam Won, Deepa Gopinath, and Jessica Hodgins. 2020. A Scalable Approach to Control Diverse Behaviors for Physically Simulated Characters. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
56. Jungdam Won, Kyungho Lee, Carol O’Sullivan, Jessica K Hodgins, and Jehee Lee. 2014. Generating and ranking diverse multi-character interactions. ACM Transactions on Graphics 33, 6, Article 219 (2014).Google ScholarDigital Library
57. Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to Train Your Dragon: Example-guided Control of Flapping Flight. ACM Transactions on Graphics 36, 6, Article 198 (2017).Google ScholarDigital Library
58. Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics control of flying creatures via self-regulated learning. ACM Transactions on Graphics 37, 6, Article 181 (2018).Google ScholarDigital Library
59. He Zhang, Sebastian Starke, Taku Komura, and Jun Saito. 2018. Mode-adaptive neural networks for quadruped motion control. ACM Transactions on Graphics 37, 4, Article 145 (2018).Google ScholarDigital Library
60. Liming Zhao and Alla Safonova. 2009. Achieving good connectivity in motion graphs. Graphical Models 71, 4 (2009), 139–152.Google ScholarDigital Library
61. Victor Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic response for motion capture animation. ACM Transactions on Graphics 24 (2005), 697–701.Google ScholarDigital Library