“Guided Learning of Control Graphs for Physics-Based Characters” by Liu, Panne and Yin

  • ©

Conference:


Type(s):


Title:

    Guided Learning of Control Graphs for Physics-Based Characters

Session/Category Title:   MOTION CONTROL

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    The difficulty of developing control strategies has been a primary bottleneck in the adoption of physics-based simulations of human motion. We present a method for learning robust feedback strategies around given motion capture clips as well as the transition paths between clips. The output is a control graph that supports real-time physics-based simulation of multiple characters, each capable of a diverse range of robust movement skills, such as walking, running, sharp turns, cartwheels, spin-kicks, and flips. The control fragments that compose the control graph are developed using guided learning. This leverages the results of open-loop sampling-based reconstruction in order to produce state-action pairs that are then transformed into a linear feedback policy for each control fragment using linear regression. Our synthesis framework allows for the development of robust controllers with a minimal amount of prior knowledge.

References:


    1. Mazen Al Borno, Martin de Lasa, and Aaron Hertzmann. 2013. Trajectory optimization for full-body movements with complex contacts. TVCG 19, 8 (2013), 1405–1414. 
    2. Mazen Al Borno, Eugene Fiume, Aaron Hertzmann, and Martin de Lasa. 2014. Feedback control for rotational movements in feature space. Comput. Graph. Forum 33, 2 (2014), 225–233. 
    3. Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust task-based control policies for physics-based characters. ACM Trans. Graph. 28, 5, Article 170 (Dec. 2009), 9 pages. DOI:http://dx.doi.org/ 10.1145/1618452.1618516 
    4. Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized biped walking control. ACM Trans. Graph. 29, 4, Article 130 (July 2010), 9 pages. 
    5. Marco Da Silva, Yeuhi Abe, and J. Popović. 2008. Simulation of human motion data using short-horizon model-predictive control. In Comput. Graph. Forum, Vol. 27. Wiley Online Library, 371–380.
    6. Marco da Silva, Frédo Durand, and Jovan Popović. 2009. Linear Bellman combination for control of character animation. ACM Trans. Graph. 28, 3, Article 82 (July 2009), 10 pages. DOI:http://dx.doi.org/10.1145/ 1531326.1531388 
    7. Martin de Lasa, Igor Mordatch, and Aaron Hertzmann. 2010. Feature-based locomotion controllers. ACM Trans. Graph. 29, 4, Article 131 (July 2010), 10 pages. DOI:http://dx.doi.org/ 10.1145/1778765.1781157 
    8. Kai Ding, Libin Liu, Michiel van de Panne, and KangKang Yin. 2015. Learning reduced-order feedback policies for motion skills. In Proceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’15). ACM, New York, NY, 83–92. DOI:http://dx.doi.org/ 10.1145/2786784.2786802 
    9. Arnaud Doucet and Adam M. Johansen. 2011. A tutorial on particle filtering and smoothing: Fifteen years later. In Handbook of Nonlinear Filtering. Oxford, UK: Oxford University Press.
    10. Petros Faloutsos, Michiel van de Panne, and Demetri Terzopoulos. 2001. Composable controllers for physics-based character animation. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01). ACM, New York, NY, USA, 251–260. DOI:http://dx.doi.org/10.1145/383259.383287 
    11. Thomas Geijtenbeek and Nicolas Pronost. 2012. Interactive character animation using simulated physics: A state-of-the-art review. In Comput. Graph. Forum, Vol. 31. Wiley Online Library, 2492–2515. 
    12. Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Trans. Graph. 32, 6 (2013), 206. 
    13. Sehoon Ha, Yuting Ye, and C. Karen Liu. 2012. Falling and landing motion control for character animation. ACM Trans. Graph. 31, 6, Article 155 (Nov. 2012), 9 pages. 
    14. Hirotaka Hachiya, Jan Peters, and Masashi Sugiyama. 2009. Efficient sample reuse in EM-based policy search. In Machine Learning and Knowledge Discovery in Databases (Lecture Notes in Computer Science), Vol. 5781. Springer, Berlin, 469–484.
    15. Jessica K. Hodgins, Wayne L. Wooten, David C. Brogan, and James F. O’Brien. 1995. Animating human athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95), Susan G. Mair and Robert Cook (Eds.). ACM, New York, NY, USA, 71–78. DOI:http://dx.doi.org/10.1145/218380.218414 
    16. Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion graphs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’02). ACM, New York, NY, 473–482. 
    17. Taesoo Kwon and Jessica Hodgins. 2010. Control systems for human running using an inverted pendulum model and a reference motion capture sequence. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’10). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 129–138. 
    18. Jehee Lee and Kang Hoon Lee. 2006. Precomputing avatar behavior from human motion data. Graph. Models 68, 2 (2006), 158–174. 
    19. Yoonsang Lee, Sungeun Kim, and Jehee Lee. 2010a. Data-driven biped control. ACM Trans. Graph. 29, 4, Article 129 (July 2010), 8 pages. 
    20. Yongjoon Lee, Kevin Wampler, Gilbert Bernstein, Jovan Popović, and Zoran Popović. 2010b. Motion fields for interactive character locomotion. ACM Trans. Graph. 29, 6, Article 138 (Dec. 2010), 8 pages. DOI:http://dx.doi. org/10.1145/1882261.1866160 
    21. Sergey Levine and Vladlen Koltun. 2013. Guided policy search. In Proceedings of the 30th International Conference on Machine Learning (ICML’13).
    22. Sergey Levine and Vladlen Koltun. 2014. Learning complex neural network policies with trajectory optimization. In Proceedings of the 31st International Conference on Machine Learning (ICML’14).
    23. Libin Liu, KangKang Yin, and Baining Guo. 2015. Improving sampling-based motion control. Comput. Graph. Forum 34, 2 (May 2015), 415–423. DOI:http://dx.doi.org/10.1111/cgf.12571 
    24. Libin Liu, KangKang Yin, Michiel van de Panne, and Baining Guo. 2012. Terrain runner: Control, parameterization, composition, and planning for highly dynamic motions. ACM Trans. Graph. 31, 6 (2012), Article 154. 
    25. Libin Liu, KangKang Yin, Michiel van de Panne, Tianjia Shao, and Weiwei Xu. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph. 29, 4 (2010), Article 128. 
    26. Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and control of skeleton-driven soft body characters. ACM Trans. Graph. 32, 6 (2013), Article 215. 
    27. Adriano Macchietto, Victor Zordan, and Christian R. Shelton. 2009. Momentum control for balance. ACM Trans. Graph. 28, 3, Article 80 (July 2009), 8 pages. DOI:http://dx.doi.org/10.1145/1531326.1531386 
    28. Igor Mordatch, Martin de Lasa, and Aaron Hertzmann. 2010. Robust physics-based locomotion using low-dimensional planning. ACM Trans. Graph. 29, 4, Article 71 (July 2010), 8 pages. 
    29. Igor Mordatch and Emo Todorov. 2014. Combining the benefits of function approximation and trajectory optimization. In Proceedings of Robotics: Science and Systems. Berkeley, CA.
    30. Igor Mordatch, Emanuel Todorov, and Zoran Popović. 2012. Discovery of complex behaviors through contact-invariant optimization. ACM Trans. Graph. 31, 4, Article 43 (July 2012), 8 pages. 
    31. Uldarico Muico, Yongjoon Lee, Jovan Popović, and Zoran Popović. 2009. Contact-aware nonlinear control of dynamic characters. ACM Trans. Graph. 28, 3, Article 81 (July 2009), 9 pages. DOI:http://dx.doi.org/10.1145/1531326.1531387 
    32. Uldarico Muico, Jovan Popović, and Zoran Popović. 2011. Composite control of physically simulated characters. ACM Trans. Graph. 30, 3, Article 16 (May 2011), 11 pages. DOI:http://dx.doi.org/10.1145/ 1966394.1966395 
    33. Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2015. Dynamic terrain traversal skills using reinforcement learning. ACM Trans. Graph. 34, 4, Article 80 (July 2015), 11 pages. DOI:http://dx.doi.org/10.1145/ 2766910 
    34. Jan Peters and Stefan Schaal. 2007. Reinforcement learning by reward-weighted regression for operational space control. In Proceedings of the 24th International Conference on Machine Learning (ICML’07). ACM, New York, NY, 745–750. DOI:http://dx.doi.org/ 10.1145/1273496.1273590 
    35. Jan Peters and Stefan Schaal. 2008. Reinforcement learning of motor skills with policy gradients. Neural Netw. 21, 4 (May 2008), 682–697. DOI:http://dx.doi.org/ 10.1016/j.neunet.2008.02.003 
    36. Zoran Popović and Andrew Witkin. 1999. Physically based motion transformation. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., 11–20. 
    37. Marc H. Raibert and Jessica K. Hodgins. 1991. Animation of dynamic legged locomotion. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 349–358. DOI:http://dx.doi.org/10.1145/127719.122755 
    38. Stephane Ross, Geoffrey Gordon, and J. Andrew (Drew) Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS’11).
    39. Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26, 3 (2007), Article 107. 
    40. Adnan Sulejmanpašić and Jovan Popović. 2005. Adaptation of performed ballistic motion. ACM Trans. Graph.24, 1 (2005), 165–179. 
    41. Jie Tan, Yuting Gu, C. Karen Liu, and Greg Turk. 2014. Learning bicycle stunts. ACM Trans. Graph. 33, 4, Article 50 (July 2014), 12 pages. 
    42. Jie Tan, C. Karen Liu, and Greg Turk. 2011. Stable proportional-derivative controllers. IEEE Comput. Graph. Appl. 31, 4 (2011), 34–44. 
    43. Yuval Tassa, Tom Erez, and Emanuel Todorov. 2012. Synthesis and stabilization of complex behaviors through online trajectory optimization. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’12). IEEE, 4906–4913.
    44. Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal character animation with continuous control. ACM Trans. Graph. 26, 3 (July 2007), Article 7. DOI:http://dx.doi.org/10.1145/1276377.1276386 
    45. Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Trans. Graph. 28, 3 (2009), Article 60. 
    46. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2009. Optimizing walking controllers. ACM Trans. Graph. 28, 5 (2009), Article 168. 
    47. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2010. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29, 4, Article 73 (July 2010), 8 pages. DOI:http://dx.doi.org/ 10.1145/1778765.1778810 
    48. Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31, 4 (2012), 25. 
    49. Yuting Ye and C. Karen Liu. 2010. Optimal feedback control for character animation using an abstract model. ACM Trans. Graph. 29, 4, Article 74 (July 2010), 9 pages. DOI:http://dx.doi.org/10.1145/1778765.1778811 
    50. KangKang Yin, Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2008. Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 3 (2008), Article 81. 
    51. KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: Simple biped locomotion control. ACM Trans. Graph. 26, 3 (2007), Article 105. 
    52. Victor Zordan, David Brown, Adriano Macchietto, and KangKang Yin. 2014. Control of rotational dynamics for ground and aerial behavior. IEEE Trans. Visual. Comput. Graphics 20, 10 (Oct 2014), 1356–1366. DOI:http://dx.doi.org/ 10.1109/TVCG.2014.2330610
    53. Victor Brian Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic response for motion capture animation. ACM Trans. Graph. 24, 3 (July 2005), 697–701. DOI:http://dx.doi.org/10.1145/1073204.1073249

ACM Digital Library Publication:


Overview Page: