“Fast and deep deformation approximations” by Bailey, Otte, DiLorenzo and O’Brien
Conference:
Type(s):
Entry Number: 119
Title:
- Fast and deep deformation approximations
Session/Category Title: Deep Thoughts on How Things Move
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Character rigs are procedural systems that compute the shape of an animated character for a given pose. They can be highly complex and must account for bulges, wrinkles, and other aspects of a character’s appearance. When comparing film-quality character rigs with those designed for real-time applications, there is typically a substantial and readily apparent difference in the quality of the mesh deformations. Real-time rigs are limited by a computational budget and often trade realism for performance. Rigs for film do not have this same limitation, and character riggers can make the rig as complicated as necessary to achieve realistic deformations. However, increasing the rig complexity slows rig evaluation, and the animators working with it can become less efficient and may experience frustration. In this paper, we present a method to reduce the time required to compute mesh deformations for film-quality rigs, allowing better interactivity during animation authoring and use in real-time games and applications. Our approach learns the deformations from an existing rig by splitting the mesh deformation into linear and nonlinear portions. The linear deformations are computed directly from the transformations of the rig’s underlying skeleton. We use deep learning methods to approximate the remaining nonlinear portion. In the examples we show from production rigs used to animate lead characters, our approach reduces the computational time spent on evaluating deformations by a factor of 5X-10X. This significant savings allows us to run the complex, film-quality rigs in real-time even when using a CPU-only implementation on a mobile device.
References:
1. Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popović. 2002. Interactive Skeleton-driven Dynamic Deformations. ACM Trans. Graph. 21, 3 (July 2002), 586–593. Google ScholarDigital Library
2. Edilson De Aguiar, Christian Theobalt, Sebastian Thrun, and Hans-Peter Seidel. 2008. Automatic Conversion of Mesh Animations into Skeleton-based Animations. Computer Graphics Forum 27, 2 (2008), 389–397.Google ScholarCross Ref
3. Wei-Wen Feng, Byung-Uck Kim, and Yizhou Yu. 2008. Real-time Data Driven Deformation Using Kernel Canonical Correlation Analysis. ACM Trans. Graph. 27, 3, Article 91 (Aug. 2008), 9 pages. Google ScholarDigital Library
4. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep Sparse Rectifier Neural Networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Geoffrey Gordon, David Dunson, and Miroslav Dudik (Eds.), Vol. 15. PMLR, Fort Lauderdale, FL, USA, 315–323. http://proceedings.mlr.press/v15/glorotlla.htmlGoogle Scholar
5. Keith Grochow, Steven L. Martin, Aaron Hertzmann, and Zoran Popović. 2004. Style-based Inverse Kinematics. ACM Trans. Graph. 23, 3 (Aug. 2004), 522–531. Google ScholarDigital Library
6. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rig-space Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012), 8 pages. Google ScholarDigital Library
7. Nils Hasler, Thorsten Thormählen, Bodo Rosenhahn, and Hans-Peter Seidel. 2010. Learning Skeletons for Shape and Pose. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D ’10). ACM, New York, NY, USA, 23–30. Google ScholarDigital Library
8. Jim Hejl. 2004. Hardware Skinning with Quaternions. In Game Programming Gems 4, Andrew Kirmse (Ed.). Charles River Media, 487–495.Google Scholar
9. Daniel Holden, Taku Komura, and Jun Saito. 2017. Phase-functioned Neural Networks for Character Control. ACM Trans. Graph. 36, 4, Article 42 (July 2017), 13 pages. Google ScholarDigital Library
10. Daniel Holden, Jun Saito, and Taku Komura. 2016. A Deep Learning Framework for Character Motion Synthesis and Editing. ACM Trans. Graph. 35, 4, Article 138 (July 2016), 11 pages. Google ScholarDigital Library
11. Daniel Holden, Jun Saito, and Taku Komura. 2017. Learning Inverse Rig Mappings by Nonlinear Regression. IEEE Transactions on Visualization and Computer Graphics 23, 3 (March 2017), 1167–1178. Google ScholarDigital Library
12. Kurt Hornik. 1991. Approximation Capabilities of Multilayer Feedforward Networks. Neural Netw. 4, 2 (March 1991), 251–257. Google ScholarDigital Library
13. Doug L. James and Christopher D. Twigg. 2005. Skinning Mesh Animations. ACM Trans. Graph. 24, 3 (July 2005), 399–407. Google ScholarDigital Library
14. Pushkar Joshi, Mark Meyer, Tony DeRose, Brian Green, and Tom Sanocki. 2007. Harmonic Coordinates for Character Articulation. ACM Trans. Graph. 26, 3, Article 71 (July 2007). Google ScholarDigital Library
15. Tao Ju, Qian-Yi Zhou, Michiel van de Panne, Daniel Cohen-Or, and Ulrich Neumann. 2008. Reusable Skinning Templates Using Cage-based Deformations. ACM Trans. Graph. 27, 5, Article 122 (Dec. 2008), 10 pages. Google ScholarDigital Library
16. Ladislav Kavan, Steven Collins, and Carol O’Sullivan. 2009. Automatic Linearization of Nonlinear Skinning. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (I3D ’09). ACM, New York, NY, USA, 49–56. Google ScholarDigital Library
17. Ladislav Kavan, Rachel McDonnell, Simon Dobbyn, Jiří Žára, and Carol O’Sullivan. 2007. Skinning Arbitrary Deformations. In Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games (I3D ’07). ACM, New York, NY, USA, 53–60. Google ScholarDigital Library
18. L. Kavan, P.-P. Sloan, and C. O Sullivan. 2010. Fast and Efficient Skinning of Animated Meshes. Computer Graphics Forum (2010).Google Scholar
19. Ladislav Kavan and Jiří Žára. 2005. Spherical Blend Skinning: A Real-time Deformation of Articulated Models. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games (I3D ’05). ACM, New York, NY, USA, 9–16. Google ScholarDigital Library
20. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980Google Scholar
21. Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Trans. Graph. 21, 3 (July 2002), 473–482. Google ScholarDigital Library
22. Tsuneya Kurihara and Natsuki Miyata. 2004. Modeling Deformable Human Hands from Medical Images. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’04). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 355–363. Google ScholarDigital Library
23. Samuli Laine, Tero Karras, Timo Aila, Antti Herva, Shunsuke Saito, Ronald Yu, Hao Li, and Jaakko Lehtinen. 2017. Production-level Facial Performance Capture Using Deep Convolutional Neural Networks. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’17). ACM, New York, NY, USA, Article 10, 10 pages. Google ScholarDigital Library
24. Binh Huy Le and Zhigang Deng. 2012. Smooth Skinning Decomposition with Rigid Bones. ACM Trans. Graph. 31, 6, Article 199 (Nov. 2012), 10 pages. Google ScholarDigital Library
25. Binh Huy Le and Zhigang Deng. 2013. Two-layer Sparse Compression of Dense-weight Blend Skinning. ACM Trans. Graph. 32, 4, Article 124 (July 2013), 10 pages. Google ScholarDigital Library
26. Binh Huy Le and Zhigang Deng. 2014. Robust and Accurate Skeletal Rigging from Mesh Sequences. ACM Trans. Graph. 33, 4, Article 84 (July 2014), 10 pages. Google ScholarDigital Library
27. Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009), 17 pages. Google ScholarDigital Library
28. Sergey Levine, Jack M. Wang, Alexis Haraux, Zoran Popović, and Vladlen Koltun. 2012. Continuous Character Control with Low-dimensional Embeddings. ACM Trans. Graph. 31, 4, Article 28 (July 2012), 10 pages. Google ScholarDigital Library
29. J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-driven Deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’00). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 165–172. Google ScholarDigital Library
30. N. Magnenat-Thalmann, R. Laperrière, and D. Thalmann. 1988. Joint-dependent Local Deformations for Hand Animation and Object Grasping. In Proceedings on Graphics Interface ’88. Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 26–33. http://dl.acm.org/citation.cfm?id=102313.102317 Google ScholarDigital Library
31. Joe Mancewicz, Matt L. Derksen, Hans Rijpkema, and Cyrus A. Wilson. 2014. Delta Mush: Smoothing Deformations While Preserving Detail. In Proceedings of the Fourth Symposium on Digital Production (DigiPro ’14). ACM, New York, NY, USA, 7–11. Google ScholarDigital Library
32. Tim McLaughlin, Larry Cutler, and David Coleman. 2011. Character Rigging, Deformations, and Simulations in Film and Game Production. In ACM SIGGRAPH 2011 Courses (SIGGRAPH ’11). ACM, New York, NY, USA, Article 5, 18 pages. Google ScholarDigital Library
33. Alex Mohr and Michael Gleicher. 2003. Building Efficient, Accurate Character Skins from Examples. ACM Trans. Graph. 22, 3 (July 2003), 562–568. Google ScholarDigital Library
34. Tomohiko Mukai and Shigeru Kuriyama. 2016. Efficient Dynamic Skinning with Low-rank Helper Bone Controllers. ACM Trans. Graph. 35, 4, Article 36 (July 2016), 11 pages. Google ScholarDigital Library
35. S. Schaefer and C. Yuksel. 2007. Example-based Skeleton Extraction. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing (SGP ’07). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 153–162. http://dl.acm.org/citation.cfm?id=1281991.1282013 Google ScholarDigital Library
36. Peter-Pike J. Sloan, Charles F. Rose, III, and Michael F. Cohen. 2001. Shape by Example. In Proceedings of the 2001 Symposium on Interactive 3D Graphics (I3D ’01). ACM, New York, NY, USA, 135–143. Google ScholarDigital Library
37. Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints abs/1605.02688 (May 2016). http://arxiv.org/abs/1605.02688Google Scholar
38. Jean-Marc Thiery, Émilie Guy, Tamy Boubekeur, and Elmar Eisemann. 2016. Animated Mesh Approximation With Sphere-Meshes. ACM Trans. Graph. 35, 3, Article 30 (May 2016), 13 pages. Google ScholarDigital Library
39. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2008. Gaussian Process Dynamical Models for Human Motion. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2 (Feb. 2008), 283–298. Google ScholarDigital Library
40. Robert Y. Wang, Kari Pulli, and Jovan Popović. 2007. Real-time Enveloping with Rotational Regression. ACM Trans. Graph. 26, 3, Article 73 (July 2007). Google ScholarDigital Library
41. Xiaohuan Corina Wang and Cary Phillips. 2002. Multi-weight Enveloping: Least-squares Approximation Techniques for Skin Animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’02). ACM, New York, NY, USA, 129–138. Google ScholarDigital Library
42. Martin Watt, Lawrence D. Cutler, Alex Powell, Brendan Duncan, Michael Hutchinson, and Kevin Ochs. 2012. LibEE: A Multithreaded Dependency Graph for Character Animation. In Proceedings of the Digital Production Symposium (DigiPro ’12). ACM, New York, NY, USA, 59–66. Google ScholarDigital Library
