“Direct delta mush skinning compression with continuous examples” by Le, Villeneuve and Gonzalez-Ochoa

Direct Delta Mush (DDM) is a high-quality, direct skinning method with a low setup cost. However, its storage and run-time computing cost are relatively high for two reasons: its skinning weights are 4 X 4 matrices instead of scalars like other direct skinning methods, and its computation requires one 3 X 3 Singular Value Decomposition per vertex.In this paper, we introduce a compression method that takes a DDM model and splits it into two layers: the first layer is a smaller DDM model that computes a set of virtual bone transformations and the second layer is a Linear Blend Skinning model that computes per-vertex transformations from the output of the first layer. The two-layer model can approximate the deformation of the original DDM model with significantly lower costs.Our main contribution is a novel problem formulation for the DDM compression based on a continuous example-based technique, in which we minimize the compression error on an uncountable set of example poses. This formulation provides an elegant metric for the compression error and simplifies the problem to the common linear matrix factorization. Our formulation also takes into account the skeleton hierarchy of the model, the bind pose, and the range of motions. In addition, we propose a new update rule to optimize DDM weights of the first layer and a modification to resolve the floating-point cancellation issue of DDM.

1. M. Aharon, M. Elad, and A. Bruckstein. 2006. K -SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Trans. Sig. Proc. 54, 11 (Nov. 2006), 4311–4322. Google ScholarDigital Library
2. Marc Alexa. 2002. Linear Combination of Transformations. ACM Trans. Graph. 21, 3 (July 2002), 380–387. Google ScholarDigital Library
3. Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape Completion and Animation of People. ACM Trans. Graph. 24, 3 (July 2005), 408–416. Google ScholarDigital Library
4. Paolo Baerlocher. 2001. Inverse kinematics techniques of the interactive posture control of articulated figures. Ph.D. Dissertation. École Polytechnique Fédérale de Lausanne. Google ScholarCross Ref
5. Stephen W. Bailey, Dave Otte, Paul Dilorenzo, and James F. O’Brien. 2018. Fast and Deep Deformation Approximations. ACM Trans. Graph. 37, 4, Article 119 (July 2018). Google ScholarDigital Library
6. Ake Bjorck. 1996. Numerical methods for least squares problems. Google ScholarCross Ref
7. Wei-Wen Feng, Byung-Uck Kim, and Yizhou Yu. 2008. Real-time Data Driven Deformation Using Kernel Canonical Correlation Analysis. ACM Trans. Graph. 27, 3, Article 91 (Aug. 2008). Google ScholarDigital Library
8. Lin Gao, Yu-Kun Lai, Dun Liang, Shu-Yu Chen, and Shihong Xia. 2016. Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling. ACM Trans. Graph. 35, 5, Article 158 (July 2016). Google ScholarDigital Library
9. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rig-space Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012). Google ScholarDigital Library
10. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W. Sumner, and Markus Gross. 2013. Efficient Simulation of Secondary Motion in Rig-space. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2013). 165–171. Google ScholarDigital Library
11. Nils Hasler, Thorsten Thormählen, Bodo Rosenhahn, and Hans-Peter Seidel. 2010. Learning Skeletons for Shape and Pose. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2010). Association for Computing Machinery, New York, NY, USA, 23–30. Google ScholarDigital Library
12. Alexandru-Eugen Ichim, Petr Kadleček, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based Face Modeling and Animation. ACM Trans. Graph. 36, 4, Article 153 (July 2017). Google ScholarDigital Library
13. Alec Jacobson, Ilya Baran, Ladislav Kavan, Jovan Popović, and Olga Sorkine. 2012. Fast Automatic Skinning Transformations. ACM Trans. Graph. 31, 4, Article 77 (July 2012). Google ScholarDigital Library
14. Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Trans. Graph. 30, 4, Article 78 (July 2011). Google ScholarDigital Library
15. Alec Jacobson, Zhigang Deng, Ladislav Kavan, and JP Lewis. 2014. Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses. http://skinning.org/Google Scholar
16. Alec Jacobson and Olga Sorkine. 2011. Stretchable and Twistable Bones for Skeletal Shape Deformation. ACM Trans. Graph. 30, 6 (Dec. 2011). Google ScholarDigital Library
17. Doug L. James and Christopher D. Twigg. 2005. Skinning Mesh Animations. ACM Trans. Graph. 24, 3 (July 2005), 399–407. Google ScholarDigital Library
18. Ben Jones, Nils Thuerey, Tamar Shinar, and Adam W. Bargteil. 2016. Example-based Plastic Deformation of Rigid Bodies. ACM Trans. Graph. 35, 4, Article 34 (July 2016). Google ScholarDigital Library
19. Petr Kadleček, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6, Article 213 (Nov. 2016). Google ScholarDigital Library
20. Ladislav Kavan, Steven Collins, and Carol O’Sullivan. 2009. Automatic Linearization of Nonlinear Skinning. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2009). 49–56. Google ScholarDigital Library
21. Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O’Sullivan. 2008. Geometric Skinning with Approximate Dual Quaternion Blending. ACM Trans. Graph. 27, 4, Article 105 (Nov. 2008). Google ScholarDigital Library
22. L. Kavan, P.-P. Sloan, and C. O’Sullivan. 2010. Fast and Efficient Skinning of Animated Meshes. Computer Graphics Forum 29, 2 (2010), 327–336. Google ScholarCross Ref
23. Ladislav Kavan and Olga Sorkine. 2012. Elasticity-Inspired Deformers for Character Articulation. ACM Trans. Graph. 31, 6, Article 196 (Nov. 2012). Google ScholarDigital Library
24. Ladislav Kavan and Jiří Žára. 2005. Spherical Blend Skinning: A Real-time Deformation of Articulated Models. In Proc. ACM Symposium on Interactive 3D Graphics and Games (I3D 2005). 9–16. Google ScholarDigital Library
25. Hyunsoo Kim and Haesun Park. 2008. Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method. SIAM J. Matrix Anal. Appl. 30, 2 (July 2008), 713–730. Google ScholarDigital Library
26. Paul G. Kry, Doug L. James, and Dinesh K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2002). 153–159. Google ScholarDigital Library
27. Eric Landreneau and Scott Schaefer. 2010. Poisson-Based Weight Reduction of Animated Meshes. Computer Graphics Forum 29, 6 (2010), 1945–1954. Google ScholarCross Ref
28. Binh Huy Le and Zhigang Deng. 2012. Smooth Skinning Decomposition with Rigid Bones. ACM Trans. Graph. 31, 6, Article 199 (Nov. 2012). Google ScholarDigital Library
29. Binh Huy Le and Zhigang Deng. 2013. Two-layer Sparse Compression of Dense-weight Blend Skinning. ACM Trans. Graph. 32, 4, Article 124 (July 2013). Google ScholarDigital Library
30. Binh Huy Le and Zhigang Deng. 2014. Robust and Accurate Skeletal Rigging from Mesh Sequences. ACM Trans. Graph. 33, 4, Article 84 (July 2014). Google ScholarDigital Library
31. Binh Huy Le and Jessica K. Hodgins. 2016. Real-time Skeletal Skinning with Optimized Centers of Rotation. ACM Trans. Graph. 35, 4, Article 37 (July2016). Google ScholarDigital Library
32. Binh Huy Le and J P Lewis. 2019. Direct Delta Mush Skinning and Variants. ACM Trans. Graph. 38, 4, Article 113 (July 2019). Google ScholarDigital Library
33. Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for Non-Negative Matrix Factorization. In Proc. of International Conference on Neural Information Processing (NIPS 2000). 535–541.Google Scholar
34. Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009). Google ScholarDigital Library
35. J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-driven Deformation. In Proc. of International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2000). 165–172. Google ScholarDigital Library
36. Duo Li, Shinjiro Sueda, Debanga R. Neog, and Dinesh K. Pai. 2013. Thin Skin Elastodynamics. ACM Trans. Graph. 32, 4, Article 49 (July 2013). Google ScholarDigital Library
37. Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-driven Soft Body Characters. ACM Trans. Graph. 32, 6, Article 215 (Nov. 2013). Google ScholarDigital Library
38. S. Lloyd. 1982. Least Squares Quantization in PCM. IEEE Transactions on Information Theory 28, 2 (Sept. 1982), 129–137. Google ScholarDigital Library
39. Matthew Loper, Naureen Mahmood, and Michael J. Black. 2014. MoSh: Motion and Shape Capture from Sparse Markers. ACM Trans. Graph. 33, 6, Article 220 (Nov. 2014). Google ScholarDigital Library
40. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015). Google ScholarDigital Library
41. N. Magnenat-Thalmann, F. Cordier, Hyewon Seo, and G. Papagianakis. 2004. Modeling of bodies and clothes for virtual environments. In IEEE International Conference on Cyberworlds (CW 2004). 201–208. Google ScholarDigital Library
42. N. Magnenat-Thalmann, R. Laperrière, and D. Thalmann. 1988. Joint-dependent Local Deformations for Hand Animation and Object Grasping. In Proc. on Graphics Interface (GI 1988). 26–33. http://dl.acm.org/citation.cfm?id=102313.102317Google Scholar
43. Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. 2010. Online Learning for Matrix Factorization and Sparse Coding. Journal of Machine Learning Research 11 (March 2010), 19–60. http://dl.acm.org/citation.cfm?id=1756006.1756008Google Scholar
44. Joe Mancewicz, Matt L. Derksen, Hans Rijpkema, and Cyrus A. Wilson. 2014. Delta Mush: Smoothing Deformations While Preserving Detail. In Proc. of ACM Symposium on Digital Production (DigiPro 2014). 7–11. Google ScholarDigital Library
45. Aleka McAdams, Andrew Selle, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011a. Computing the singular value decomposition of 3×3 matrices with minimal branching and elementary floating point operations. Technical Report. University of Wisconsin-Madison.Google Scholar
46. Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011b. Efficient Elasticity for Character Skinning with Contact and Collisions. ACM Trans. Graph. 30, 4, Article 37 (July 2011). Google ScholarDigital Library
47. Alex Mohr and Michael Gleicher. 2003. Building Efficient, Accurate Character Skins from Examples. ACM Trans. Graph. 22, 3 (July 2003), 562–568. Google ScholarDigital Library
48. Tomohiko Mukai. 2015. Building Helper Bone Rigs from Examples. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2015). 77–84. Google ScholarDigital Library
49. Tomohiko Mukai and Shigeru Kuriyama. 2016. Efficient Dynamic Skinning with Low-rank Helper Bone Controllers. ACM Trans. Graph. 35, 4, Article 36 (July 2016). Google ScholarDigital Library
50. Olivier Rémillard and Paul G. Kry. 2013. Embedded Thin Shells for Wrinkle Simulation. ACM Trans. Graph. 32, 4, Article 50 (July 2013). Google ScholarDigital Library
51. Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational Bodybuilding: Anatomically-based Modeling of Human Bodies. ACM Trans. Graph. 34, 4, Article 41 (July 2015). Google ScholarDigital Library
52. Thomas Schlömer, Daniel Heck, and Oliver Deussen. 2011. Farthest-Point Optimized Point Sets with Maximized Minimum Distance. In Proc. of ACM SIGGRAPH Symposium on High Performance Graphics (HPG 2011). 135–142. Google ScholarDigital Library
53. Peter-Pike J. Sloan, Charles F. Rose, III, and Michael F. Cohen. 2001. Shape by Example. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2001). 135–143. Google ScholarDigital Library
54. Breannan Smith, Fernando De Goes, and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37, 2, Article 12 (March 2018). Google ScholarDigital Library
55. Olga Sorkine and Marc Alexa. 2007. As-Rigid-as-Possible Surface Modeling. In Proc. of Eurographics Symposium on Geometry Processing (SGP 2007). 109–116.Google Scholar
56. Robert W. Sumner and Jovan Popović. 2004. Deformation Transfer for Triangle Meshes. ACM Trans. Graph. 23, 3 (Aug. 2004), 399–405. Google ScholarDigital Library
57. Robert W. Sumner, Matthias Zwicker, Craig Gotsman, and Jovan Popović. 2005. Mesh-based Inverse Kinematics. ACM Trans.Graph. 24, 3 (July 2005), 488–495. Google ScholarDigital Library
58. Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005. Robust Quasistatic Finite Elements and Flesh Simulation. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2005). 181–190. Google ScholarDigital Library
59. Rodolphe Vaillant, Loïc Barthe, Gaël Guennebaud, Marie-Paule Cani, Damien Rohmer, Brian Wyvill, Olivier Gourmel, and Mathias Paulin. 2013. Implicit Skinning: Real-time Skin Deformation with Contact Modeling. ACM Trans. Graph. 32, 4, Article 125 (July 2013). Google ScholarDigital Library
60. Rodolphe Vaillant, Gäel Guennebaud, Loïc Barthe, Brian Wyvill, and Marie-Paule Cani. 2014. Robust Iso-surface Tracking for Interactive Character Skinning. ACM Trans. Graph. 33, 6, Article 189 (Nov. 2014). Google ScholarDigital Library
61. Robert Y. Wang, Kari Pulli, and Jovan Popović. 2007. Real-time Enveloping with Rotational Regression. ACM Trans. Graph. 26, 3, Article 73 (July 2007). Google ScholarDigital Library