“Data-driven physics for human soft tissue animation” by Kim, Pons-Moll, Pujades, Bang, Kim, et al. …

  • ©

Conference:


Type(s):


Title:

    Data-driven physics for human soft tissue animation

Session/Category Title:   Clever Solids

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Data driven models of human poses and soft-tissue deformations can produce very realistic results, but they only model the visible surface of the human body and cannot create skin deformation due to interactions with the environment. Physical simulations can generalize to external forces, but their parameters are difficult to control. In this paper, we present a layered volumetric human body model learned from data. Our model is composed of a data-driven inner layer and a physics-based external layer. The inner layer is driven with a volumetric statistical body model (VSMPL). The soft tissue layer consists of a tetrahedral mesh that is driven using the finite element method (FEM). Model parameters, namely the segmentation of the body into layers and the soft tissue elasticity, are learned directly from 4D registrations of humans exhibiting soft tissue deformations. The learned two layer model is a realistic full-body avatar that generalizes to novel motions and external forces. Experiments show that the resulting avatars produce realistic results on held out sequences and react to external forces. Moreover, the model supports the retargeting of physical properties from one avatar when they share the same topology.

References:


    1. Dicko AliHamadi, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure, Olivier Palombi, and MariePaule Cani. 2013. Anatomy Transfer. ACM Trans. Graph. 32, 6, Article 188 (Nov. 2013), 8 pages. Google ScholarDigital Library
    2. Nadia Alkhouli, Jessica Mansfield, Ellen Green, James Bell, Beatrice Knight, Neil Liversedge, Ji Chung Tham, Richard Welbourn, Angela C Shore, Katarina Kos, and others. 2013. The mechanical properties of human adipose tissues and their relationships to the structure and composition of the extracellular matrix. American Journal of Physiology-Endocrinology and Metabolism 305, 12 (2013), E1427–E1435. Google ScholarCross Ref
    3. Brett Allen, Brian Curless, Zoran Popović, and Aaron Hertzmann. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 147–156.Google ScholarDigital Library
    4. Steven S An, Theodore Kim, and Doug L James. 2008. Optimizing cubature for efficient integration of subspace deformations. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 165.Google ScholarDigital Library
    5. Alexis Angelidis and Karan Singh. 2007. Kinodynamic Skinning Using Volume-preserving Deformations. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’07). 129–140. http://dl.acm.org/citation.cfm?id=1272690.1272709Google ScholarDigital Library
    6. Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: shape completion and animation of people. In ACM Transactions on Graphics (TOG), Vol. 24. ACM, 408–416. Google ScholarDigital Library
    7. Oscar Kin-Chung Au, Chiew-Lan Tai, Hung-Kuo Chu, Daniel Cohen-Or, and Tong-Yee Lee. 2008. Skeleton extraction by mesh contraction. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 44. Google ScholarDigital Library
    8. Ilya Baran and Jovan Popović. 2007. Automatic rigging and animation of 3d characters. In ACM Transactions on Graphics (TOG), Vol. 26. ACM, 72.Google ScholarDigital Library
    9. Jernej Barbič and Doug L James. 2005. Real-time subspace integration for St. Venant-Kirchhoff deformable models. In ACM transactions on graphics (TOG), Vol. 24. ACM, 982–990.Google Scholar
    10. Xiang Chen, Changxi Zheng, Weiwei Xu, and Kun Zhou. 2014. An Asymptotic Numerical Method for Inverse Elastic Shape Design. ACM Trans. Graph. 33, 4, Article 95 (July 2014), 11 pages. Google ScholarDigital Library
    11. Crispin Deul and Jan Bender. 2013. Physically-Based Character Skinning. VRIPHYS 13 (2013), 25–34.Google Scholar
    12. Olivier Dionne and Martin de Lasa. 2013. Geodesic voxel binding for production character meshes. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. ACM, 173–180. Google ScholarDigital Library
    13. Ming Gao, Nathan Mitchell, and Eftychios Sifakis. 2014. Steklov-Poincaré Skinning. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’14). 139–148. http://dl.acm.org/citation.cfm?id=2849517.2849541Google Scholar
    14. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rigspace Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012), 8 pages. Google ScholarDigital Library
    15. Naoya Iwamoto, Hubert PH Shum, Longzhi Yang, and Shigeo Morishima. 2015. Multilayer Lattice Model for Real-Time Dynamic Character Deformation. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 99–109.Google Scholar
    16. Alec Jacobson, Ilya Baran, Ladislav Kavan, Jovan Popović, and Olga Sorkine. 2012. Fast Automatic Skinning Transformations. ACM Trans. Graph. 31, 4, Article 77 (July 2012), 10 pages. Google ScholarDigital Library
    17. Alec Jacobson, Ilya Baran, Jovan Popovic, and Olga Sorkine. 2011. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph. 30, 4 (2011), 78. Google ScholarDigital Library
    18. Doug L. James and Dinesh K. Pai. 2002. DyRT: Dynamic Response Textures for Real Time Deformation Simulation with Graphics Hardware. ACM Trans. Graph. 21, 3 (July 2002), 582–585. Google ScholarDigital Library
    19. Doug L. James and Christopher D. Twigg. 2005. Skinning Mesh Animations. In ACM SIGGRAPH 2005 Papers (SIGGRAPH ’05). ACM, New York, NY, USA, 399–407. Google ScholarDigital Library
    20. Pushkar Joshi, Mark Meyer, Tony DeRose, Brian Green, and Tom Sanocki. 2007. Harmonic Coordinates for Character Articulation. ACM Trans. Graph. 26, 3, Article 71 (July 2007). Google ScholarDigital Library
    21. Petr Kadleček, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6, Article 213 (Nov. 2016), 13 pages. Google ScholarDigital Library
    22. Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O’Sullivan. 2008. Geometric skinning with approximate dual quaternion blending. ACM Transactions on Graphics (TOG) 27, 4 (2008), 105.Google ScholarDigital Library
    23. Ladislav Kavan and Olga Sorkine. 2012. Elasticity-inspired Deformers for Character Articulation. ACM Trans. Graph. 31, 6, Article 196 (Nov. 2012), 8 pages. Google ScholarDigital Library
    24. Junggon Kim and Nancy S. Pollard. 2011. Fast Simulation of Skeleton-driven Deformable Body Characters. ACM Trans. Graph. 30, 5, Article 121 (Oct. 2011), 19 pages. Google ScholarDigital Library
    25. Theodore Kim and Doug L. James. 2011. Physics-based Character Skinning Using Multi-domain Subspace Deformations. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’11). 63–72. Google ScholarDigital Library
    26. Yeara Kozlov, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross. 2017. Enriching Facial Blendshape Rigs with Physical Simulation. Computer Graphics Forum 36, 2 (2017). Google ScholarCross Ref
    27. Paul G. Kry, Doug L. James, and Dinesh K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’02). 153–159. Google ScholarDigital Library
    28. Caroline Larboulette, Marie-Paule Cani, and Bruno Arnaldi. 2005. Dynamic skinning: adding real-time dynamic effects to an existing character animation. In Proceedings of the 21st spring conference on Computer graphics. ACM, 87–93. Google ScholarDigital Library
    29. Binh Huy Le and Jessica K Hodgins. 2016. Real-time skeletal skinning with optimized centers of rotation. ACM Transactions on Graphics (TOG) 35, 4 (2016), 37.Google ScholarDigital Library
    30. Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009), 17 pages. Google ScholarDigital Library
    31. Yuencheng Lee, Demetri Terzopoulos, and Keith Waters. 1995. Realistic Modeling for Facial Animation. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’95). ACM, New York, NY, USA, 55–62. Google ScholarDigital Library
    32. J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-driven Deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’00). 165–172. Google ScholarDigital Library
    33. Duo Li, Shinjiro Sueda, Debanga R. Neog, and Dinesh K. Pai. 2013. Thin Skin Elastodynamics. ACM Trans. Graph. 32, 4, Article 49 (July 2013), 10 pages. Google ScholarDigital Library
    34. Siwang Li, Jin Huang, Fernando de Goes, Xiaogang Jin, Hujun Bao, and Mathieu Desbrun. 2014. Space-time editing of elastic motion through material optimization and reduction. ACM Transactions on Graphics (TOG) 33, 4 (2014), 108.Google ScholarDigital Library
    35. Yaron Lipman, David Levin, and Daniel Cohen-Or. 2008. Green Coordinates. ACM Trans. Graph. 27, 3, Article 78 (Aug. 2008), 10 pages. Google ScholarDigital Library
    36. Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-driven Soft Body Characters. ACM Trans. Graph. 32, 6, Article 215 (Nov. 2013), 8 pages. Google ScholarDigital Library
    37. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2015. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics (TOG) 34, 6 (2015), 248.Google ScholarDigital Library
    38. M.I.A. Lourakis. Jul. 2004. levmar: Levenberg-Marquardt nonlinear least squares algorithms in C/C++. http://www.ics.forth.gr/lourakis/levmar/. (Jul. 2004).Google Scholar
    39. Nadia Magnenat-Thalmann, Richard Laperrire, and Daniel Thalmann. 1988. Joint-dependent local deformations for hand animation and object grasping. In Proceedings on Graphics interface’88. Citeseer.Google Scholar
    40. Richard Malgat, Benjamin Gilles, David IW Levin, Matthieu Nesme, and François Faure. 2015. Multifarious hierarchies of mechanical models for artist assigned levels-of-detail. In Proceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 27–36.Google ScholarDigital Library
    41. Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011. Efficient Elasticity for Character Skinning with Contact and Collisions. ACM Trans. Graph. 30, 4, Article 37 (July 2011), 12 pages. Google ScholarDigital Library
    42. Eder Miguel, Derek Bradley, Bernhard Thomaszewski, Bernd Bickel, Wojciech Matusik, Miguel A Otaduy, and Steve Marschner. 2012. Data-Driven Estimation of Cloth Simulation Models. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 519–528.Google Scholar
    43. Alex Mohr and Michael Gleicher. 2003. Building Efficient, Accurate Character Skins from Examples. In ACM SIGGRAPH 2003 Papers (SIGGRAPH ’03). ACM, New York, NY, USA, 562–568. Google ScholarDigital Library
    44. Tomohiko Mukai and Shigeru Kuriyama. 2016. Efficient Dynamic Skinning with Low-rank Helper Bone Controllers. ACM Trans. Graph. 35, 4, Article 36 (July 2016), 11 pages. Google ScholarDigital Library
    45. Matthias Müller and Markus Gross. 2004. Interactive virtual materials. In Proceedings of Graphics Interface 2004. 239–246.Google Scholar
    46. Akihiko Murai, Q Youn Hong, Katsu Yamane, and Jessica K Hodgins. 2016. Dynamic skin deformation simulation using musculoskeletal model and soft tissue dynamics. Computational Visual Media (2016), 1–12.Google Scholar
    47. Sang Il Park and Jessica K Hodgins. 2006. Capturing and animating skin deformation in human motion. In ACM Transactions on Graphics (TOG), Vol. 25. ACM, 881–889.Google ScholarDigital Library
    48. Sang Il Park and Jessica K Hodgins. 2008. Data-driven modeling of skin and muscle deformation. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 96.Google ScholarDigital Library
    49. A. Pentland and J. Williams. 1989. Good Vibrations: Modal Dynamics for Graphics and Animation. Vol. 23. 207–214 pages.Google ScholarDigital Library
    50. Gerard Pons-Moll, Sergi Pujades, Sonny Hu, and Michael Black. 2017. ClothCap: Seamless 4D Clothing Capture and Retargeting. ACM Transactions on Graphics, (Proc. SIGGRAPH) 36, 4 (2017).Google Scholar
    51. Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J Black. 2015. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics (TOG) 34, 4 (2015), 120.Google ScholarDigital Library
    52. Gerard Pons-Moll and Bodo Rosenhahn. 2011. Model-Based Pose Estimation. Springer, Chapter 9, 139–170. Google ScholarCross Ref
    53. Hyewon Seo, Frederic Cordier, and Nadia Magnenat-Thalmann. 2003. Synthesizing animatable body models with parameterized shape modifications. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation. 120–125.Google ScholarDigital Library
    54. Hang Si. 2015. TetGen, a Quality Tetrahedral Mesh Generator. AMC Trans. Math. Softw. 41, 2 (2015), 11. http://wias-berlin.de/software/tetgen/Google ScholarDigital Library
    55. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic Biomechanical Simulation and Control of Human Swimming. ACM Trans. Graph. 34, 1, Article 10 (Dec. 2014), 15 pages. Google ScholarDigital Library
    56. PeterPike J Sloan, Charles F Rose III, and Michael F Cohen. 2001. Shape by example. In Proceedings of the 2001 symposium on Interactive 3D graphics. ACM, 135–143.Google ScholarDigital Library
    57. Timo von Marcard, Gerard Pons-Moll, and Bodo Rosenhahn. 2016. Human Pose Estimation from Video and IMUs. Transactions on Pattern Analysis and Machine Intelligence PAMI (Jan. 2016).Google Scholar
    58. Timo von Marcard, Bodo Rosenhahn, Michael Black, and Gerard Pons-Moll. 2017. Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs. Computer Graphics Forum 36(2), Proceedings of the 38th Annual Conference of the European Association for Computer Graphics (Eurographics) (2017).Google Scholar
    59. Bin Wang, Longhua Wu, KangKang Yin, Uri Ascher, Libin Liu, and Hui Huang. 2015. Deformation capture and modeling of soft objects. ACM Transactions on Graphics (TOG) 34, 4 (2015), 94.Google ScholarDigital Library
    60. Xiaohuan Corina Wang and Cary Phillips. 2002. Multi-weight enveloping: least-squares approximation techniques for skin animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. ACM, 129–138. Google ScholarDigital Library
    61. Xiaofeng Wu, Rajaditya Mukherjee, and Huamin Wang. 2015. A Unified Approach for Subspace Simulation of Deformable Bodies in Multiple Domains. ACM Trans. Graph. 34, 6, Article 241 (Oct. 2015), 9 pages. Google ScholarDigital Library
    62. Hongyi Xu and Jernej Barbič. 2016. Pose-space Subspace Dynamics. ACM Trans. Graph. 35, 4, Article 35 (July 2016), 14 pages. Google ScholarDigital Library
    63. Hongyi Xu, Yijing Li, Yong Chen, and Jernej Barbivč. 2015. Interactive material design using model reduction. ACM Transactions on Graphics (TOG) 34, 2 (2015), 18.Google ScholarDigital Library
    64. Yin Yang, Weiwei Xu, Xiaohu Guo, Kun Zhou, and Baining Guo. 2013. Boundary-Aware Multidomain Subspace Deformation. IEEE Transactions on Visualization and Computer Graphics 19, 10 (2013), 1633–1645. DOI:https://doi.org/doi.ieeecomputersociety.org/10.1109/TVCG.2013.12Google ScholarCross Ref
    65. Chao Zhang, Sergi Pujades, Michael Black, and Gerard Pons-Moll. 2017. Detailed, accurate, human shape estimation from clothed 3D scan sequences. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
    66. Lifeng Zhu, Xiaoyan Hu, and Ladislav Kavan. 2015. Adaptable Anatomical Models for Realistic Bone Motion Reconstruction. Comput. Graph. Forum 34, 2 (May 2015), 459–471. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: