“Dyna: a model of dynamic human shape in motion” by Pons-Moll, Romero, Mahmood and Black

To look human, digital full-body avatars need to have soft-tissue deformations like those of real people. We learn a model of soft-tissue deformations from examples using a high-resolution 4D capture system and a method that accurately registers a template mesh to sequences of 3D scans. Using over 40,000 scans of ten subjects, we learn how soft-tissue motion causes mesh triangles to deform relative to a base 3D body model. Our Dyna model uses a low-dimensional linear subspace to approximate soft-tissue deformation and relates the subspace coefficients to the changing pose of the body. Dyna uses a second-order auto-regressive model that predicts soft-tissue deformations based on previous deformations, the velocity and acceleration of the body, and the angular velocities and accelerations of the limbs. Dyna also models how deformations vary with a person’s body mass index (BMI), producing different deformations for people with different shapes. Dyna realistically represents the dynamics of soft tissue for previously unseen subjects and motions. We provide tools for animators to modify the deformations and apply them to new stylized characters.

1. Alexa, M., and Müller, W. 2000. Representing animations by principal components. Comp. Graph. Forum 19, 3, 411–418.Google ScholarCross Ref
2. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Trans. Graph. 21, 3 (July), 612–619. Google ScholarDigital Library
3. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics (TOG) 22, 3, 587–594. Google ScholarDigital Library
4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. 24, 3, 408–416. Google ScholarDigital Library
5. Assassi, L., Becker, M., and Magnenat-Thalmann, N. 2012. Dynamic skin deformation based on biomechanical modeling. In Proc. 25th Conf. Comp. Anim. and Social Agents.Google Scholar
6. Aubel, A., and Thalmann, D. 2001. Interactive modeling of the human musculature. In Proc. Comp. Anim., 167–255.Google Scholar
7. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1–89:9. Google ScholarDigital Library
8. Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, 3794–3801. Google ScholarDigital Library
9. Capell, S., Green, S., Curless, B., Duchamp, T., and Popović, Z. 2002. Interactive skeleton-driven dynamic deformations. ACM Trans. Graph. 21, 3 (July), 586–593. Google ScholarDigital Library
10. Capell, S., Burkhart, M., Curless, B., Duchamp, T., and Popović, Z. 2007. Physically based rigging for deformable characters. Graph. Models 69, 1 (Jan.), 71–87. Google ScholarDigital Library
11. Chadwick, J. E., Haumann, D. R., and Parent, R. E. 1989. Layered construction for deformable animated characters. SIGGRAPH Comput. Graph. 23, 3 (July), 243–252. Google ScholarDigital Library
12. Chen, Y., Liu, Z., and Zhang, Z. 2013. Tensor-based human body modeling. In CVPR, 105–112. Google ScholarDigital Library
13. de Aguiar, E., and Ukita, N. 2012. Representing mesh-based character animations. Computers & Graphics 38 (Feb.), 10–17. Google ScholarDigital Library
14. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1–98:10. Google ScholarDigital Library
15. de Aguiar, E., Sigal, L., Treuille, A., and Hodgins, J. K. 2010. Stable spaces for real-time clothing. ACM Trans. Graph. 29, 4 (July), 106:1–106:9. Google ScholarDigital Library
16. Fan, Y., Litven, J., and Pai, D. K. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. 33, 4 (July), 152:1–152:9. Google ScholarDigital Library
17. Guan, P., Reiss, L., Hirshberg, D., Weiss, A., and Black, M. J. 2012. Drape: Dressing any person. ACM Trans. Graph. 31, 4 (July), 35:1–35:10. Google ScholarDigital Library
18. Hahn, F., Martin, S., Thomaszewski, B., Sumner, R., Coros, S., and Gross, M. 2012. Rig-space physics. ACM Trans. Graph. 31, 4 (July), 72:1–72:8. Google ScholarDigital Library
19. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337–346.Google ScholarCross Ref
20. Hirshberg, D. A., Loper, M., Rachlin, E., and Black, M. J. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In ECCV, vol. 7577 of LNCS. Springer, 242–255. Google ScholarDigital Library
21. James, D. L., and Pai, D. K. 2002. Dyrt: Dynamic response textures for real time deformation simulation with graphics hardware. ACM Trans. Graph. 21, 3 (July), 582–585. Google ScholarDigital Library
22. Karni, Z., and Gotsman, C. 2004. Compression of soft-body animation sequences. Computers & Graphics 28, 25–34.Google ScholarCross Ref
23. Kim, T., and James, D. L. 2009. Skipping steps in deformable simulation with online model reduction. ACM Trans. Graph. 28, 5 (Dec.), 123:1–123:9. Google ScholarDigital Library
24. Kim, T., and James, D. L. 2011. Physics-based character skinning using multi-domain subspace deformations. In Proc. ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 63–72. Google ScholarDigital Library
25. Kry, P. G., James, D. L., and Pai, D. K. 2002. EigenSkin: Real time large deformation character skinning in hardware. In Proc. 2002 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 153–159. Google ScholarDigital Library
26. Larboulette, C., Cani, M.-P., and Arnaldi, B. 2005. Dynamic skinning: Adding real-time dynamic effects to an existing character animation. In Proc. 21st Spring Conf. Comp. Graph., ACM, SCCG ’05, 87–93. Google ScholarDigital Library
27. Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (Sept.), 99:1–99:17. Google ScholarDigital Library
28. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proc. SIGGRAPH, ACM, 165–172. Google ScholarDigital Library
29. Loper, M. M., Mahmood, N., and Black, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph. 33, 6 (Nov.), 220:1–220:13. Google ScholarDigital Library
30. Maurel, W., Wu, Y., Magnenat-Thalmann, N., and Thalmann, D. 1998. Biomechanical Models for Soft Tissue Simulation. Springer-Verlag, Berlin.Google Scholar
31. Metaxas, D., and Terzopoulos, D. 1993. Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15, 6 (June), 580–591. Google ScholarDigital Library
32. Neumann, T., Varanasi, K., Hasler, N., Wacker, M., Magnor, M., and Theobalt, C. 2013. Capture and statistical modeling of arm-muscle deformations. Computer Graphics Forum 32, 2 (May), 285–294.Google ScholarCross Ref
33. Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M., and Theobalt, C. 2013. Sparse localized deformation components. ACM Trans. Graph. 32, 6 (Nov.), 179:1–179:10. Google ScholarDigital Library
34. Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3 (July), 881–889. Google ScholarDigital Library
35. Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM Trans. Graph. 27, 3 (Aug.), 96:1–96:6. Google ScholarDigital Library
36. Powell, M. 1970. A hybrid method for nonlinear equations. In Numerical Methods for Nonlinear Algebraic Equations, Gordon and Breach Science, London, P. Rabinowitz, Ed., 87–144.Google Scholar
37. Pratscher, M., Coleman, P., Laszlo, J., and Singh, K. 2005. Outside-in anatomy based character rigging. In Proc. 2005 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 329–338. Google ScholarDigital Library
38. Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.Google Scholar
39. Scheepers, F., Parent, R. E., Carlson, W. E., and May, S. F. 1997. Anatomy-based modeling of the human musculature. In Proc. SIGGRAPH, ACM, 163–172. Google ScholarDigital Library
40. Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2008. Example-based dynamic skinning in real time. ACM Trans. Graph. 27, 3 (Aug.), 29:1–29:8. Google ScholarDigital Library
41. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3 (July), 417–425. Google ScholarDigital Library
42. Stark, J., and Hilton, A. 2007. Surface capture for performance-based animation. IEEE Computer Graphics and Applications 27, 3, 21–31. Google ScholarDigital Library
43. Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399–405. Google ScholarDigital Library
44. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. Vis. and Comp. Graph. 11, 3 (May), 317–328. Google ScholarDigital Library
45. Terzopoulos, D., and Waters, K. 1990. Physically-based facial modelling, analysis, and animation. J. Vis. and Comp. Anim. 1, 2 (Dec.), 73–80.Google ScholarCross Ref
46. Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph. 33, 4 (July), 52:1–52:11. Google ScholarDigital Library
47. Wilhelms, J., and Van Gelder, A. 1997. Anatomically based modeling. In Proc. SIGGRAPH, ACM, 173–180. Google ScholarDigital Library