“Dyna: a model of dynamic human shape in motion” by Pons-Moll, Romero, Mahmood and Black

  • ©Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J. Black




    Dyna: a model of dynamic human shape in motion



    To look human, digital full-body avatars need to have soft-tissue deformations like those of real people. We learn a model of soft-tissue deformations from examples using a high-resolution 4D capture system and a method that accurately registers a template mesh to sequences of 3D scans. Using over 40,000 scans of ten subjects, we learn how soft-tissue motion causes mesh triangles to deform relative to a base 3D body model. Our Dyna model uses a low-dimensional linear subspace to approximate soft-tissue deformation and relates the subspace coefficients to the changing pose of the body. Dyna uses a second-order auto-regressive model that predicts soft-tissue deformations based on previous deformations, the velocity and acceleration of the body, and the angular velocities and accelerations of the limbs. Dyna also models how deformations vary with a person’s body mass index (BMI), producing different deformations for people with different shapes. Dyna realistically represents the dynamics of soft tissue for previously unseen subjects and motions. We provide tools for animators to modify the deformations and apply them to new stylized characters.


    1. Alexa, M., and Müller, W. 2000. Representing animations by principal components. Comp. Graph. Forum 19, 3, 411–418.Google ScholarCross Ref
    2. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Trans. Graph. 21, 3 (July), 612–619. Google ScholarDigital Library
    3. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics (TOG) 22, 3, 587–594. Google ScholarDigital Library
    4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. 24, 3, 408–416. Google ScholarDigital Library
    5. Assassi, L., Becker, M., and Magnenat-Thalmann, N. 2012. Dynamic skin deformation based on biomechanical modeling. In Proc. 25th Conf. Comp. Anim. and Social Agents.Google Scholar
    6. Aubel, A., and Thalmann, D. 2001. Interactive modeling of the human musculature. In Proc. Comp. Anim., 167–255.Google Scholar
    7. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1–89:9. Google ScholarDigital Library
    8. Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, 3794–3801. Google ScholarDigital Library
    9. Capell, S., Green, S., Curless, B., Duchamp, T., and Popović, Z. 2002. Interactive skeleton-driven dynamic deformations. ACM Trans. Graph. 21, 3 (July), 586–593. Google ScholarDigital Library
    10. Capell, S., Burkhart, M., Curless, B., Duchamp, T., and Popović, Z. 2007. Physically based rigging for deformable characters. Graph. Models 69, 1 (Jan.), 71–87. Google ScholarDigital Library
    11. Chadwick, J. E., Haumann, D. R., and Parent, R. E. 1989. Layered construction for deformable animated characters. SIGGRAPH Comput. Graph. 23, 3 (July), 243–252. Google ScholarDigital Library
    12. Chen, Y., Liu, Z., and Zhang, Z. 2013. Tensor-based human body modeling. In CVPR, 105–112. Google ScholarDigital Library
    13. de Aguiar, E., and Ukita, N. 2012. Representing mesh-based character animations. Computers & Graphics 38 (Feb.), 10–17. Google ScholarDigital Library
    14. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1–98:10. Google ScholarDigital Library
    15. de Aguiar, E., Sigal, L., Treuille, A., and Hodgins, J. K. 2010. Stable spaces for real-time clothing. ACM Trans. Graph. 29, 4 (July), 106:1–106:9. Google ScholarDigital Library
    16. Fan, Y., Litven, J., and Pai, D. K. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. 33, 4 (July), 152:1–152:9. Google ScholarDigital Library
    17. Guan, P., Reiss, L., Hirshberg, D., Weiss, A., and Black, M. J. 2012. Drape: Dressing any person. ACM Trans. Graph. 31, 4 (July), 35:1–35:10. Google ScholarDigital Library
    18. Hahn, F., Martin, S., Thomaszewski, B., Sumner, R., Coros, S., and Gross, M. 2012. Rig-space physics. ACM Trans. Graph. 31, 4 (July), 72:1–72:8. Google ScholarDigital Library
    19. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337–346.Google ScholarCross Ref
    20. Hirshberg, D. A., Loper, M., Rachlin, E., and Black, M. J. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In ECCV, vol. 7577 of LNCS. Springer, 242–255. Google ScholarDigital Library
    21. James, D. L., and Pai, D. K. 2002. Dyrt: Dynamic response textures for real time deformation simulation with graphics hardware. ACM Trans. Graph. 21, 3 (July), 582–585. Google ScholarDigital Library
    22. Karni, Z., and Gotsman, C. 2004. Compression of soft-body animation sequences. Computers & Graphics 28, 25–34.Google ScholarCross Ref
    23. Kim, T., and James, D. L. 2009. Skipping steps in deformable simulation with online model reduction. ACM Trans. Graph. 28, 5 (Dec.), 123:1–123:9. Google ScholarDigital Library
    24. Kim, T., and James, D. L. 2011. Physics-based character skinning using multi-domain subspace deformations. In Proc. ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 63–72. Google ScholarDigital Library
    25. Kry, P. G., James, D. L., and Pai, D. K. 2002. EigenSkin: Real time large deformation character skinning in hardware. In Proc. 2002 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 153–159. Google ScholarDigital Library
    26. Larboulette, C., Cani, M.-P., and Arnaldi, B. 2005. Dynamic skinning: Adding real-time dynamic effects to an existing character animation. In Proc. 21st Spring Conf. Comp. Graph., ACM, SCCG ’05, 87–93. Google ScholarDigital Library
    27. Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (Sept.), 99:1–99:17. Google ScholarDigital Library
    28. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proc. SIGGRAPH, ACM, 165–172. Google ScholarDigital Library
    29. Loper, M. M., Mahmood, N., and Black, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph. 33, 6 (Nov.), 220:1–220:13. Google ScholarDigital Library
    30. Maurel, W., Wu, Y., Magnenat-Thalmann, N., and Thalmann, D. 1998. Biomechanical Models for Soft Tissue Simulation. Springer-Verlag, Berlin.Google Scholar
    31. Metaxas, D., and Terzopoulos, D. 1993. Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15, 6 (June), 580–591. Google ScholarDigital Library
    32. Neumann, T., Varanasi, K., Hasler, N., Wacker, M., Magnor, M., and Theobalt, C. 2013. Capture and statistical modeling of arm-muscle deformations. Computer Graphics Forum 32, 2 (May), 285–294.Google ScholarCross Ref
    33. Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M., and Theobalt, C. 2013. Sparse localized deformation components. ACM Trans. Graph. 32, 6 (Nov.), 179:1–179:10. Google ScholarDigital Library
    34. Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3 (July), 881–889. Google ScholarDigital Library
    35. Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM Trans. Graph. 27, 3 (Aug.), 96:1–96:6. Google ScholarDigital Library
    36. Powell, M. 1970. A hybrid method for nonlinear equations. In Numerical Methods for Nonlinear Algebraic Equations, Gordon and Breach Science, London, P. Rabinowitz, Ed., 87–144.Google Scholar
    37. Pratscher, M., Coleman, P., Laszlo, J., and Singh, K. 2005. Outside-in anatomy based character rigging. In Proc. 2005 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 329–338. Google ScholarDigital Library
    38. Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.Google Scholar
    39. Scheepers, F., Parent, R. E., Carlson, W. E., and May, S. F. 1997. Anatomy-based modeling of the human musculature. In Proc. SIGGRAPH, ACM, 163–172. Google ScholarDigital Library
    40. Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2008. Example-based dynamic skinning in real time. ACM Trans. Graph. 27, 3 (Aug.), 29:1–29:8. Google ScholarDigital Library
    41. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3 (July), 417–425. Google ScholarDigital Library
    42. Stark, J., and Hilton, A. 2007. Surface capture for performance-based animation. IEEE Computer Graphics and Applications 27, 3, 21–31. Google ScholarDigital Library
    43. Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399–405. Google ScholarDigital Library
    44. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. Vis. and Comp. Graph. 11, 3 (May), 317–328. Google ScholarDigital Library
    45. Terzopoulos, D., and Waters, K. 1990. Physically-based facial modelling, analysis, and animation. J. Vis. and Comp. Anim. 1, 2 (Dec.), 73–80.Google ScholarCross Ref
    46. Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph. 33, 4 (July), 52:1–52:11. Google ScholarDigital Library
    47. Wilhelms, J., and Van Gelder, A. 1997. Anatomically based modeling. In Proc. SIGGRAPH, ACM, 173–180. Google ScholarDigital Library

ACM Digital Library Publication: