“SMPL: a skinned multi-person linear model”
Conference:
Type(s):
Title:
- SMPL: a skinned multi-person linear model
Session/Category Title: Deformable Models
Presenter(s)/Author(s):
Abstract:
We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. We quantitatively evaluate variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend-SCAPE model trained on the same data. We also extend SMPL to realistically model dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.
References:
1. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Trans. Graph. (Proc. SIGGRAPH) 21, 3 (July), 612–619.
2. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. (Proc. SIGGRAPH) 22, 3, 587–594.
3. Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’06, 147–156.
4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. (Proc. SIGGRAPH 24, 3, 408–416.
5. Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3D characters. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (July).
6. Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 3794–3801.
7. Chang, W., and Zwicker, M. 2009. Range scan registration using reduced deformable models. Computer Graphics Forum 28, 2, 447–456.
8. Chen, Y., Liu, Z., and Zhang, Z. 2013. Tensor-based human body modeling. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 105–112.
9. 2000. CMU graphics lab motion capture database. http://mocap.cs.cmu.edu. Accessed: 2012-12-11.
10. Corazza, S., and Gambaretto, E., 2014. Automatic generation of 3D character animation from 3D meshes, Aug. 5. US Patent 8,797,328.
11. De Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008. Automatic conversion of mesh animations into skeleton-based animations. Computer Graphics Forum 27, 2, 389–397.
12. 2015. Dyna dataset. http://dyna.is.tue.mpg.de/. Accessed: 2015-05-15.
13. Freifeld, O., and Black, M. J. 2012. Lie bodies: A manifold representation of 3D human shape. In European Conf. on Computer Vision (ECCV), Springer-Verlag, A. Fitzgibbon et al. (Eds.), Ed., Part I, LNCS 7572, 1–14.
14. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337–346.
15. Hasler, N., Thormählen, T., Rosenhahn, B., and Seidel, H.-P. 2010. Learning skeletons for shape and pose. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’10, 23–30.
16. Hirshberg, D., Loper, M., Rachlin, E., and Black, M. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In European Conf. on Computer Vision (ECCV), Springer-Verlag, A. F. et al. (Eds.), Ed., LNCS 7577, Part IV, 242–255.
17. James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Trans. Graph. 24, 3 (July), 399–407.
18. Kavan, L., and Žára, J. 2005. Spherical blend skinning: A real-time deformation of articulated models. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’05, 9–16.
19. Kavan, L., Collins, S., Žára, J., and O’Sullivan, C. 2008. Geometric skinning with approximate dual quaternion blending. ACM Transactions on Graphics (TOG) 27, 4, 105:1–105:23.
20. Kavan, L., Collins, S., and O’Sullivan, C. 2009. Automatic linearization of nonlinear skinning. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’09, 49–56.
21. Kry, P. G., James, D. L., and Pai, D. K. 2002. EigenSkin: Real time large deformation character skinning in hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA ’02, 153–159.
22. Kurihara, T., and Miyata, N. 2004. Modeling deformable human hands from medical images. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’04, 355–363.
23. Lawson, C. L., and Hanson, R. J. 1995. Solving least squares problems. Classics in applied mathematics. SIAM, Philadelphia, PA. SIAM : Society of industrial and applied mathematics.
24. Le, B. H., and Deng, Z. 2012. Smooth skinning decomposition with rigid bones. ACM Trans. Graph. 31, 6 (Nov.), 199:1–199:10.
25. Le, B. H., and Deng, Z. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Trans. Graph. 33, 4 (July), 84:1–84:10.
26. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH ’00, 165–172.
27. Loper, M. M., and Black, M. J. 2014. OpenDR: An approximate differentiable renderer. In Computer Vision — ECCV 2014, Springer, Heidelberg, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds., vol. 8695 of Lecture Notes in Computer Science, 154–169.
28. Loper, M. M., Mahmood, N., and Black, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph., (Proc. SIGGRAPH Asia) 33, 6 (Nov.), 220:1–220:13.
29. Merry, B., Marais, P., and Gain, J. 2006. Animation space: A truly linear framework for character animation. ACM Trans. Graph. 25, 4 (Oct.), 1400–1423.
30. Miller, C., Arikan, O., and Fussell, D. 2010. Frankenrigs: Building character rigs from multiple sources. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’10, 31–38.
31. Mohr, A., and Gleicher, M. 2003. Building efficient, accurate character skins from examples. ACM Trans. Graph. (Proc. SIGGRAPH), 562–568.
32. Nocedal, J., and Wright, S. J. 2006. Numerical Optimization, 2nd ed. Springer, New York.
33. Pons-Moll, G., Romero, J., Mahmood, N., and Black, M. J. 2015. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics, (Proc. SIGGRAPH) 34, 4 (July), 120:1–120:14.
34. Rhee, T., Lewis, J., and Neumann, U. 2006. Real-time weighted pose-space deformation on the GPU. EUROGRAPHICS 25, 3.
35. Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.
36. Rouet, C., and Lewis, J., 1999. Method and apparatus for creating lifelike digital representations of computer animated objects by providing corrective enveloping, Mar. 16. US Patent 5,883,638.
37. Schaefer, S., and Yuksel, C. 2007. Example-based skeleton extraction. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SGP ’07, 153–162.
38. Seo, H., Cordier, F., and Magnenat-Thalmann, N. 2003. Synthesizing animatable body models with parameterized shape modifications. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’03, 120–125.
39. Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph., (Proc. SIGGRAPH) 33, 4 (July), 52:1–52:11.
40. Wang, X. C., and Phillips, C. 2002. Multi-weight enveloping: Least-squares approximation techniques for skin animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA ’02, 129–138.
41. Wang, R. Y., Pulli, K., and Popović, J. 2007. Real-time enveloping with rotational regression. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (July).
42. Weber, O., Sorkine, O., Lipman, Y., and Gotsman, C. 2007. Context-aware skeletal shape deformation. Computer Graphics Forum 26, 3 (Sept.), 265–274.
