“High-quality single-shot capture of facial geometry” by Beeler, Bickel, Beardsley, Sumner and Gross

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.

1. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The Digital Emily Project: Photoreal facial modeling and animation. ACM Trans. Graph.Google Scholar
2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single shot capture of facial geometry: Implementation details. Tech. Rep. 671, ETH Zurich.Google Scholar
3. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. Google ScholarDigital Library
4. Chen, T., Goesele, M., and Seidel, H. 2006. Mesostructure from specularity. CVPR. Google ScholarDigital Library
5. DI3D. 2009. Dimensional imaging. http://www.di3d.com.Google Scholar
6. Donner, C., and Jensen, H. 2006. A spectral bssrdf for shading human skin. Eurographics Symposium on Rendering.Google Scholar
7. Donner, C., Weyrich, T., d’Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph. Google ScholarDigital Library
8. Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. CVPR.Google Scholar
9. Glencross, M., Ward, G., Melendez, F., Jay, C., Liu, J., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Trans. Graph. Google ScholarDigital Library
10. Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., and Funkhouser, T. 2006. A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph. Google ScholarDigital Library
11. Hartley, R., and Zisserman, A. 2000. Multiple View Geometry, second ed. Cambridge University Press. Google ScholarDigital Library
12. Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Shadows in three-source photometric stereo. ECCV. Google ScholarDigital Library
13. Hiep, V., Keriven, R., Labatut, P., and Pons, J. 2009. Towards high-resolution large-scale multi-view stereo. CVPR.Google Scholar
14. Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. SIGGRAPH 2005 Courses. Google ScholarDigital Library
15. Intel. 2001. Opencv reference manual. http://developer.intel.com.Google Scholar
16. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In SGP. Google ScholarDigital Library
17. Ma, W., Hawkins, T., Peers, P., Chabert, C., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques. Google ScholarDigital Library
18. Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nister, D., and Pollefeys, M. 2007. Real-time visibility-based fusion of depth maps. ICCV.Google Scholar
19. Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III.Google Scholar
20. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. Google ScholarDigital Library
21. Parke, F. 1974. A parametric model for human faces. PhD Thesis, University of Utah. Google ScholarDigital Library
22. Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM Trans. Graph.Google Scholar
23. Robert, L., and Deriche, R. 1996. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In ECCV. Google ScholarDigital Library
24. Scharstein, D., and Szeliski, R. 1996. Stereo matching with non-linear diffusion. CVPR. Google ScholarDigital Library
25. Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV. Google ScholarDigital Library
26. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. CVPR. Google ScholarDigital Library
27. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. Google ScholarDigital Library
28. Svoboda, T. Multi camera self-calibration. http://cmp.felk.cvut.cz/svoboda/SelfCal/index.html.Google Scholar
29. Torralba, A., and Freeman, W. 2003. Properties and applications of shape recipes. CVPR.Google Scholar
30. Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. CVPR.Google Scholar
31. Weise, T., Li, H., Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. SCA. Google ScholarDigital Library
32. Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph. Google ScholarDigital Library
33. Woodford, O., Torr, P., Reid, I., and Fitzgibbon, A. 2008. Global stereo reconstruction under second order smoothness priors. CVPR.Google Scholar