“High-quality single-shot capture of facial geometry” by Beeler, Bickel, Beardsley, Sumner and Gross

  • ©Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, and Markus Gross




    High-quality single-shot capture of facial geometry



    This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.


    1. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The Digital Emily Project: Photoreal facial modeling and animation. ACM Trans. Graph.Google Scholar
    2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single shot capture of facial geometry: Implementation details. Tech. Rep. 671, ETH Zurich.Google Scholar
    3. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. Google ScholarDigital Library
    4. Chen, T., Goesele, M., and Seidel, H. 2006. Mesostructure from specularity. CVPR. Google ScholarDigital Library
    5. DI3D. 2009. Dimensional imaging. http://www.di3d.com.Google Scholar
    6. Donner, C., and Jensen, H. 2006. A spectral bssrdf for shading human skin. Eurographics Symposium on Rendering.Google Scholar
    7. Donner, C., Weyrich, T., d’Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph. Google ScholarDigital Library
    8. Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. CVPR.Google Scholar
    9. Glencross, M., Ward, G., Melendez, F., Jay, C., Liu, J., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Trans. Graph. Google ScholarDigital Library
    10. Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., and Funkhouser, T. 2006. A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph. Google ScholarDigital Library
    11. Hartley, R., and Zisserman, A. 2000. Multiple View Geometry, second ed. Cambridge University Press. Google ScholarDigital Library
    12. Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Shadows in three-source photometric stereo. ECCV. Google ScholarDigital Library
    13. Hiep, V., Keriven, R., Labatut, P., and Pons, J. 2009. Towards high-resolution large-scale multi-view stereo. CVPR.Google Scholar
    14. Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. SIGGRAPH 2005 Courses. Google ScholarDigital Library
    15. Intel. 2001. Opencv reference manual. http://developer.intel.com.Google Scholar
    16. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In SGP. Google ScholarDigital Library
    17. Ma, W., Hawkins, T., Peers, P., Chabert, C., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques. Google ScholarDigital Library
    18. Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nister, D., and Pollefeys, M. 2007. Real-time visibility-based fusion of depth maps. ICCV.Google Scholar
    19. Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III.Google Scholar
    20. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. Google ScholarDigital Library
    21. Parke, F. 1974. A parametric model for human faces. PhD Thesis, University of Utah. Google ScholarDigital Library
    22. Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM Trans. Graph.Google Scholar
    23. Robert, L., and Deriche, R. 1996. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In ECCV. Google ScholarDigital Library
    24. Scharstein, D., and Szeliski, R. 1996. Stereo matching with non-linear diffusion. CVPR. Google ScholarDigital Library
    25. Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV. Google ScholarDigital Library
    26. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. CVPR. Google ScholarDigital Library
    27. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. Google ScholarDigital Library
    28. Svoboda, T. Multi camera self-calibration. http://cmp.felk.cvut.cz/svoboda/SelfCal/index.html.Google Scholar
    29. Torralba, A., and Freeman, W. 2003. Properties and applications of shape recipes. CVPR.Google Scholar
    30. Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. CVPR.Google Scholar
    31. Weise, T., Li, H., Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. SCA. Google ScholarDigital Library
    32. Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph. Google ScholarDigital Library
    33. Woodford, O., Torr, P., Reid, I., and Fitzgibbon, A. 2008. Global stereo reconstruction under second order smoothness priors. CVPR.Google Scholar

ACM Digital Library Publication: