“Coupled 3D reconstruction of sparse facial hair and skin” by Beeler, Bickel, Noris, Beardsley, Marschner, et al. …

  • ©Thabo Beeler, Bernd Bickel, Gioacchino Noris, Paul Beardsley, Steve Marschner, Robert W. Sumner, and Markus Gross




    Coupled 3D reconstruction of sparse facial hair and skin



    Although facial hair plays an important role in individual expression, facial-hair reconstruction is not addressed by current face-capture systems. Our research addresses this limitation with an algorithm that treats hair and skin surface capture together in a coupled fashion so that a high-quality representation of hair fibers as well as the underlying skin surface can be reconstructed. We propose a passive, camera-based system that is robust against arbitrary motion since all data is acquired within the time period of a single exposure. Our reconstruction algorithm detects and traces hairs in the captured images and reconstructs them in 3D using a multiview stereo approach. Our coupled skin-reconstruction algorithm uses information about the detected hairs to deliver a skin surface that lies underneath all hairs irrespective of occlusions. In dense regions like eyebrows, we employ a hair-synthesis method to create hair fibers that plausibly match the image data. We demonstrate our scanning system on a number of individuals and show that it can successfully reconstruct a variety of facial-hair styles together with the underlying skin surface.


    1. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The digital Emily project: photo-real facial modeling and animation. In ACM SIGGRAPH 2009 Courses, ACM, New York, NY, USA, SIGGRAPH ’09, 12:1–12:15. Google ScholarDigital Library
    2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 4 (July), 40:1–40:9. Google ScholarDigital Library
    3. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. (Proc. SIGGRAPH) 30, 4 (Aug.), 75:1–75:10. Google ScholarDigital Library
    4. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH ’99, 187–194. Google ScholarDigital Library
    5. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. (Proc. SIGGRPAH) 29 (July), 41:1–41:10. Google ScholarDigital Library
    6. Canny, J. 1986. A computational approach to edge detection. Pattern Analysis and Machine Intelligence, IEEE Trans. PAMI-8, 6 (nov.), 679–698. Google ScholarDigital Library
    7. Dalí, S., and Halsman, P. 1954. Dali’s mustache: a photographic interview. Flammarion.Google Scholar
    8. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH ’00, 145–156. Google ScholarDigital Library
    9. Furukawa, Y., and Ponce, J. 2009. Dense 3D motion capture for human faces. In Computer Vision and Pattern Recognition, 1674–1681.Google Scholar
    10. Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. 2011. Multiview face capture using polarized spherical gradient illumination. ACM, New York, NY, USA, vol. 30, 129:1–129:10. Google ScholarDigital Library
    11. Grabli, S., Sillion, F. X., Marschner, S. R., and Lengyel, J. E. 2002. Image-based hair capture by inverse lighting. In Proceedings of Graphics Interface (GI), 51–58.Google Scholar
    12. Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 28, 5 (Dec.), 164:1–164:9. Google ScholarDigital Library
    13. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. Proceedings of the fourth Eurographics symposium on Geometry processing, 61–70. Google ScholarDigital Library
    14. Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. E. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques 9. Google ScholarDigital Library
    15. Paris, S., Briceño, H. M., and Sillion, F. X. 2004. Capture of hair geometry from multiple images. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (Aug.), 712–719. Google ScholarDigital Library
    16. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3 (Aug.), 30:1–30:9. Google ScholarDigital Library
    17. Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3 (July), 816–820. Google ScholarDigital Library
    18. Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H. W., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (July), 1013–1024. Google ScholarDigital Library

ACM Digital Library Publication: