“High-quality hair modeling from a single portrait photo” by Chai, Luo, Sunkavalli, Carr, Hadap, et al. … – ACM SIGGRAPH HISTORY ARCHIVES

“High-quality hair modeling from a single portrait photo” by Chai, Luo, Sunkavalli, Carr, Hadap, et al. …

  • 2015 SA Technical Papers_Chai_High-Quality Hair Modeling from A Single Portrait Photo

Conference:


Type(s):


Title:

    High-quality hair modeling from a single portrait photo

Session/Category Title:   Modelling and More

Presenter(s)/Author(s):


Abstract:


    We propose a novel system to reconstruct a high-quality hair depth map from a single portrait photo with minimal user input. We achieve this by combining depth cues such as occlusions, silhouettes, and shading, with a novel 3D helical structural prior for hair reconstruction. We fit a parametric morphable face model to the input photo and construct a base shape in the face, hair and body regions using occlusion and silhouette constraints. We then estimate the normals in the hair region via a Shape-from-Shading-based optimization that uses the lighting inferred from the face model and enforces an adaptive albedo prior that models the typical color and occlusion variations of hair. We introduce a 3D helical hair prior that captures the geometric structure of hair, and show that it can be robustly recovered from the input photo in an automatic manner. Our system combines the base shape, the normals estimated by Shape from Shading, and the 3D helical hair prior to reconstruct high-quality 3D hair models. Our single-image reconstruction closely matches the results of a state-of-the-art multi-view stereo applied on a multi-view stereo dataset. Our technique can reconstruct a wide variety of hairstyles ranging from short to long and from straight to messy, and we demonstrate the use of our 3D hair models for high-quality portrait relighting, novel view synthesis and 3D-printed portrait reliefs.

References:


    1. Barron, J. T., and Malik, J. 2012. Color constancy, intrinsic images, and shape estimation. In ECCV, Springer, 57–70.
    2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4, 40:1–40:9.
    3. Beeler, T., Bickel, B., Noris, G., Beardsley, P., Marschner, S., Sumner, R. W., and Gross, M. 2012. Coupled 3D reconstruction of sparse facial hair and skin. ACM Trans. Graph. 31, 4, 117:1–117:10.
    4. Bertails, F., Audoly, B., Cani, M.-P., Querleux, B., Leroy, F., and Lévêque, J.-L. 2006. Super-helices for predicting the dynamics of natural hair. ACM Trans. Graph. 25, 3, 1180–1187.
    5. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of SIGGRAPH, ACM, 187–194.
    6. Bonneel, N., Paris, S., van de Panne, M., Durand, F., and Drettakis, G. 2009. Single photo estimation of hair appearance. In EGSR, Eurographics Association, 1171–1180.
    7. Cao, C., Weng, Y., Zhou, S., Tong, Y., and Zhou, K. 2014. Face Warehouse: A 3D facial expression database for visual computing. IEEE Trans. TVCG 20, 3, 413–425.
    8. Chai, M., Wang, L., Weng, Y., Yu, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1–116:8.
    9. Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4, 75:1–75:8.
    10. Cherin, N., Cordier, F., and Melkemi, M. 2014. Modeling piecewise helix curves from 2D sketches. Comput. Aided Des. 46, 258–262.
    11. Durou, J.-D., Falcone, M., and Sagona, M. 2008. Numerical methods for shape-from-shading: A new survey with benchmarks. Comput. Vis. Image Underst. 109, 1, 22–43.
    12. Echevarria, J. I., Bradley, D., Gutierrez, D., and Beeler, T. 2014. Capturing and stylizing hair for 3D fabrication. ACM Trans. Graph. 33, 4, 125:1–125:11.
    13. Fuhrmann, S., Langguth, F., and Goesele, M. 2014. MVE – a multi-view reconstruction environment. In Eurographics Workshop on Graphics and Cultural Heritage, The Eurographics Association, 11–18.
    14. Garrido, P., Valgaert, L., Wu, C., and Theobalt, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32, 6, 158:1–158:10.
    15. Han, Y., Lee, J.-Y., and Kweon, I. S. 2013. High quality shape from a single RGB-D image under uncalibrated natural illumination. In ICCV, IEEE, 1617–1624.
    16. Haque, S. M., Chatterjee, A., and Govindu, V. M. 2014. High quality photometric reconstruction using a depth camera. In CVPR, IEEE, 2283–2290.
    17. Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph. 33, 4, 126:1–126:10.
    18. Hu, L., Ma, C., Luo, L., Wei, L.-Y., and Li, H. 2014. Capturing braided hairstyles. ACM Trans. Graph. 33, 6, 225:1–225:9.
    19. Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4, 125:1–125:9.
    20. Johnson, M. K., and Adelson, E. H. 2011. Shape estimation in natural illumination. In CVPR, IEEE, 2553–2560.
    21. Karsch, K., Liao, Z., Rock, J., Barron, J. T., and Hoiem, D. 2013. Boundary cues for 3D object shape recovery. In CVPR, IEEE, 2163–2170.
    22. Karypis, G., and Kumar, V. 1998. Multilevel K-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput. 48, 1, 96–129.
    23. Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4, 76:1–76:12.
    24. Marschner, S. R., Jensen, H. W., Cammarano, M., Worley, S., and Hanrahan, P. 2003. Light scattering from human hair fibers. ACM Trans. Graph. 22, 3, 780–791.
    25. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3D geometry. ACM Trans. Graph. 24, 3, 536–543.
    26. Oxholm, G., and Nishino, K. 2012. Shape and reflectance from natural illumination. In ECCV, Springer, 528–541.
    27. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1–30:9.
    28. Suwajanakorn, S., Kemelmacher-Shlizerman, I., and Seitz, S. M. 2014. Total moving face reconstruction. In ECCV, Springer, 796–812.
    29. Sýkora, D., Kavan, L., Čadik, M., Jamriška, O., Jacobson, A., Whited, B., Simmons, M., and Sorkine-Hornung, O. 2014. Ink-and-ray: Bas-relief meshes for adding global illumination effects to hand-drawn characters. ACM Trans. Graph. 33, 2, 16:1–16:15.
    30. Valgaerts, L., Wu, C., Bruhn, A., Seidel, H.-P., and Theobalt, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. ACM Trans. Graph. 31, 6, 187:1–187:11.
    31. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426–433.
    32. Wither, J., Bertails, F., and Cani, M.-P. 2007. Realistic hair from a sketch. In SMI, IEEE, 33–42.
    33. Wu, C., Varanasi, K., Liu, Y., Seidel, H.-P., and Theobalt, C. 2011. Shading-based dynamic shape refinement from multi-view video under general illumination. In ICCV, IEEE, 1108–1115.
    34. Zhang, R., Tsai, P.-S., Cryer, J. E., and Shah, M. 1999. Shape from shading: A survey. IEEE Trans. PAMI 21, 8, 690–706.

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org