“AutoHair: fully automatic hair modeling from a single image”

  • ©Pascal Bérard, Derek Bradley, Markus Gross, and Thabo Beeler



Session Title:



    AutoHair: fully automatic hair modeling from a single image




    We introduce AutoHair, the first fully automatic method for 3D hair modeling from a single portrait image, with no user interaction or parameter tuning. Our method efficiently generates complete and high-quality hair geometries, which are comparable to those generated by the state-of-the-art methods, where user interaction is required. The core components of our method are: a novel hierarchical deep neural network for automatic hair segmentation and hair growth direction estimation, trained over an annotated hair image database; and an efficient and automatic data-driven hair matching and modeling algorithm, based on a large set of 3D hair exemplars. We demonstrate the efficacy and robustness of our method on Internet photos, resulting in a database of around 50K 3D hair models and a corresponding hairstyle space that covers a wide variety of real-world hairstyles. We also show novel applications enabled by our method, including 3D hairstyle space navigation and hair-aware image retrieval.


    1. Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In CVPR, 1–8.Google Scholar
    2. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH ’99, 187–194. Google ScholarDigital Library
    3. Bookstein, F. L. 1989. Principal warps: thin-plate splines and the decomposition of deformations. Trans. PAMI 11, 6, 567–585. Google ScholarDigital Library
    4. Cao, X., Wei, Y., Wen, F., and Sun, J. 2012. Face alignment by explicit shape regression. In CVPR, 2887–2894. Google ScholarDigital Library
    5. Chai, M., Wang, L., Weng, Y., Yu, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1–116:8. Google ScholarDigital Library
    6. Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4, 75:1–75:8. Google ScholarDigital Library
    7. Chai, M., Luo, L., Sunkavalli, K., Carr, N., Hadap, S., and Zhou, K. 2015. High-quality hair modeling from a single portrait photo. ACM Trans. Graph. 34, 6, 204:1–204:10. Google ScholarDigital Library
    8. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. 2015. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR.Google Scholar
    9. Cheng, M.-M., Zheng, S., Lin, W.-Y., Vineet, V., Sturgess, P., Crook, N., Mitra, N. J., and Torr, P. 2014. Imagespirit: Verbal guided image parsing. ACM Trans. Graph. 34, 1, 3:1–3:11. Google ScholarDigital Library
    10. Dai, J., He, K., and Sun, J. 2015. BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In ICCV, 1635–1643. Google ScholarDigital Library
    11. Desbrun, M., Meyer, M., Schröder, P., and Barr, A. H. 1999. Implicit fairing of irregular meshes using diffusion and curvature flow. In Proc. SIGGRAPH ’99, 317–324. Google ScholarDigital Library
    12. Echevarria, J. I., Bradley, D., Gutierrez, D., and Beeler, T. 2014. Capturing and stylizing hair for 3D fabrication. ACM Trans. Graph. 33, 4, 125:1–125:11. Google ScholarDigital Library
    13. Electronic Arts, 2014. The Sims Resource. http://www.thesimsresource.com/.Google Scholar
    14. Forney Jr, G. D. 1973. The Viterbi algorithm. Proceedings of the IEEE 61, 3, 268–278.Google ScholarCross Ref
    15. Gephi, 2016. The Open Graph Viz Platform. https://gephi.org.Google Scholar
    16. Girshick, R., Donahue, J., Darrell, T., and Malik, J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 580–587. Google ScholarDigital Library
    17. Herrera, T. L., Zinke, A., and Weber, A. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Trans. Graph. 31, 6, 146:1–146:9. Google ScholarDigital Library
    18. Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph. 33, 4, 126:1–126:10. Google ScholarDigital Library
    19. Hu, L., Ma, C., Luo, L., Wei, L.-Y., and Li, H. 2014. Capturing braided hairstyles. ACM Trans. Graph. 33, 6, 225:1–225:9. Google ScholarDigital Library
    20. Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4, 125:1–125:9. Google ScholarDigital Library
    21. Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.-Y., Teng, S.-H., Bao, H., Guo, B., and Shum, H.-Y. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3, 1126–1134. Google ScholarDigital Library
    22. Huang, S.-S., Shamir, A., Shen, C.-H., Zhang, H., Sheffer, A., Hu, S.-M., and Cohen-Or, D. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Trans. Graph. 32, 4, 71:1–71:10. Google ScholarDigital Library
    23. Hueting, M., Ovsjanikov, M., and Mitra, N. J. 2015. CrossLink: Joint understanding of image and 3D model collections through shape and camera pose variations. ACM Trans. Graph. 34, 6, 233:1–233:13. Google ScholarDigital Library
    24. Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1–164:9. Google ScholarDigital Library
    25. Jia, Y., 2013. Caffe: An open source convolutional architecture for fast feature embedding.Google Scholar
    26. Karypis, G., and Kumar, V. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing 20, 1, 359–392. Google ScholarDigital Library
    27. Kim, V. G., Li, W., Mitra, N. J., DiVerdi, S., and Funkhouser, T. 2012. Exploring collections of 3D models using fuzzy correspondences. ACM Trans. Graph. 31, 4, 54:1–54:11. Google ScholarDigital Library
    28. Krähenbühl, P., and Koltun, V. 2011. Efficient inference in fully connected CRFs with Gaussian edge potentials. In NIPS, 109–117.Google Scholar
    29. Li, Y., Su, H., Qi, C. R., Fish, N., Cohen-Or, D., and Guibas, L. J. 2015. Joint embeddings of shapes and images via CNN image purification. ACM Trans. Graph. 34, 6, 234:1–234:12. Google ScholarDigital Library
    30. Ling, H., and Okada, K. 2007. An efficient earth mover’s distance algorithm for robust histogram comparison. Trans. PAMI 29, 5, 840–853. Google ScholarDigital Library
    31. Liu, J., Sun, J., and Shum, H.-Y. 2009. Paint selection. ACM Trans. Graph. 28, 3, 69:1–69:7. Google ScholarDigital Library
    32. Liu, Z., Li, X., Luo, P., Loy, C. C., and Tang, X. 2015. Semantic image segmentation via deep parsing network. In ICCV, 1377–1385. Google ScholarDigital Library
    33. Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 2, 129–137. Google ScholarDigital Library
    34. Long, J., Shelhamer, E., and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR, 3431–3440.Google Scholar
    35. Luo, P., Wang, X., and Tang, X. 2012. Hierarchical face parsing via deep learning. In CVPR, 2480–2487. Google ScholarDigital Library
    36. Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4, 76:1–76:12. Google ScholarDigital Library
    37. O’Donovan, P., Lībeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory font selection using crowdsourced attributes. ACM Trans. Graph. 33, 4, 92:1–92:9. Google ScholarDigital Library
    38. Ovsjanikov, M., Li, W., Guibas, L., and Mitra, N. J. 2011. Exploration of continuous variability in collections of 3D shapes. ACM Trans. Graph. 30, 4, 33:1–33:10. Google ScholarDigital Library
    39. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1–30:9. Google ScholarDigital Library
    40. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. In ICCV, 1–8.Google Scholar
    41. Ren, S., He, K., Girshick, R., and Sun, J. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 91–99. Google ScholarDigital Library
    42. Selle, A., Lentine, M., and Fedkiw, R. 2008. A mass spring model for hair simulation. ACM Trans. Graph. 27, 3, 64:1–64:11. Google ScholarDigital Library
    43. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 1–15. Google ScholarDigital Library
    44. Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
    45. Smith, B. M., Zhang, L., Brandt, J., Lin, Z., and Yang, J. 2013. Exemplar-based face parsing. In CVPR, 3484–3491. Google ScholarDigital Library
    46. Uijlings, J. R., van de Sande, K. E., Gevers, T., and Smeulders, A. W. 2013. Selective search for object recognition. IJCV 104, 2, 154–171. Google ScholarDigital Library
    47. Wang, D., Chai, X., Zhang, H., Chang, H., Zeng, W., and Shan, S. 2011. A novel coarse-to-fine hair segmentation method. In IEEE Automatic Face Gesture Recognition and Workshops, 233–238.Google Scholar
    48. Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: Styling, simulation, and rendering. TVCG 13, 2, 213–234. Google ScholarDigital Library
    49. Warrell, J., and Prince, S. J. D. 2009. Labelfaces: Parsing facial features by multiclass labeling with an epitome prior. In ICIP, 2481–2484. Google ScholarDigital Library
    50. Weng, Y., Wang, L., Li, X., Chai, M., and Zhou, K. 2013. Hair interpolation for portrait morphing. In Computer Graphics Forum, vol. 32, 79–84.Google ScholarCross Ref
    51. Yuksel, C., Schaefer, S., and Keyser, J. 2009. Hair meshes. ACM Trans. Graph. 28, 5, 166:1–166:7. Google ScholarDigital Library
    52. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., and Torr, P. H. S. 2015. Conditional random fields as recurrent neural networks. In ICCV, 1529–1537. Google ScholarDigital Library

ACM Digital Library Publication: