“AutoHair: fully automatic hair modeling from a single image”
Conference:
Type:
Title:
- AutoHair: fully automatic hair modeling from a single image
Session/Category Title: CAPTURING HUMANS
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We introduce AutoHair, the first fully automatic method for 3D hair modeling from a single portrait image, with no user interaction or parameter tuning. Our method efficiently generates complete and high-quality hair geometries, which are comparable to those generated by the state-of-the-art methods, where user interaction is required. The core components of our method are: a novel hierarchical deep neural network for automatic hair segmentation and hair growth direction estimation, trained over an annotated hair image database; and an efficient and automatic data-driven hair matching and modeling algorithm, based on a large set of 3D hair exemplars. We demonstrate the efficacy and robustness of our method on Internet photos, resulting in a database of around 50K 3D hair models and a corresponding hairstyle space that covers a wide variety of real-world hairstyles. We also show novel applications enabled by our method, including 3D hairstyle space navigation and hair-aware image retrieval.
References:
1. Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In CVPR, 1–8.Google Scholar
2. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH ’99, 187–194. Google ScholarDigital Library
3. Bookstein, F. L. 1989. Principal warps: thin-plate splines and the decomposition of deformations. Trans. PAMI 11, 6, 567–585. Google ScholarDigital Library
4. Cao, X., Wei, Y., Wen, F., and Sun, J. 2012. Face alignment by explicit shape regression. In CVPR, 2887–2894. Google ScholarDigital Library
5. Chai, M., Wang, L., Weng, Y., Yu, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1–116:8. Google ScholarDigital Library
6. Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4, 75:1–75:8. Google ScholarDigital Library
7. Chai, M., Luo, L., Sunkavalli, K., Carr, N., Hadap, S., and Zhou, K. 2015. High-quality hair modeling from a single portrait photo. ACM Trans. Graph. 34, 6, 204:1–204:10. Google ScholarDigital Library
8. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. 2015. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR.Google Scholar
9. Cheng, M.-M., Zheng, S., Lin, W.-Y., Vineet, V., Sturgess, P., Crook, N., Mitra, N. J., and Torr, P. 2014. Imagespirit: Verbal guided image parsing. ACM Trans. Graph. 34, 1, 3:1–3:11. Google ScholarDigital Library
10. Dai, J., He, K., and Sun, J. 2015. BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In ICCV, 1635–1643. Google ScholarDigital Library
11. Desbrun, M., Meyer, M., Schröder, P., and Barr, A. H. 1999. Implicit fairing of irregular meshes using diffusion and curvature flow. In Proc. SIGGRAPH ’99, 317–324. Google ScholarDigital Library
12. Echevarria, J. I., Bradley, D., Gutierrez, D., and Beeler, T. 2014. Capturing and stylizing hair for 3D fabrication. ACM Trans. Graph. 33, 4, 125:1–125:11. Google ScholarDigital Library
13. Electronic Arts, 2014. The Sims Resource. http://www.thesimsresource.com/.Google Scholar
14. Forney Jr, G. D. 1973. The Viterbi algorithm. Proceedings of the IEEE 61, 3, 268–278.Google ScholarCross Ref
15. Gephi, 2016. The Open Graph Viz Platform. https://gephi.org.Google Scholar
16. Girshick, R., Donahue, J., Darrell, T., and Malik, J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 580–587. Google ScholarDigital Library
17. Herrera, T. L., Zinke, A., and Weber, A. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Trans. Graph. 31, 6, 146:1–146:9. Google ScholarDigital Library
18. Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph. 33, 4, 126:1–126:10. Google ScholarDigital Library
19. Hu, L., Ma, C., Luo, L., Wei, L.-Y., and Li, H. 2014. Capturing braided hairstyles. ACM Trans. Graph. 33, 6, 225:1–225:9. Google ScholarDigital Library
20. Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4, 125:1–125:9. Google ScholarDigital Library
21. Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.-Y., Teng, S.-H., Bao, H., Guo, B., and Shum, H.-Y. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3, 1126–1134. Google ScholarDigital Library
22. Huang, S.-S., Shamir, A., Shen, C.-H., Zhang, H., Sheffer, A., Hu, S.-M., and Cohen-Or, D. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Trans. Graph. 32, 4, 71:1–71:10. Google ScholarDigital Library
23. Hueting, M., Ovsjanikov, M., and Mitra, N. J. 2015. CrossLink: Joint understanding of image and 3D model collections through shape and camera pose variations. ACM Trans. Graph. 34, 6, 233:1–233:13. Google ScholarDigital Library
24. Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1–164:9. Google ScholarDigital Library
25. Jia, Y., 2013. Caffe: An open source convolutional architecture for fast feature embedding.Google Scholar
26. Karypis, G., and Kumar, V. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing 20, 1, 359–392. Google ScholarDigital Library
27. Kim, V. G., Li, W., Mitra, N. J., DiVerdi, S., and Funkhouser, T. 2012. Exploring collections of 3D models using fuzzy correspondences. ACM Trans. Graph. 31, 4, 54:1–54:11. Google ScholarDigital Library
28. Krähenbühl, P., and Koltun, V. 2011. Efficient inference in fully connected CRFs with Gaussian edge potentials. In NIPS, 109–117.Google Scholar
29. Li, Y., Su, H., Qi, C. R., Fish, N., Cohen-Or, D., and Guibas, L. J. 2015. Joint embeddings of shapes and images via CNN image purification. ACM Trans. Graph. 34, 6, 234:1–234:12. Google ScholarDigital Library
30. Ling, H., and Okada, K. 2007. An efficient earth mover’s distance algorithm for robust histogram comparison. Trans. PAMI 29, 5, 840–853. Google ScholarDigital Library
31. Liu, J., Sun, J., and Shum, H.-Y. 2009. Paint selection. ACM Trans. Graph. 28, 3, 69:1–69:7. Google ScholarDigital Library
32. Liu, Z., Li, X., Luo, P., Loy, C. C., and Tang, X. 2015. Semantic image segmentation via deep parsing network. In ICCV, 1377–1385. Google ScholarDigital Library
33. Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 2, 129–137. Google ScholarDigital Library
34. Long, J., Shelhamer, E., and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR, 3431–3440.Google Scholar
35. Luo, P., Wang, X., and Tang, X. 2012. Hierarchical face parsing via deep learning. In CVPR, 2480–2487. Google ScholarDigital Library
36. Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4, 76:1–76:12. Google ScholarDigital Library
37. O’Donovan, P., Lībeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory font selection using crowdsourced attributes. ACM Trans. Graph. 33, 4, 92:1–92:9. Google ScholarDigital Library
38. Ovsjanikov, M., Li, W., Guibas, L., and Mitra, N. J. 2011. Exploration of continuous variability in collections of 3D shapes. ACM Trans. Graph. 30, 4, 33:1–33:10. Google ScholarDigital Library
39. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1–30:9. Google ScholarDigital Library
40. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. In ICCV, 1–8.Google Scholar
41. Ren, S., He, K., Girshick, R., and Sun, J. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 91–99. Google ScholarDigital Library
42. Selle, A., Lentine, M., and Fedkiw, R. 2008. A mass spring model for hair simulation. ACM Trans. Graph. 27, 3, 64:1–64:11. Google ScholarDigital Library
43. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 1–15. Google ScholarDigital Library
44. Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
45. Smith, B. M., Zhang, L., Brandt, J., Lin, Z., and Yang, J. 2013. Exemplar-based face parsing. In CVPR, 3484–3491. Google ScholarDigital Library
46. Uijlings, J. R., van de Sande, K. E., Gevers, T., and Smeulders, A. W. 2013. Selective search for object recognition. IJCV 104, 2, 154–171. Google ScholarDigital Library
47. Wang, D., Chai, X., Zhang, H., Chang, H., Zeng, W., and Shan, S. 2011. A novel coarse-to-fine hair segmentation method. In IEEE Automatic Face Gesture Recognition and Workshops, 233–238.Google Scholar
48. Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: Styling, simulation, and rendering. TVCG 13, 2, 213–234. Google ScholarDigital Library
49. Warrell, J., and Prince, S. J. D. 2009. Labelfaces: Parsing facial features by multiclass labeling with an epitome prior. In ICIP, 2481–2484. Google ScholarDigital Library
50. Weng, Y., Wang, L., Li, X., Chai, M., and Zhou, K. 2013. Hair interpolation for portrait morphing. In Computer Graphics Forum, vol. 32, 79–84.Google ScholarCross Ref
51. Yuksel, C., Schaefer, S., and Keyser, J. 2009. Hair meshes. ACM Trans. Graph. 28, 5, 166:1–166:7. Google ScholarDigital Library
52. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., and Torr, P. H. S. 2015. Conditional random fields as recurrent neural networks. In ICCV, 1529–1537. Google ScholarDigital Library
