“Style transfer for headshot portraits” by Shih, Paris, Barnes, Freeman and Durand
Conference:
Type:
Title:
- Style transfer for headshot portraits
Session/Category Title: Changing Your Perception
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Headshot portraits are a popular subject in photography but to achieve a compelling visual style requires advanced skills that a casual photographer will not have. Further, algorithms that automate or assist the stylization of generic photographs do not perform well on headshots due to the feature-specific, local retouching that a professional photographer typically applies to generate such portraits. We introduce a technique to transfer the style of an example headshot photo onto a new one. This can allow one to easily reproduce the look of renowned artists. At the core of our approach is a new multiscale technique to robustly transfer the local statistics of an example portrait onto a new one. This technique matches properties such as the local contrast and the overall lighting direction while being tolerant to the unavoidable differences between the faces of two different people. Additionally, because artists sometimes produce entire headshot collections in a common style, we show how to automatically find a good example to use as a reference for a given portrait, enabling style transfer without the user having to search for a suitable example for each input. We demonstrate our approach on data taken in a controlled environment as well as on a large set of photos downloaded from the Internet. We show that we can successfully handle styles by a variety of different artists.
References:
1. Ahonen, T., Hadid, A., and Pietikainen, M. 2006. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence. Google ScholarDigital Library
2. An, X., and Pellacini, F. 2010. User-controllable color transfer. In Computer Graphics Forum, vol. 29, 263–271.Google ScholarCross Ref
3. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. In ACM Trans. Graphics. Google ScholarDigital Library
4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graphics. Google ScholarDigital Library
5. Beier, T., and Neely, S. 1992. Feature-based image metamorphosis. In ACM Trans. Graphics, vol. 26. Google ScholarDigital Library
6. Brand, M., and Pletscher, P. 2008. A conditional random field for automatic photo editing. In IEEE Conf. Computer Vision and Pattern Recognition.Google Scholar
7. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In European Conference on Computer Vision. 25–36.Google Scholar
8. Burt, P., and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications 31, 4, 532–540.Google ScholarCross Ref
9. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graphics 25. Google ScholarDigital Library
10. Daugman, J. G. 1993. High confidence visual recognition of persons by a test of statistical independence. IEEE Trans. Pattern Analysis and Machine Intelligence 15, 11, 1148–1161. Google ScholarDigital Library
11. Guo, D., and Sim, T. 2009. Digital face makeup by example. In IEEE Conf. Computer Vision and Pattern Recognition.Google Scholar
12. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. In ACM Trans. Graphics, vol. 30. Google ScholarDigital Library
13. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In ACM Trans. Graphics, ACM, 229–238. Google ScholarDigital Library
14. Joshi, N., Matusik, W., Adelson, E. H., and Kriegman, D. J. 2010. Personal photo enhancement using example images. ACM Transaction on Graphics (TOG) 29, 2, 1–15. Google ScholarDigital Library
15. Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. Pattern Analysis and Machine Intelligence 30, 2, 228–242. Google ScholarDigital Library
16. Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2008. Data-driven enhancement of facial attractiveness. In ACM Transactions on Graphics (TOG), vol. 27, ACM, 38. Google ScholarDigital Library
17. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graphics 24. Google ScholarDigital Library
18. Liu, C., Shum, H.-Y., and Freeman, W. T. 2007. Face hallucination: Theory and practice. International Journal of Computer Vision 75, 1, 115–134. Google ScholarDigital Library
19. Liu, C., Yuen, J., and Torralba, A. 2011. Sift flow: Dense correspondence across scenes and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 5. Google ScholarDigital Library
20. Malik, J., and Perona, P. 1990. Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A 7.Google ScholarCross Ref
21. Mohammed, U., Prince, S. J., and Kautz, J. 2009. Visiolization: generating novel facial images. In ACM Transactions on Graphics (TOG), vol. 28, 57. Google ScholarDigital Library
22. Nars, F. 2004. Makeup your mind. PowerHouse Books.Google Scholar
23. Oliva, A., Torralba, A., and Schyns, P. G. 2006. Hybrid images. In ACM Transactions on Graphics (TOG), vol. 25, 527–532. Google ScholarDigital Library
24. Peers, P., Tamura, N., Matusik, W., and Debevec, P. 2007. Post-production facial performance relighting using reflectance transfer. ACM Transactions on Graphics (TOG) 26, 3, 52. Google ScholarDigital Library
25. Phillips, N. 2004. Lighting techniques for low key portrait photography. Amherst Media, 12–16.Google Scholar
26. Pitié, F., Kokaram, A. C., and Dahyot, R. 2005. N-dimensional probability density function transfer and its application to color transfer. In IEEE Conference on Computer Vision. Google ScholarDigital Library
27. Reinhard, E., Adhikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications 21, 5, 34–41. Google ScholarDigital Library
28. Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. In ACM Transactions on Graphics (TOG), vol. 23, 309–314. Google ScholarDigital Library
29. Saragih, J. M., Lucey, S., and Cohn, J. F. 2009. Face alignment through subspace constrained mean-shifts. In IEEE Conference on Computer Vision, 1034–1041.Google Scholar
30. Shih, Y., Paris, S., Durand, F., and Freeman, W. T. 2013. Data-driven hallucination for different times of day from a single outdoor photo. ACM Transactions on Graphics (Proc. SIGGRAPH Asia). Google ScholarDigital Library
31. Su, S., Durand, F., and Agrawala, M. 2005. De-emphasis of distracting image regions using texture power maps. In Proc. of ICCV Workshop on Texture Analysis and Synthesis.Google Scholar
32. Sunkavalli, K., Johnson, M. K., Matusik, W., and Pfister, H. 2010. Multi-scale image harmonization. ACM Trans. Graphics 29, 4, 125. Google ScholarDigital Library
33. Tai, Y.-W., Jia, J., and Tang, C.-K. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1. Google ScholarDigital Library
34. Tong, W.-S., Tang, C.-K., Brown, M. S., and Xu, Y.-Q. 2007. Example-based cosmetic transfer. In IEEE Pacific Graphics. Google ScholarDigital Library
35. Tuzel, O., Porikli, F., and Meer, P. 2006. Region covariance: A fast descriptor for detection and classification. In European Conference on Computer Vision. 589–600. Google ScholarDigital Library
36. Wang, B., Yu, Y., Wong, T., Chen, C., and Xu, Y. 2010. Data-driven image color theme enhancement. In ACM Transactions on Graphics, vol. 29, 146. Google ScholarDigital Library
37. Wang, B., Yu, Y., and Xu, Y.-Q. 2011. Example-based image color and tone style enhancement. ACM Transactions on Graphics 30, 4. Google ScholarDigital Library
38. Wen, Z., Liu, Z., and Huang, T. S. 2003. Face relighting with radiance environment maps. In IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, II-158.Google Scholar
39. Zhang, L., Wang, S., and Samaras, D. 2005. Face synthesis and recognition from a single image under arbitrary unknown lighting using a spherical harmonic basis morphable model. In IEEE Conf. Computer Vision and Pattern Recognition. Google ScholarDigital Library
