“Data-driven enhancement of facial attractiveness” by Leyvand, Cohen-Or, Dror and Lischinski

When human raters are presented with a collection of shapes and asked to rank them according to their aesthetic appeal, the results often indicate that there is a statistical consensus among the raters. Yet it might be difficult to define a succinct set of rules that capture the aesthetic preferences of the raters. In this work, we explore a data-driven approach to aesthetic enhancement of such shapes. Specifically, we focus on the challenging problem of enhancing the aesthetic appeal (or the attractiveness) of human faces in frontal photographs (portraits), while maintaining close similarity with the original.The key component in our approach is an automatic facial attractiveness engine trained on datasets of faces with accompanying facial attractiveness ratings collected from groups of human raters. Given a new face, we extract a set of distances between a variety of facial feature locations, which define a point in a high-dimensional “face space”. We then search the face space for a nearby point with a higher predicted attractiveness rating. Once such a point is found, the corresponding facial distances are embedded in the plane and serve as a target to define a 2D warp field which maps the original facial features to their adjusted locations. The effectiveness of our technique was experimentally validated by independent rating experiments, which indicate that it is indeed capable of increasing the facial attractiveness of most portraits that we have experimented with.

1. Alley, T. R., and Cunningham, M. R. 1991. Averaged faces are attractive, but very attractive faces are not average. Psychological Science 2, 123–125.Google ScholarCross Ref
2. Beier, T., and Neely, S. 1992. Feature-based image metamorphosis. In Proc. ACM SIGGRAPH 92, ACM Press, 35–42. Google ScholarDigital Library
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. ACM SIGGRAPH 99, ACM Press, 187–194. Google ScholarDigital Library
4. Blanz, V., 2003. Manipulation of facial attractiveness. {web page} http://www.mpi-inf.mpg.de/~blanz/data/attractiveness/.Google Scholar
5. Bradski, G. 2000. The OpenCV library. Dr. Dobbs Journal 25, 11 (Nov.), 120, 122–125.Google Scholar
6. Cohen, J. D. 1997. Drawing graphs to convey proximity: an incremental arrangement method. ACM Trans. Comput.-Hum. Interact. 4, 3, 197–229. Google ScholarDigital Library
7. Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. 1995. Active shape models — their training and their applications. Comput. Vis. Image Underst. 61, 1 (Jan.), 38–59. Google ScholarDigital Library
8. Cunningham, M. R., Roberts, A. R., Wu, C.-H., Barbee, A. P., and Druen, P. B. 1995. Their ideas of beauty are, on the whole, the same as ours: Consistency and variability in the cross-cultural perception of female attractiveness. Journal of Personality and Social Psychology 68, 1, 261–279.Google ScholarCross Ref
9. Eisenthal, Y., Dror, G., and Ruppin, E. 2006. Facial attractiveness: Beauty and the machine. Neural Computation 18, 1, 119–142. Google ScholarDigital Library
10. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proc. ACM SIGGRAPH 98, ACM Press, 55–66. Google ScholarDigital Library
11. Joachims, T. 1999. Making large-scale SVM learning practical. In Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, USA, 169–184. Google ScholarDigital Library
12. Jones, D. 1996. Physical Attractiveness and the Theory of Sexual Selection: Results from Five Populations. University of Michigan Press, Ann Arbor, Michigan.Google Scholar
13. Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., and Ruppin, E. 2007. A humanlike predictor of facial attractiveness. In Advances in Neural Information Processing Systems 19, B. Schölkopf, J. Platt, and T. Hoffman, Eds. MIT Press.Google Scholar
14. Langlois, J. H., Roggman, L. A., Casey, R. J., M. Ritter, J., Rieser-Danner, L. A., and Jenkins, V. Y. 1987. Infant preferences for attractive faces: Rudiments of a stereotype? Developmental Psychology 23, 5, 363–369.Google ScholarCross Ref
15. Lanitis, A., Taylor, C. J., and Cootes, T. F. 2002. Toward automatic simulation of aging effects on face images. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4, 442–455. Google ScholarDigital Library
16. Lee, Y., Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. In Proc. ACM SIGGRAPH 95, ACM Press, 55–62. Google ScholarDigital Library
17. Lee, S.-Y., Wolberg, G., Chwa, K.-Y., and Shin, S. Y. 1996. Image metamorphosis with scattered feature constraints. IEEE Trans. Visualization and Computer Graphics 2, 4 (Dec.). Google ScholarDigital Library
18. Lee, S.-Y., Wolberg, G., and Shin, S. Y. 1997. Scattered data interpolation with multilevel B-splines. IEEE Trans. Visualization and Computer Graphics 3, 3 (July–Sept.). Google ScholarDigital Library
19. Levenberg, K. 1944. A method for the solution of certain problems in least squares. Quart. Appl. Math., 2, 164–168.Google ScholarCross Ref
20. Lourakis, M., 2004. levmar: Levenberg-Marquardt non-linear least squares algorithms in C/C++. {web page} http://www.ics.forth.gr/~lourakis/levmar/.Google Scholar
21. Marquardt, D. 1963. An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math., 11, 431–441.Google ScholarCross Ref
22. Martinez, A. M., and Benavente, R. 1998. The AR face database. CVC Technical Report 24.Google Scholar
23. O’Toole, A. J., Price, T., Vetter, T., Bartlett, J. C., and Blanz, V. 1999. 3D shape and 2D surface textures of human faces: the role of “averages” in attractiveness and age. Image Vision Comput. 18, 1, 9–19.Google ScholarCross Ref
24. Parke, F. I., and Waters, K. 1996. Computer Facial Animation. A K Peters, Wellesley, Massachusetts. Google ScholarDigital Library
25. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313–318. Google ScholarDigital Library
26. Perrett, D. I., May, K. A., and Yoshikawa, S. 1994. Facial shape and judgements of female attractiveness. Nature 368, 239–242.Google ScholarCross Ref
27. Perrett, D. I., Burt, D. M., Penton-Voak, I. S., Lee, K. J., Rowland, D. A., and Edwards, R. 1999. Symmetry and human facial attractiveness. Evolution and Human Behavior 20, 295–307.Google ScholarCross Ref
28. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proc. ACM SIGGRAPH 98, 75–84. Google ScholarDigital Library
29. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. 1992. Numerical Recipes: The Art of Scientific Computing, 2nd ed. Cambridge University Press. Google ScholarDigital Library
30. Sammon, J. W. 1969. A nonlinear mapping for data structure analysis. IEEE Transactions on Computers C-18, 5, 401–409. Google ScholarDigital Library
31. Slater, A., Der Schulenberg, C. V., Brown, E., Butter-worth, G., Badenoch, M., and Parsons, S. 1998. Newborn infants prefer attractive faces. Infant Behavior and Development, 345–354.Google Scholar
32. Vapnik, V. 1995. The nature of statistical learning theory. Springer, New York. Google ScholarDigital Library
33. Yang, M.-H., Kriegman, D., and Ahuja, N. 2002. Detecting faces in images: A survey. IEEE Trans. PAMI 24, 1, 34–58. Google ScholarDigital Library
34. Zhao, W., Chellappa, R., Phillips, P. J., and Rosenfeld, A. 2003. Face recognition: A literature survey. ACM Comput. Surv. 35, 4, 399–458. Google ScholarDigital Library
35. Zhou, Y., Gu, L., and Zhang, H.-J. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proc. CVPR 2003, 109–118. Google ScholarDigital Library