“A statistical model for synthesis of detailed facial geometry” by Golovinskiy, Matusik, Pfister, Rusinkiewicz and Funkhouser

Detailed surface geometry contributes greatly to the visual realism of 3D face models. However, acquiring high-resolution face geometry is often tedious and expensive. Consequently, most face models used in games, virtual reality, or computer vision look unrealistically smooth. In this paper, we introduce a new statistical technique for the analysis and synthesis of small three-dimensional facial features, such as wrinkles and pores. We acquire high-resolution face geometry for people across a wide range of ages, genders, and races. For each scan, we separate the skin surface details from a smooth base mesh using displaced subdivision surfaces. Then, we analyze the resulting displacement maps using the texture analysis/synthesis framework of Heeger and Bergen, adapted to capture statistics that vary spatially across a face. Finally, we use the extracted statistics to synthesize plausible detail on face meshes of arbitrary subjects. We demonstrate the effectiveness of this method in several applications, including analysis of facial texture in subjects with different ages and genders, interpolation between high-resolution face scans, adding detail to low-resolution face scans, and adjusting the apparent age of faces. In all cases, we are able to re-produce fine geometric details consistent with those observed in high resolution scans.

1. Bando, Y., Kuratate, T., and Nishita, T. 2002. A simple method for modeling wrinkles on human skin. In Pacific Conference on Computer Graphics and Applications, 166–175.]] Google ScholarDigital Library
2. Barsky, S., and Petrou, M. 2001. Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Eighth IEEE International Conference on Computer Vision, vol. 2, 600–605.]]Google Scholar
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of ACM SIGGRAPH 99, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 187–194.]] Google ScholarDigital Library
4. Boissieux, L., Kiss, G., Magnenat-Thalmann, N., and Kalra, P. 2000. Simulation of skin aging and wrinkles with cosmetics insight. In Computer Animation and Simulation 2000, 15–27.]]Google Scholar
5. Brox, T., and Weickert, J. 2006. Level set segmentation with multiple regions. IEEE Transactions on Image Processing.]] Google ScholarDigital Library
6. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH 2001, 145–156.]] Google ScholarDigital Library
7. Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., and Hawkins, T. 2002. A lighting reproduction approach to live-action compositing. ACM Transactions on Graphics (SIGGRAPH 2002) 21, 3 (July), 547–556.]] Google ScholarDigital Library
8. DeCarlo, D., Metaxas, D., and Stone, M. 1998. An anthropometric face model using variational techniques. In Proceedings of ACM SIGGRAPH 98, ACM Press, New York, NY, USA, 67–74.]] Google ScholarDigital Library
9. Dyn, N., Levine, D., and Gregory, J. A. 1990. A butterfly subdivision scheme for surface interpolation with tension control. ACM Trans. Graph. 9, 2, 160–169.]] Google ScholarDigital Library
10. Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV ’99: Proceedings of the International Conference on Computer Vision-Volume 2, IEEE Computer Society, Washington, DC, USA, 1033.]] Google ScholarDigital Library
11. Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2005. Reflectance from images: A model-based approach for human faces. Research Report MPI-I-2005-4-001, Max-Planck-Institut für Informatik, Stuhlsatzenhausweg 85, 66123 Saarbrücken, Germany. Accepted for publication in IEEE TVCG.]]Google Scholar
12. Guskov, I., Vidimče, K., Sweldens, W., and Schröer, P. 2000. Normal meshes. In Proceedings of ACM SIGGRAPH 2000, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 95–102.]] Google ScholarDigital Library
13. Haro, A., Essa, I., and Guenter, B. 2001. Real-time, Photo-Realistic, physically based rendering of fine scale human skin structure. In Rendering Techniques, 53–62.]] Google ScholarDigital Library
14. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of ACM SIGGRAPH 95, ACM Press, New York, NY, USA, 229–238.]] Google ScholarDigital Library
15. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of ACM SIGGRAPH 2001, ACM Press, New York, NY, USA, 327–340.]] Google ScholarDigital Library
16. Igarashi, T., Nishino, K., and Nayar, S. 2005. The appearance of human skin. Tech. Rep. CUCS-024-05, Department of Computer Science, Columbia University, June.]]Google Scholar
17. Larboulette, C., and Cani, M.-P. 2004. Real-time dynamic wrinkles. In Computer Graphics International, IEEE Computer Society Press. Greece.]] Google ScholarDigital Library
18. Lee, A., Moreton, H., and Hoppe, H. 2000. Displaced subdivision surfaces. In Proceedings of ACM SIGGRAPH 2000, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 85–94.]] Google ScholarDigital Library
19. Liu, C., Shum, H.-Y., and Zhang, C. 2001. A two-step approach to hallucinating faces: Global parametric model and local nonparametric model. In CVPR (1), 192–198.]]Google Scholar
20. Loop, C. 1987. Smooth Subdivision Surfaces Based on Triangles. Master’s thesis, University of Utah.]]Google Scholar
21. Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Trans. Graph. 24, 3, 787–794.]] Google ScholarDigital Library
22. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3, 536–543.]] Google ScholarDigital Library
23. Perlin, K. 1985. An image synthesizer. In Computer Graphics (Proceedings of ACM SIGGRAPH 85), ACM Press, New York, NY, USA, 287–296.]] Google ScholarDigital Library
24. Pighin, F., and Lewis, J. 2005. Digital face cloning. In SIGGRAPH 2005 Course Notes.]] Google ScholarDigital Library
25. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40, 1, 49–70.]] Google ScholarDigital Library
26. Rusinkiewicz, S., Burns, M., and DeCarlo, D. 2006. Exaggerated shading for depicting shape and detail. ACM Trans. Graphics (Proc. SIGGRAPH) 25, 3.]] Google ScholarDigital Library
27. Simoncelli, E. P., and Freeman, W. T. 1995. The steerable pyramid: a flexible architecture for multi-scale derivative computation. In ICIP ’95: Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3, IEEE Computer Society, Washington, DC, USA, 3444.]] Google ScholarDigital Library
28. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426–433.]] Google ScholarDigital Library
29. Weyrich, T., Matusik, W., Pfister, H., Ngan, A., and Gross, M. 2005. Measuring skin reflectance and subsurface scattering. Tech. Rep. TR2005–046, Mitsubishi Electric Research Laboratories (MERL).]]Google Scholar
30. Wu, Y., Thalmann, N. M., and Thalmann, D. 1995. A dynamic wrinkle model in facial animation and skin ageing. The Journal of Visualization and Computer Animation 6, 4, 195–206.]]Google ScholarCross Ref
31. Wu, Y., Kalra, P., and Thalmann, N. M. 1997. Physically-based wrinkle simulation & skin rendering. In Computer Animation and Simulation ’97, Eurographics, 69–79.]]Google Scholar