“Analysis of human faces using a measurement-based skin reflectance model” by Weyrich, Matusik, Pfister, Bickel, Donner, et al. …

We have measured 3D face geometry, skin reflectance, and subsurface scattering using custom-built devices for 149 subjects of varying age, gender, and race. We developed a novel skin reflectance model whose parameters can be estimated from measurements. The model decomposes the large amount of measured skin data into a spatially-varying analytic BRDF, a diffuse albedo map, and diffuse subsurface scattering. Our model is intuitive, physically plausible, and — since we do not use the original measured data — easy to edit as well. High-quality renderings come close to reproducing real photographs. The analysis of the model parameters for our sample population reveals variations according to subject age, gender, skin type, and external factors (e.g., sweat, cold, or makeup). Using our statistics, a user can edit the overall appearance of a face (e.g., changing skin type and age) or change small-scale features using texture synthesis (e.g., adding moles and freckles). We are making the collected statistics publicly available to the research community for applications in face synthesis and analysis.

1. Angelopoulou, E., Molana, R., and Daniilidis, K. 2001. Multispectral skin color modeling. In IEEE Conf. on Comp. Vision and Pattern Rec. (CVPR), 635–642.Google Scholar
2. Barsky, S., and Petrou, M. 2001. Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Eighth IEEE International Conference on Computer Vision, vol. 2, 600–605.Google Scholar
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. Computer Graphics 33, Annual Conference Series, 187–194. Google ScholarDigital Library
4. Blinn, J. 1977. Models of light reflection for computer synthesized pictures. Computer Graphics 11, Annual Conference Series, 192–198. Google ScholarDigital Library
5. Borshukov, G., and Lewis, J. 2003. Realistic human face rendering for the matix reloaded. In ACM SIGGRAPH 2003 Conference Abstracts and Applications (Sketch). Google ScholarDigital Library
6. Cula, O., and Dana, K. 2002. Image-based skin analysis. In Texture 2002, The Second International Workshop on Texture Analysis and Synthesis, 35–42.Google Scholar
7. Cula, O., Dana, K., Murphy, F., and Rao, B. 2004. Bidirectional imaging and modeling of skin texture. IEEE Transactions on Biomedical Engineering 51, 12 (Dec.), 2148–2159.Google ScholarCross Ref
8. Cula, O., Dana, K., Murphy, F., and Rao, B. 2005. Skin texture modeling. International Journal of Computer Vision 62, 1–2 (April–May), 97–119. Google ScholarDigital Library
9. Dana, K. J., van Ginneken, B., Nayar, S. K., and Koen-Derink, J. J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 1, 18, 1–34. Google ScholarDigital Library
10. Debevec, P., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Computer Graphics, SIGGRAPH 97 Proceedings, 369–378. Google ScholarDigital Library
11. Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Computer Graphics, SIGGRAPH 96 Proceedings, 11–20. Google ScholarDigital Library
12. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Computer Graphics, SIGGRAPH 2000 Proceedings, 145–156. Google ScholarDigital Library
13. Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., and Hawkins, T. 2002. A lighting reproduction approach to live-action compositing. ACM Transactions on Graphics (SIGGRAPH 2002) 21, 3 (July), 547–556. Google ScholarDigital Library
14. Donner, C., and Jensen, H. W. 2005. Light diffusion in multi-layered translucent materials. ACM Transactions on Graphics 24, 3 (July), 1032–1039. Google ScholarDigital Library
15. Fitzpatrick, T. 1988. The validity and practicality of sun-reactive skin types i through vi. Arch. Dermatology 124, 6 (June), 869–871.Google ScholarCross Ref
16. Fuchs, C., Goesele, M., Chen, T., and Seidel, H.-P. 2005. An empirical model for heterogeneous translucent objects. Tech. Rep. MPI-I-2005-4-006, Max-Planck Institute für Informatik.Google Scholar
17. Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2005. Reflectance from images: A model-based approach for human faces. Research Report MPI-I-2005-4-001, Max-Planck-Institut für Informatik, Stuhlsatzenhausweg 85, 66123 Saarbrücken, Germany. Accepted for publication in IEEE TVCG.Google Scholar
18. Georghiades, A., Belhumeur, P., and Kriegman, D. 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47–54. Google ScholarDigital Library
19. Georghiades, A. 2003. Recovering 3-D shape and reflectance from a small number of photographs. In Rendering Techniques, 230–240. Google ScholarDigital Library
20. Goesele, M., Lensch, H., Lang, J., Fuchs, C., and Seidel, H. 2004. Disco — acquisition of translucent objects. ACM Transactions on Graphics 24, 3, 835–844. Google ScholarDigital Library
21. Green, P., Halbert, M., and Robinson, P. 1966. Canonical analysis: An exposition and illustrative application. Marketing Research 3 (Feb.), 32–39.Google ScholarCross Ref
22. Hanrahan, P., and Krueger, W. 1993. Reflection from layered surfaces due to subsurface scattering. In Computer Graphics, SIGGRAPH 93 Proceedings, 165–174. Google ScholarDigital Library
23. Hawkins, T., Wenger, A., Tchou, C., Goransson, F., and Debevec, P. 2004. Animatable facial reflectance fields. In Rendering Techniques ’04 (Proceedings of the Second Eurographics Symposium on Rendering). Google ScholarDigital Library
24. Heeger, D., and Bergen, J. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 229–238. Google ScholarDigital Library
25. Hery, C., 2005. Face cloning at ILM. Digital Face Cloning, Siggraph Course Notes.Google Scholar
26. Igarashi, T., Nishino, K., and Nayar, S. 2005. The appearance of human skin. Tech. Rep. CUCS-024-05, Department of Computer Science, Columbia University, June.Google Scholar
27. Jensen, H. W., and Buhler, J. 2002. A rapid hierarchical rendering technique for translucent materials. In Computer Graphics, SIGGRAPH 2002 Proceedings, 576–581. Google ScholarDigital Library
28. Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. 2001. A practical model for subsurface light transport. In Computer Graphics, SIGGRAPH 2001 Proceedings, 511–518. Google ScholarDigital Library
29. Jones, A., Gardner, A., Bolas, M., McDowall, I., and Debevec, P. 2005. Performance geometry capture for spatially varying relighting. In ACM SIGGRAPH 2005 Conference Abstracts and Applications (Sketch). Google ScholarDigital Library
30. Koenderink, J., and Pont, S. 2003. The secret of velvety skin. Machine Vision and Application, 14, 260–268. Special Issue on Human Modeling, Analysis, and Synthesis. Google ScholarDigital Library
31. Krishnaswamy, A., and Baranoski, G. 2004. A biophysically-based spectral model of light interaction with human skin. Computer Graphics Forum 23, 3 (Sept.), 331–340.Google ScholarCross Ref
32. Lafortune, E., Foo, S.-C., Torrance, K., and Greenberg, D. 1997. Non-linear approximation of reflectance functions. Computer Graphics 31, Annual Conference Series, 117–126. Google ScholarDigital Library
33. Larsson, M., Nilsson, H., and Strömberg, T. 2003. In vivo determination of local skin optical properties and photon path length by use of spatially resolved diffuse reflectance with applications in laser doppler flowmetry. Applied Optics 42, 1 (Jan.), 124–134.Google ScholarCross Ref
34. Lawrence, J., Rusinkiewicz, S., and Ramamoorthi, R. 2004. Efficient brdf importance sampling using a factored representation. ACM Transactions on Graphics 23, 3, 496–505. Google ScholarDigital Library
35. Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2001. Image-based reconstruction of spatially varying materials. In Proceedings of the 12th Eurographics Workshop on Rendering, 104–115. Google ScholarDigital Library
36. Loop, C. 1987. Smooth Subdivision Surfaces based on Triangles. Master’s thesis, Department of Mathematics, University of Utah.Google Scholar
37. Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-based BRDF measurement including human skin. In Proceedings of the 10th Eurographics Workshop on Rendering, 139–152. Google ScholarDigital Library
38. Marschner, S., Guneter, B., and Raghupathy, S. 2000. Modeling and rendering for realistic facial animation. In 11th Eurographics Workshop on Rendering, 231–242. Google ScholarDigital Library
39. Marschner, S., Westin, S., Lafortune, E., and Torrance, K. 2000. Image-based measurement of the Bidirectional Reflectance Distribution Function. Applied Optics 39, 16 (June), 2592–2600.Google ScholarCross Ref
40. Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics 24, 3, 787–794. Google ScholarDigital Library
41. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3, 536–543. Google ScholarDigital Library
42. Ng, C., and Li, L. 2001. A multi-layered reflection model of natural human skin. In Computer Graphics International, 249256. Google ScholarDigital Library
43. Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of brdf models. In Eurographics Symposium on Rendering, 117–126. Google ScholarDigital Library
44. Nickell, S., Hermann, M., Essenpreis, M., Farrell, T. J., Kramer, U., and Patterson, M. S. 2000. Anisotropy of light propagation in human skin. Phys. Med. Biol. 45, 2873–2886.Google ScholarCross Ref
45. Nicodemus, F., Richmond, J., Hsia, J., Ginsberg, I., and Limperis, T. 1977. Geometric considerations and nomenclature for reflectance. Monograph 160, National Bureau of Standards (US), October.Google Scholar
46. Paris, S., Sillion, F., and Quan, L. 2003. Lightweight face relighting. In Proceedings of Pacific Graphics, 41–50. Google ScholarDigital Library
47. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. 1998. Synthesizing realistic facial expressions from photographs. In Computer Graphics, vol. 32 of SIGGRAPH 98 Proceedings, 75–84. Google ScholarDigital Library
48. Sander, P., Gosselin, D., and Mitchell, J. 2004. Real-time skin rendering on graphics hardware. SIGGRAPH 2004 Sketch. Google ScholarDigital Library
49. Sheffer, A., Levy, B., Mogilnitsky, M., and Bogomyakov, A. 2005. ABF++: Fast and robust angle based flattening. ACM Transactions on Graphics 24, 2, 311–330. Google ScholarDigital Library
50. Stam, J. 2001. An illumination model for a skin layer bounded by rough surfaces. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques, Springer, Wien, Vienna, Austria, 39–52. Google ScholarDigital Library
51. Tong, X., Wang, J., Lin, S., Guo, B., and Shum, H.-Y. 2005. Modeling and rendering of quasi-homogeneous materials. ACM Transactions on Graphics 24, 3, 1054–1061. Google ScholarDigital Library
52. Torrance, K. E., and Sparrow, E. M. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America 57, 1104–1114.Google ScholarCross Ref
53. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Transactions on Graphics 22, 3, 770–779. Google ScholarDigital Library
54. Tuchin, V. 2000. Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. SPIE Press.Google Scholar
55. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3, 756–764. Google ScholarDigital Library
56. Weyrich, T., Matusik, W., Pfister, H., Ngan, A., and Gross, M. 2005. Measuring skin reflectance and subsurface scattering. Tech. Rep. TR2005-046, Mitsubishi Electric Research Laboratories (MERL).Google Scholar
57. Williams, L. 2005. Case study: The gemini man. In SIGGRAPH 2005 Course ‘Digital Face Cloning’.Google Scholar