“Making faces” by Guenter, Grimm, Wood, Malvar and Pighin

We have created a system for capturing both the three-dimensional geometry and color and shading information for human facial expressions. We use this data to reconstruct photorealistic, 3D animations of the captured expressions. The system uses a large set of sampling points on the face to accurately track the three dimensional deformations of the face. Simultaneously with the tracking of the geometric data, we capture multiple high resolution, registered video images of the face. These images are used to create a texture map sequence for a three dimensional polygonal face model which can then be rendered on standard 3D graphics hardware. The resulting facial animation is surprisingly life-like and looks very much like the original live performance. Separating the capture of the geometry from the texture images eliminates much of the variance in the image data due to motion, which increases compression ratios. Although the primary emphasis of our work is not compression we have investigated the use of a novel method to compress the geometric data based on principal components analysis. The texture sequence is compressed using an MPEG4 video codec. Animations reconstructed from 512×512 pixel textures look good at data rates as low as 240 Kbits per second.

1. BEIER, T., AND NEELY, S. Feature-based image metamorphosis. In Computer Graphics (SIGGRAPH ’92 Proceedings) (July 1992), E. E. Catmull, Ed., vol. 26, pp. 35-42.
2. BREGLER, C., COVELL, M., AND SLANEY, M. Video rewrite: Driving visual speech with audio. Computer Graphics 31, 2 (Aug. 1997), 353-361.
3. CASSELL, J., PELACHAUD, C., BADLER, N., STEEDMAN, M., ACHORN, B., BECKET, T., DOUVILLE, B., PREVOST, S., AND STONE, M. Animated conversation: Rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. Computer Graphics 28, 2 (Aug. 1994), 413-420.
4. DECARLO, D., AND METAXAS, D. The integration of optical flow and deformable models with applications to human face shape and motion estimation. Proceedings CVPR (1996), 231-238.
5. ESSA, I., AND PENTLAND, A. Coding, analysis, interpretation and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 7 (1997), 757-763.
6. FAUGERAS, O. Three-dimensional computer vision. MIT Press, Cambridge, MA, 1993.
7. FISCHLER, M. A., AND BOOLES, R. C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24, 6 (Aug. 1981), 381-395.
8. HOPPE, H. Progressive meshes. In SIGGRAPH 96 Conference Proceedings (Aug. 1996), H. Rushmeier, Ed., Annual Conference Series, ACM SIGGRAPH, Addison Wesley, pp. 99-108. held in New Orleans, Louisiana, 04-09 August 1996.
9. HORN, B. K. P. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America 4, 4 (Apr. 1987).
10. LEE, Y., TERZOPOULOS, D., AND WATERS, K. Realistic modeling for facial animation. Computer Graphics 29, 2 (July 1995), 55-62.
11. PIGHIN, F., AUSLANDER, J., LISHINSKI, D., SZELISKI, R., AND SALESIN, D. Realistic facial animation using image based 3d morphing. Tech. Report TR-97-01-03, Department of Computer Science and Engineering, University of Washington, Seattle, Wa, 1997.
12. SCHURMANN, J. Pattern Classification: A Unified View of Statistical and Neural Approaches. John Wiley and Sons, Inc., New York, 1996.
13. STRANG. Linear Algebra and its Application. HBJ, 1988.
14. WATERS, K. A muscle model for animating threedimensional facial expression. In Computer Graphics (SIG- GRAPH ’87 Proceedings) (July 1987), M. C. Stone, Ed., vol. 21, pp. 17-24.
15. WILLIAMS, L. Performance-driven facial animation. Computer Graphics 24, 2 (Aug. 1990), 235-242.