“Driving high-resolution facial blendshapes with video performance capture” by Fyffe, Jones, Alexander, Ichikari, Graham, et al. …

  • ©Graham Fyffe, Andrew C. Jones, Oleg Alexander, Ryosuke Ichikari, Paul Graham, Koki Nagano, Jay Busch, and Paul E. Debevec




    Driving high-resolution facial blendshapes with video performance capture



    We present a technique for creating realistic facial animation from a set of high-resolution static scans of an actor’s face driven by passive video of the actor from one or more viewpoints. We capture high-resolution static geometry using multi-view stereo and gradient-based photometric stereo [Ghosh et al. 2011]. The scan set includes around 30 expressions largely inspired by the Facial Action Coding System (FACS). Examples of the input scan geometry can be seen in Figure 1 (a). The base topology is defined by an artist for the neutral scan of each subject. The dynamic performance can be shot under existing environmental illumination using one or more off-the shelf HD video cameras.


    1. Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. 2011. Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA ’11, 129:1–129:10.
    2. Werlberger, M. 2012. Convex Approaches for High Performance Video Processing. PhD thesis, Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria.

ACM Digital Library Publication:

Overview Page: