“Driving high-resolution facial blendshapes with video performance capture” by Fyffe, Jones, Alexander, Ichikari, Graham, et al. …

  • ©Graham Fyffe, Andrew C. Jones, Oleg Alexander, Ryosuke Ichikari, Paul Graham, Koki Nagano, Jay Busch, and Paul E. Debevec

Conference:


Type(s):


Title:

    Driving high-resolution facial blendshapes with video performance capture

Presenter(s)/Author(s):



Abstract:


    We present a technique for creating realistic facial animation from a set of high-resolution static scans of an actor’s face driven by passive video of the actor from one or more viewpoints. We capture high-resolution static geometry using multi-view stereo and gradient-based photometric stereo [Ghosh et al. 2011]. The scan set includes around 30 expressions largely inspired by the Facial Action Coding System (FACS). Examples of the input scan geometry can be seen in Figure 1 (a). The base topology is defined by an artist for the neutral scan of each subject. The dynamic performance can be shot under existing environmental illumination using one or more off-the shelf HD video cameras.

References:


    1. Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., and Debevec, P. 2011. Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA ’11, 129:1–129:10.
    2. Werlberger, M. 2012. Convex Approaches for High Performance Video Processing. PhD thesis, Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria.


ACM Digital Library Publication:



Overview Page: