“Simplified facial capture with head mounted cameras” by Serra, Moser, Mclean and Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

  • ©Jose Serra, Lucio Moser, David Mclean, and Douglas (Doug) Roble

Conference:


Type:


Entry Number: 11

Title:

    Simplified facial capture with head mounted cameras

Presenter(s)/Author(s):



Abstract:


    We present a unified pipeline for high-resolution facial capture that replaces the initial traditional seated capture with a head-mounted camera setup. At its core, our approach relies on improving roughly personalized blendshapes by fitting handle vertices, in a Laplacian framework, to depth and image data. Thus, refining the geometry. This pipeline has been used in production to generate high quality animation to train our proprietary market-based solution, leading to large time and cost savings.

References:


    Beeler, F. Hahn, D. Bradley, B. Bickel, P. Beardsley, C. Gotsman, R. Sumner, and M. Gross. 2011. High-Quality Passive Facial Performance Capture Using Anchor Frames. ACM Trans. Graph. 30, 4, Article 75 (July 2011), 10 pages. DI4D. 2021. DI4D Pro. Retrieved Feb 10, 2021 from https://www.di4d.com/di4d-pro/

    Fyffe, K. Nagano, L. Huynh, S. Saito, J. Busch, A. Jones, H. Li, and P. Debevec. 2017. Multi-View Stereo on Consistent Face Topology. CGF 36 (2017), 295–309.

    Hendler, L. Moser, R. Battulwar, D. Corral, P. Cramer, R. Miller, R. Cloudsdale, and D. Roble. 2018. Avengers: Capturing Thanos’s Complex Face. In ACM SIGGRAPH 2018 Talks. Article 58, 2 pages.

    ICT. 2021. ICT – Light Stages. Retrieved Feb 10, 2021 from https://vgl.ict.usc.edu/ LightStages/

    Moser, D. Hendler, and D. Roble. 2017. Masquerade: Fine-Scale Details for Head- Mounted Camera Motion Capture Data. In ACM SIGGRAPH 2017 Talks. New York, NY, USA, Article 18, 2 pages.

    Park, X. Zhang, A. Bulling, and O. Hilliges. 2018. Learning to Find Eye Region Land- marks for Remote Gaze Estimation in Unconstrained Settings. In ACM Symposium on Eye Tracking Research and Applications (Warsaw, Poland). ACM

    Sorkine. 2005. Laplacian Mesh Processing. In Eurographics 2005 – State of the Art Reports. The Eurographics Association.

Keyword(s):



PDF:



ACM Digital Library Publication: