“High-quality, cost-e!ective facial motion capture pipeline with 3D Regression” by , Williams, Hendler and Roble
Conference:
Type(s):
Entry Number: 60
Title:
- High-quality, cost-e!ective facial motion capture pipeline with 3D Regression
Presenter(s)/Author(s):
Abstract:
We present our improved marker-based facial motion capture pipeline that leverages on 3D regression from head-mounted camera (HMC) images to speed up and reduce the cost of high quality 3D marker tracking. We use machine learning to boost productivity by training regressors on traditionally tracked performances and applying those models to the remaining performances. Our specialized regressor for HMC marker-based tracking shows improvements in quality and robustness for marker tracks. The regressor results are automatically refined by a simple blob detection tool and then imported back into the tracking tool such that manual correction can be applied as needed and subsequently included as additional training data. This iterative approach reduces 70% the amount of artist time required for traditional tracking methods and does not add much setup time nor planning as alternative techniques.
References:
Chen Cao, Yanlin Weng, Stephen Lin, and Kun Zhou. 2013. 3D Shape Regression for Real-time Facial Animation. ACM Trans. Graph. 32, 4, Article 41 (July 2013), 10 pages.
X. Cao, Y. Wei, F. Wen, and J. Sun. 2012. Face alignment by Explicit Shape Regression. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2887–2894.
Martin Klaudiny, Steven McDonagh, Derek Bradley, Thabo Beeler, and Kenny Mitchell. 2017. Real-Time Multi-View Facial Capture with Synthetic Training. Computer Graphics Forum (2017).
Samuli Laine, Tero Karras, Timo Aila, Antti Herva, Shunsuke Saito, Ronald Yu, Hao Li, and Jaakko Lehtinen. 2017. Production-level Facial Performance Capture Using Deep Convolutional Neural Networks. In Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA ’17). ACM, New York, NY, USA, Article 10, 10 pages.
S. McDonagh, M. Klaudiny, D. Bradley, T. Beeler, I. Matthews, and K. Mitchell. 2016. Synthetic Prior Design for Real-Time Face Tracking. In 2016 Fourth International Conference on 3D Vision (3DV). 639–648.
Lucio Moser, Darren Hendler, and Doug Roble. 2017. Masquerade: Fine-scale Details for Head-mounted Camera Motion Capture Data. In ACM SIGGRAPH 2017 Talks (SIGGRAPH ’17). ACM, New York, NY, USA, Article 18, 2 pages.
