“Learning a Generalized Physical Face Model From Data”
Conference:
Type(s):
Title:
- Learning a Generalized Physical Face Model From Data
Presenter(s)/Author(s):
Abstract:
In this work, we aim to democratize physics-based facial animation by proposing a generalized physical face model that we learn from a large 3D face dataset. Once trained, our model can be quickly fit to any unseen identity and automatically produce a ready-to-animate physical face model.
References:
[1]
Michael Bao, Matthew Cong, St?phane Grabli, and Ronald Fedkiw. 2018. High-Quality Face Capture Using Anatomical Muscles. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (12 2018). http://arxiv.org/abs/1812.02836
[2]
Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Siggraph, Vol. 99. 187–194.
[3]
Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective dynamics: fusing constraint projections for fast simulation. ACM Transactions on Graphics 33, 4 (7 2014), 1–11.
[4]
Zenghao Chai, Haoxian Zhang, Jing Ren, Di Kang, Zhengzhuo Xu, Xuefei Zhe, Chun Yuan, and Linchao Bao. 2022. REALY: Rethinking the Evaluation of 3D Face Reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV).
[5]
Prashanth Chandran, Derek Bradley, Markus Gross, and Thabo Beeler. 2020. Semantic Deep Face Models. In 2020 International Conference on 3D Vision (3DV). IEEE, 345–354.
[6]
Prashanth Chandran, Lo?c Ciccone, Markus Gross, and Derek Bradley. 2022. Local Anatomically-Constrained Facial Performance Retargeting. ACM Trans. Graph. 41, 4, Article 168 (jul 2022).
[7]
Prashanth Chandran and Gaspard Zoss. 2023. Anatomically Constrained Implicit Face Models. arXiv:2312.07538 [cs.GR]
[8]
Prashanth Chandran, Gaspard Zoss, Paulo Gotardo, and Derek Bradley. 2023. Continuous Landmark Detection With 3D Queries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16858–16867.
[9]
Byungkuk Choi, Haekwang Eom, Benjamin Mouscadet, Stephen Cullingford, Kurt Ma, Stefanie Gassel, Suzi Kim, Andrew Moffat, Millicent Maier, Marco Revelant, Joe Letteri, and Karan Singh. 2022. Animatomy: An Animator-Centric, Anatomically Inspired System for 3D Facial Modeling, Animation and Transfer. In SIGGRAPH Asia 2022 Conference Papers. Association for Computing Machinery, Article 16, 9 pages.
[10]
Hang Dai, Nick Pears, William Smith, and Christian Duncan. 2019. Statistical Modeling of Craniofacial Shape and Texture. International Journal of Computer Vision (2019).
[11]
Radek Danecek, Michael J. Black, and Timo Bolkart. 2022. EMOCA: Emotion Driven Monocular Face Capture and Animation. In Conference on Computer Vision and Pattern Recognition (CVPR). 20311–20322.
[12]
Bernhard Egger, William A. P. Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, Christian Theobalt, Volker Blanz, and Thomas Vetter. 2020. 3D Morphable Face Models—Past, Present, and Future. ACM Transactions on Graphics 39, 5 (10 2020), 1–38.
[13]
Yao Feng, Haiwen Feng, Michael J. Black, and Timo Bolkart. 2021. Learning an Animatable Detailed 3D Face Model from In-the-Wild Images. ACM Transactions on Graphics (ToG), Proc. SIGGRAPH 40, 4 (Aug. 2021), 88:1–88:13.
[14]
Panagiotis P. Filntisis, George Retsinas, Foivos Paraperas-Papantoniou, Athanasios Katsamanis, Anastasios Roussos, and Petros Maragos. 2022. Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos. arXiv preprint arXiv:2207.11094 (2022).
[15]
Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou. 2019. GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[16]
Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos P Zafeiriou. 2021. Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
[17]
Aurel Gruber, Marco Fratarcangeli, Gaspard Zoss, Roman Cattaneo, Thabo Beeler, Markus Gross, and Derek Bradley. 2020. Interactive Sculpting of Digital Faces Using an Anatomical Modeling Paradigm. Computer Graphics Forum (2020), 93–102.
[18]
Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016).
[19]
Alexandru-Eugen Ichim, Petr Kadle?ek, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based Face Modeling and Animation. ACM Transactions on Graphics 36, 4 (7 2017), 1–14.
[20]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (12 2015). http://arxiv.org/abs/1412.6980
[21]
Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nie?ner. 2023. Nersemble: Multi-view radiance field reconstruction of human heads. ACM Transactions on Graphics (TOG) 42, 4 (2023), 1–14.
[22]
Gergely Kl?r, Andrew Moffat, Ken Museth, and Eftychios Sifakis. 2020. Shape Targeting: A Versatile Active Elasticity Constitutive Model. In ACM SIGGRAPH 2020 Talks (Virtual Event, USA) (SIGGRAPH ’20). Association for Computing Machinery, New York, NY, USA, Article 59, 2 pages.
[23]
Bo Li, Lingchen Yang, and Barbara Solenthaler. 2023. Efficient Incremental Potential Contact for Actuated Face Simulation. In SIGGRAPH Asia 2023 Technical Communications. 1–4.
[24]
Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. 2017. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 36, 6 (2017).
[25]
Hsueh-Ti Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, and Or Litany. 2022. Learning Smooth Neural Functions via Lipschitz Regularization. arXiv preprint arXiv:2202.08345 (2022).
[26]
Araceli Morales, Gemma Piella, and Federico M. Sukno. 2020. Survey on 3D face reconstruction from uncalibrated images. Comput. Sci. Rev. 40 (2020), 100400.
[27]
Christopher Otto, Prashanth Chandran, Gaspard Zoss, Markus Gross, Paulo Gotardo, and Derek Bradley. 2023. A Perceptual Shape Loss for Monocular 3D Face Reconstruction. arXiv:2310.19580 [cs.CV]
[28]
Zesong Qiu, Yuwei Li, Dongming He, Qixuan Zhang, Longwen Zhang, Yinghao Zhang, Jingya Wang, Lan Xu, Xudong Wang, Yuyao Zhang, et al. 2022. SCULPTOR: Skeleton-consistent face creation using a learned parametric generator. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1–17.
[29]
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378 (2019), 686–707.
[30]
Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Transactions on Graphics 24, 3 (7 2005), 417–425.
[31]
Vincent Sitzmann, Julien N P Martel, Alexander W Bergman, David B Lindell, and Gordon Wetzstein. 2020. Implicit Neural Representations with Periodic Activation Functions. In NeurIPS 2020, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/hash/53c04118df112c13a8c34b38343b9c10-Abstract.html
[32]
Sangeetha Grama Srinivasan, Qisi Wang, Junior Rojas, Gergely Kl?r, Ladislav Kavan, and Eftychios Sifakis. 2021. Learning active quasistatic physics-based models from data. ACM Transactions on Graphics 40, 4 (8 2021), 1–14.
[33]
Nicolas Wagner, Mario Botsch, and Ulrich Schwanecke. 2023. SoftDECA: Computationally Efficient Physics-Based Facial Animations. In Proceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG ’23). Association for Computing Machinery, New York, NY, USA, Article 11, 11 pages.
[34]
Chenglei Wu, Derek Bradley, Markus Gross, and Thabo Beeler. 2016. An anatomically-constrained local deformation model for monocular face capture. ACM Transactions on Graphics 35, 4 (7 2016), 1–12.
[35]
Ming Xu and James Yang. 2015. Human facial soft tissue thickness and mechanical properties: a literature review. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 57045. American Society of Mechanical Engineers, V01AT02A045.
[36]
Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. 2020a. FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[37]
Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. 2020b. Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face prediction. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition. 601–610.
[38]
Lingchen Yang, Byungsoo Kim, Gaspard Zoss, Baran G?zc?, Markus Gross, and Barbara Solenthaler. 2022. Implicit neural representation for physics-driven actuated soft bodies. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–10.
[39]
Lingchen Yang, Gaspard Zoss, Prashanth Chandran, Paulo Gotardo, Markus Gross, Barbara Solenthaler, Eftychios Sifakis, and Derek Bradley. 2023. An Implicit Physical Face Model Driven by Expression and Style. In SIGGRAPH Asia 2023 Conference Papers. Article 106, 12 pages.
[40]
Tianke Zhang, Xuangeng Chu, Yunfei Liu, Lijian Lin, Zhendong Yang, Zhengzhuo Xu, Chengkun Cao, Fei Yu, Changyin Zhou, Chun Yuan, and Yu Li. 2023. Accurate 3D Face Reconstruction with Facial Component Tokens. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9033–9042.
[41]
Mingwu Zheng, Hongyu Yang, Di Huang, and Liming Chen. 2022. Imface: A nonlinear 3d morphable face model with implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20343–20352.
[42]
Wojciech Zielonka, Timo Bolkart, and Justus Thies. 2022. Towards Metrical Reconstruction of Human Faces. In European Conference on Computer Vision.
