“Phace: physics-based face modeling and animation”
Conference:
Type:
Title:
- Phace: physics-based face modeling and animation
Session/Category Title: Faces & Hair
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a novel physics-based approach to facial animation. Contrary to commonly used generative methods, our solution computes facial expressions by minimizing a set of non-linear potential energies that model the physical interaction of passive flesh, active muscles, and rigid bone structures. By integrating collision and contact handling into the simulation, our algorithm avoids inconsistent poses commonly observed in generative methods such as blendshape rigs. A novel muscle activation model leads to a robust optimization that faithfully reproduces complex facial articulations. We show how person-specific simulation models can be built from a few expression scans with a minimal data acquisition process and an almost entirely automated processing pipeline. Our method supports temporal dynamics due to inertia or external forces, incorporates skin sliding to avoid unnatural stretching, and offers full control of the simulation parameters, which enables a variety of advanced animation effects. For example, slimming or fattening the face is achieved by simply scaling the volume of the soft tissue elements. We show a series of application demos, including artistic editing of the animation model, simulation of corrective facial surgery, or dynamic interaction with external forces and objects.
References:
1. Oleg Alexander, Mike Rogers, William Lambeth, Jen-Yuan Chiang, Wan-Chun Ma, Chuan-Chang Wang, and Paul Debevec. 2010. The digital Emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, IEEE 30, 4 (2010), 20–31. Google ScholarDigital Library
2. Jérémie Allard, Stéphane Cotin, François Faure, Pierre-Jean Bensoussan, François Poyer, Christian Duriez, Hervé Delingette, and Laurent Grisoni. 2007. Sofa-an open source framework for medical simulation. In MMVR 15-Medicine Meets Virtual Reality, Vol. 125. IOP Press, 13–18.Google Scholar
3. Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani, and Thomas Vetter. 2007. Reconstructing high quality face-surfaces using model based stereo. In ICCV 2007. IEEE, 1–8. Google ScholarCross Ref
4. Vincent Barrielle, Nicolas Stoiber, and Cedric Cagniart. 2016. Blendforces, a Dynamic Framework for Facial Animation. Comp. Graph. Forum (2016).Google Scholar
5. Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, and Markus Gross. 2010. High-Quality Single-Shot Capture of Facial Geometry. ACM Trans. Graph. 29, 3 (2010), 40:1–40:9.Google ScholarDigital Library
6. Thabo Beeler and Derek Bradley. 2014. Rigid stabilization of facial expressions. ACM Trans. Graph. 33, 4 (2014), 44. Google ScholarDigital Library
7. Pascal Bérard, Derek Bradley, Markus Gross, and Thabo Beeler. 2016. Lightweight eye capture using a parametric model. ACM Trans. Graph. 35, 4 (2016), 117. Google ScholarDigital Library
8. Pascal Bérard, Derek Bradley, Maurizio Nitti, Thabo Beeler, and Markus H Gross. 2014. High-quality capture of eyes. ACM Trans. Graph. 33, 6 (2014), 223–1.Google ScholarDigital Library
9. Amit Bermano, Thabo Beeler, Yeara Kozlov, Derek Bradley, Bernd Bickel, and Markus Gross. 2015. Detailed spatio-temporal reconstruction of eyelids. ACM Trans. Graph. 34, 4 (2015), 44. Google ScholarDigital Library
10. Amit H Bermano, Derek Bradley, Thabo Beeler, Fabio Zund, Derek Nowrouzezahrai, Ilya Baran, Olga Sorkine-Hornung, Hanspeter Pfister, Robert W Sumner, Bernd Bickel, and others. 2014. Facial performance enhancement using dynamic shape space analysis. ACM Trans. Graph. 33, 2 (2014), 13. Google ScholarDigital Library
11. Bernd Bickel, Moritz Bächer, Miguel A Otaduy, Wojciech Matusik, Hanspeter Pfister, and Markus Gross. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (2009). Google ScholarDigital Library
12. Bernd Bickel, Peter Kaufmann, Mélina Skouras, Bernhard Thomaszewski, Derek Bradley, Thabo the, Phil Jackson, Steve Marschner, Wojciech Matusik, and Markus Gross. 2012. Physical face cloning. ACM Trans. Graph. 31, 4 (2012), 118. Google ScholarDigital Library
13. Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. of the 26th annual conf. on Comp. graph. and interactive techniques. 187–194.Google ScholarDigital Library
14. Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. 2010. Polygon mesh processing. CRC press. Google ScholarCross Ref
15. Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective dynamics: fusing constraint projections for fast simulation. ACM Trans. Graph. 33, 4 (2014), 154. Google ScholarDigital Library
16. Sofien Bouaziz, Yangang Wang, and Mark Pauly. 2013. Online modeling for realtime facial animation. ACM Trans. Graph. 32, 4 (2013), 40. Google ScholarDigital Library
17. Richard P. Brent. 1971. An algorithm with guaranteed convergence for finding a zero of a function. Comput. J. 14, 4 (1971). Google ScholarCross Ref
18. Vicki Bruce and Andy Young. 1986. Understanding face recognition. British journal of psychology 77, 3 (1986), 305–327. Google ScholarCross Ref
19. Chen Cao, Derek Bradley, Kun Zhou, and Thabo Beeler. 2015. Real-time high-fidelity facial performance capture. ACM Trans. Graph. 34, 4 (2015), 46. Google ScholarDigital Library
20. Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. 2014. Facewarehouse: a 3d facial expression database for visual computing. Visualization and Computer Graphics, IEEE Transactions on 20, 3 (2014), 413–425.Google ScholarDigital Library
21. John E Chadwick, David R Haumann, and Richard E Parent. 1989. Layered construction for deformable animated characters. In ACM Siggraph Computer Graphics, Vol. 23. ACM, 243–252.Google ScholarDigital Library
22. Hon Fai Choi and Silvia S Blemker. 2013. Skeletal muscle fascicle arrangements can be reconstructed using a laplacian vector field simulation. PloS one 8, 10 (2013), e77576.Google ScholarCross Ref
23. Matthew Cong, Michael Bao, Kiran S Bhat, Ronald Fedkiw, and others. 2015. Fully automatic generation of anatomical face simulation models. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. ACM, 175–183.Google ScholarDigital Library
24. Matthew Cong, Kiran S Bhat, and Ronald Fedkiw. 2016. Art-directed muscle simulation for high-end facial animation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. 119–127.Google ScholarDigital Library
25. Ali-Hamadi Dicko, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure, Olivier Palombi, and Marie-Paule Cani. 2013. Anatomy transfer. ACM Trans. Graph. 32, 6 (2013), 188. Google ScholarDigital Library
26. Paul Ekman and Wallace V Friesen. 1977. Facial action coding system. (1977).Google Scholar
27. Ye Fan, Joshua Litven, and Dinesh K Pai. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. 33, 4 (2014), 152. Google ScholarDigital Library
28. Pablo Garrido, Michael Zollhöfer, Chenglei Wu, Derek Bradley, Patrick Pérez, Thabo Beeler, and Christian Theobalt. 2016. Corrective 3D reconstruction of lips from monocular video. ACM Trans. Graph. 35, 6 (2016), 219. Google ScholarDigital Library
29. Liwen Hu, Chongyang Ma, Linjie Luo, and Hao Li. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4 (2015). Google ScholarDigital Library
30. Alexandru Ichim, Ladislav Kavan, Merlin Nimier-David, and Mark Pauly. 2016. Building and Animating User-Specific Volumetric Face Rigs. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim.Google ScholarDigital Library
31. Alexandru Eugen Ichim, the Bouaziz, and Mark Pauly. 2015. Dynamic 3D Avatar Creation from Hand-held Video Input. ACM Trans. Graph. (2015).Google Scholar
32. Alec Jacobson, Ladislav Kavan, and Olga Sorkine-Hornung. 2013. Robust inside-outside segmentation using generalized winding numbers. ACM Trans. Graph. 32, 4 (2013), 33. Google ScholarDigital Library
33. Petr Kadlecek, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Krivanek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6 (2016). Google ScholarDigital Library
34. Kolja Kähler, Jörg Haber, and Hans-Peter Seidel. 2003. Reanimating the dead: reconstruction of expressive faces from skull data. In ACM Trans. Graph., Vol. 22. ACM, 554–561. Google ScholarDigital Library
35. Oliver Klehm, Fabrice Rousselle, Marios Papas, Derek Bradley, Christophe Hery, Bernd Bickel, Wojciech Jarosz, and Thabo Beeler. 2015. Recent advances in facial appearance capture. In Computer Graphics Forum, Vol. 34. 709–733. Google ScholarDigital Library
36. Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (2009), 99.Google ScholarDigital Library
37. John P Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Frederic H Pighin, and Zhigang Deng. 2014. Practice and Theory of Blendshape Facial Models.. In Eurographics (State of the Art Reports). 199–218.Google Scholar
38. Duo Li, Shinjiro Sueda, Debanga R Neog, and Dinesh K Pai. 2013. Thin skin elastodynamics. ACM Trans. Graph. 32, 4 (2013), 49. Google ScholarDigital Library
39. Hao Li, Thibaut Weise, and Mark Pauly. 2010. Example-based facial rigging. In ACM Trans. Graph., Vol. 29. ACM, 32. Google ScholarDigital Library
40. Hao Li, Jihun Yu, Yuting Ye, and Chris Bregler. 2013. Realtime Facial Animation with On-the-fly Correctives. ACM Trans. Graph. 32, 4 (2013). Google ScholarDigital Library
41. Tiantian Liu, Adam W. Bargteil, James F. O’Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-Spring Systems. ACM Trans. Graph. 32, 6 (2013), 209:1–7.Google ScholarDigital Library
42. John E Lloyd, Ian Stavness, and Sidney Fels. 2012. ArtiSynth: A fast interactive biomechanical modeling toolkit combining multibody and finite element simulation. In Soft tissue biomechanical modeling for computer assisted surgery. Springer, 355–394.Google Scholar
43. Wan-Chun Ma, Yi-Hua Wang, Graham Fyffe, Bing-Yu Chen, and Paul Debevec. 2012. A blendshape model that incorporates physical interaction. Computer Animation and Virtual Worlds 23, 3–4 (2012). Google ScholarDigital Library
44. Steve A Maas, Benjamin J Ellis, Gerard A Ateshian, and Jeffrey A Weiss. 2012. FEBio: finite elements for biomechanics. Journal of biomechanical engineering 134, 1 (2012), 011005.Google ScholarCross Ref
45. Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011. Efficient elasticity for character skinning with contact and collisions. In ACM Trans. Graph., Vol. 30. 37. Google ScholarDigital Library
46. Koki Nagano, Graham Fyffe, Oleg Alexander, Jernej Barbic, Hao Li, Abhijeet Ghosh, and Paul Debevec. 2015. Skin microstructure deformation with displacement map convolution. (2015).Google Scholar
47. Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J Black. 2015. Dyna: A model of dynamic human shape in motion. ACM Trans. Graph. 34, 4 (2015), 120. Google ScholarDigital Library
48. Szymon Rusinkiewicz and Marc Levoy. 2001. Efficient variants of the ICP algorithm. In 3-D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on. IEEE, 145–152. Google ScholarCross Ref
49. Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational Bodybuilding: Anatomically-based Modeling of Human Bodies. ACM Trans. Graph. 34, 4 (2015). Google ScholarDigital Library
50. Robert Schleip, Thomas W Findley, Leon Chaitow, and Peter Huijing. 2013. Fascia: the tensional network of the human body: the science and clinical applications in manual and movement therapy. Elsevier Health Sciences.Google Scholar
51. Ken Shoemake and Tom Duff. 1992. Matrix animation and polar decomposition. In Proceedings of the conference on Graphics interface, Vol. 92. Citeseer, 258–264.Google Scholar
52. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic biomechanical simulation and control of human swimming. ACM Trans. Graph. 34, 1 (2014), 10. Google ScholarDigital Library
53. Eftychios Sifakis and Jernej Barbic. 2012. FEM simulation of 3D deformable solids: a practitioner’s guide to theory, discretization and model reduction. In ACM SIGGRAPH 2012 Courses. 20.Google ScholarDigital Library
54. Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM Trans. Graph., Vol. 24. 417–425. Google ScholarDigital Library
55. Olga Sorkine and Marc Alexa. 2007. As-rigid-as-possible surface modeling. In Symposium on Geometry processing, Vol. 4.Google ScholarDigital Library
56. Robert W Sumner and Jovan Popović. 2004. Deformation transfer for triangle meshes. In ACM Trans. Graph., Vol. 23. 399–405. Google ScholarDigital Library
57. Joseph Teran, Sylvia Blemker, V Hing, and Ronald Fedkiw. 2003. Finite volume methods for the simulation of skeletal muscle. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Eurographics Association, 68–74.Google ScholarDigital Library
58. Joseph Teran, Eftychios Sifakis, Silvia S Blemker, Victor Ng-Thow-Hing, Cynthia Lau, and Ronald Fedkiw. 2005a. Creating and simulating skeletal muscle from the visible human data set. Visualization and Computer Graphics, IEEE Transactions on 11, 3 (2005), 317–328.Google ScholarDigital Library
59. Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005b. Robust quasistatic finite elements and flesh simulation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. ACM. Google ScholarDigital Library
60. Daniel Vlasic, Matthew Brand, Hanspeter Pfister, and Jovan Popović. 2005. Face transfer with multilinear models. In ACM Trans. Graph., Vol. 24. ACM, 426–433. Google ScholarDigital Library
61. Javier von der Pahlen, Jorge Jimenez, Etienne Danvoye, Paul Debevec, Graham Fyffe, and Oleg Alexander. 2014. Digital Ira and Beyond: Creating Real-time Photoreal Digital Actors. In ACM SIGGRAPH 2014 Courses (SIGGRAPH ’14). Article 1, 384 pages. Google ScholarDigital Library
62. Andreas Wächter and Lorenz T Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006). Google ScholarDigital Library
63. Thibaut Weise, Sofien Bouaziz, Hao Li, and Mark Pauly. 2011. Realtime performance-based facial animation. In ACM Trans. Graph., Vol. 30. ACM, 77. Google ScholarDigital Library
64. Chenglei Wu, Derek Bradley, Pablo Garrido, Michael Zollhöfer, Christian Theobalt, Markus Gross, and Thabo Beeler. 2016a. Model-based Teeth Reconstruction. ACM Trans. Graph. 35, 6 (2016). Google ScholarDigital Library
65. Chenglei Wu, Derek Bradley, Markus Gross, and Thabo Beeler. 2016b. An anatomically-constrained local deformation model for monocular face capture. ACM Trans. Graph. 35, 4 (2016), 115.Google ScholarDigital Library
66. Zygote. 2016. Zygote Body. (2016). https://zygotebody.com [Online; accessed 28-Dec-2016].Google Scholar
