“Compressed Skinning for Facial Blendshapes”
Conference:
Type(s):
Title:
- Compressed Skinning for Facial Blendshapes
Presenter(s)/Author(s):
Abstract:
We present a new skinning decomposition method and its application to facial animation, specifically blendshape compression. Our new algorithm based on Adam with projection outperforms SOTA (Dem Bones) and even allows us to sparsify the bone-bone transformations for further compression.
References:
[1]
Michal Aharon, Michael Elad, and Alfred Bruckstein. 2006. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing 54, 11 (2006), 4311?4322.
[2]
Stephen W. Bailey, Dalton Omens, Paul Dilorenzo, and James F. O?Brien. 2020. Fast and Deep Facial Deformations. ACM Transactions on Graphics 39, 4 (Aug. 2020), 94:1?15. https://doi.org/10.1145/3386569.3392397 Presented at SIGGRAPH 2020, Washington D.C.
[3]
Stephen W Bailey, Dave Otte, Paul Dilorenzo, and James F O?Brien. 2018. Fast and deep deformation approximations. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1?12.
[4]
Gaurav Bharaj, Thorsten Thorm?hlen, Hans-Peter Seidel, and Christian Theobalt. 2012. Automatically rigging multi-component characters. In Computer Graphics Forum, Vol. 31. 755?764.
[5]
Samuel R Buss and Jay P Fillmore. 2001. Spherical averages and applications to spherical splines and interpolation. ACM Transactions on Graphics (TOG) 20, 2 (2001), 95?126.
[6]
Prashanth Chandran, Gaspard Zoss, Markus Gross, Paulo Gotardo, and Derek Bradley. 2022. Facial Animation with Disentangled Identity and Motion using Transformers. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 267?277.
[7]
Byungkuk Choi, Haekwang Eom, Benjamin Mouscadet, Stephen Cullingford, Kurt Ma, Stefanie Gassel, Suzi Kim, Andrew Moffat, Millicent Maier, Marco Revelant, 2022. Animatomy: An animator-centric, anatomically inspired system for 3d facial modeling, animation and transfer. In SIGGRAPH Asia 2022 Conference Papers. 1?9.
[8]
Timothy Costigan, Anton Gerdelan, Emma Carrigan, and Rachel McDonnell. 2016. Improving blendshape performance for crowds with GPU and GPGPU techniques. In Proceedings of the 9th International Conference on Motion in Games. 73?78.
[9]
David L Donoho. 2006. Compressed sensing. IEEE Transactions on information theory 52, 4 (2006), 1289?1306.
[10]
Paul Ekman and Wallace V Friesen. 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior (1978).
[11]
Electronic Arts. [n. d.]. Dem Bones: an Open Source Library for Skinning Decomposition. https://www.ea.com/seed/news/open-source-dem-bones
[12]
H. Goldstein, C.P. Poole, and J.L. Safko. 2002. Classical Mechanics. Addison Wesley. https://books.google.ch/books?id=tJCuQgAACAAJ
[13]
Nils Hasler, Thorsten Thorm?hlen, Bodo Rosenhahn, and Hans-Peter Seidel. 2010. Learning skeletons for shape and pose. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games. 23?30.
[14]
Patrik O Hoyer. 2004. Non-negative matrix factorization with sparseness constraints.Journal of machine learning research 5, 9 (2004).
[15]
Alec Jacobson, Ilya Baran, Jovan Popovic, and Olga Sorkine. 2011. Bounded biharmonic weights for real-time deformation.ACM Trans. Graph. 30, 4 (2011), 78.
[16]
Doug L James and Christopher D Twigg. 2005. Skinning mesh animations. ACM Transactions on Graphics (TOG) 24, 3 (2005), 399?407.
[17]
Ladislav Kavan, Rachel McDonnell, Simon Dobbyn, Ji? ??ra, and Carol O?Sullivan. 2007. Skinning arbitrary deformations. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 53?60.
[18]
Ladislav Kavan, P-P Sloan, and Carol O?Sullivan. 2010. Fast and efficient skinning of animated meshes. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 327?336.
[19]
Joonho Kim and Karan Singh. 2021. Optimizing UI layouts for deformable face-rig manipulation. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1?12.
[20]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[21]
Binh Huy Le and Zhigang Deng. 2012. Smooth Skinning Decomposition with Rigid Bones. ACM Trans. Graph. 31, 6 (2012).
[22]
Binh Huy Le and Zhigang Deng. 2013. Two-layer sparse compression of dense-weight blend skinning. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1?10.
[23]
Binh Huy Le and Zhigang Deng. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1?10.
[24]
Daniel D Lee and H Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788?791.
[25]
John P Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Frederic H Pighin, and Zhigang Deng. 2014. Practice and theory of blendshape facial models.Eurographics (State of the Art Reports) 1, 8 (2014), 2.
[26]
Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. 2017. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 36, 6 (2017), 194:1?194:17. https://doi.org/10.1145/3130800.3130813
[27]
Nadia Magnenat-Thalmann, Richard Laperrire, and Daniel Thalmann. 1988. Joint-dependent local deformations for hand animation and object grasping. In In Proceedings on Graphics Interface 1988.
[28]
Peter Melchior, R?my Joseph, and Fred Moolekamp. 2019. Proximal Adam: robust adaptive update scheme for constrained optimization. arXiv preprint arXiv:1910.10094 (2019).
[29]
Mark Meyer and John Anderson. 2007. Key point subspace acceleration and soft caching. In ACM SIGGRAPH 2007 papers. 74?es.
[30]
Anastasia Moutafidou, Vasileios Toulatzis, and Ioannis Fudos. 2023. Deep fusible skinning of animation sequences. The Visual Computer (2023), 1?21.
[31]
Thomas Neumann, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Christian Theobalt. 2013. Sparse localized deformation components. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1?10.
[32]
Neal Parikh, Stephen Boyd, 2014. Proximal algorithms. Foundations and trends? in Optimization 1, 3 (2014), 127?239.
[33]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
[34]
Sarah Radzihovsky, Fernando de Goes, and Mark Meyer. 2020. Facebaker: Baking character facial rigs with machine learning. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Talks. 1?2.
[35]
Scott Schaefer and Can Yuksel. 2007. Example-based skeleton extraction. In Symposium on Geometry Processing. 153?162.
[36]
Jaewoo Seo, Geoffrey Irving, John P Lewis, and Junyong Noh. 2011. Compression and direct manipulation of complex blendshape models. ACM Transactions on Graphics (TOG) 30, 6 (2011), 1?10.
[37]
Marco Tarini, Daniele Panozzo, and Olga Sorkine-Hornung. 2014. Accurate and efficient lighting for skinned models. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 421?428.
[38]
Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58, 1 (1996), 267?288.
