“NeuroSkinning: automatic skin binding for production characters with deep graph networks” by Liu, Zheng, Tang, Yuan, Fan, et al. …
Conference:
Type(s):
Title:
- NeuroSkinning: automatic skin binding for production characters with deep graph networks
Session/Category Title: Animation and Skinning
Presenter(s)/Author(s):
Abstract:
We present a deep-learning-based method to automatically compute skin weights for skeleton-based deformation of production characters. Given a character mesh and its associated skeleton hierarchy in rest pose, our method constructs a graph for the mesh, each node of which encodes the mesh-skeleton attributes of a vertex. An end-to-end deep graph convolution network is then introduced to learn the mesh-skeleton binding patterns from a set of character models with skin weights painted by artists. The network can be used to predict the skin weight map for a new character model, which describes how the skeleton hierarchy influences the mesh vertices during deformation. Our method is designed to work for non-manifold meshes with multiple disjoint or intersected components, which are common in game production and require complex skeleton hierarchies for animation control. We tested our method on the datasets of two commercial games. Experiments show that the predicted skin weight maps can be readily applied to characters in the production pipeline to generate high-quality deformations.
References:
1. Stephen W Bailey, Dave Otte, Paul Dilorenzo, and James F O’Brien. 2018. Fast and deep deformation approximations. ACM Transactions on Graphics (TOG) 37, 4 (2018), 119:1–119:12. Google ScholarDigital Library
2. Seungbae Bang, Byungkuk Choi, Roger Blanco i Ribera, Meekyoung Kim, Sung-Hee Lee, and Junyong Noh. 2015. Interactive rigging with intuitive tools. In Computer Graphics Forum, Vol. 34. 123–132. Google ScholarDigital Library
3. Seungbae Bang and Sung-Hee Lee. 2018. Spline Interface for Intuitive Skinning Weight Editing. ACM Transactions on Graphics (TOG) 37, 5 (2018), 174:1–174:14. Google ScholarDigital Library
4. Ilya Baran and Jovan Popović. 2007. Automatic rigging and animation of 3d characters. ACM Transactions on graphics (TOG) 26, 3 (2007), 72:1–72:8. Google ScholarDigital Library
5. Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. 2016. Learning shape correspondence with anisotropic convolutional neural networks. In Advances in Neural Information Processing Systems. 3189–3197. Google ScholarDigital Library
6. Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral Networks and Locally Connected Networks on Graphs. CoRR abs/1312.6203 (2013).Google Scholar
7. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. 3844–3852. Google ScholarDigital Library
8. Olivier Dionne and Martin de Lasa. 2013. Geodesic voxel binding for production character meshes. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 173–180. Google ScholarDigital Library
9. Olivier Dionne and Martin de Lasa. 2014. Geodesic Binding for Degenerate Character Geometry Using Sparse Voxelization. IEEE Transactions on Visualization and Computer Graphics 20 (2014), 1367–1378.Google ScholarCross Ref
10. Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1024–1034. Google ScholarDigital Library
11. Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep Convolutional Networks on Graph-Structured Data. CoRR abs/1506.05163 (2015).Google Scholar
12. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
13. Alec Jacobson, Ilya Baran, Jovan Popovic, and Olga Sorkine. 2011. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph. 30, 4 (2011), 78–1. Google ScholarDigital Library
14. Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O’Sullivan. 2007. Skinning with dual quaternions. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 39–46. Google ScholarDigital Library
15. Ladislav Kavan and Olga Sorkine. 2012. Elasticity-inspired deformers for character articulation. ACM Transactions on Graphics (TOG) 31, 6 (2012), 196:1–196:8. Google ScholarDigital Library
16. Ladislav Kavan and Jiří Žára. 2005. Spherical blend skinning: a real-time deformation of articulated models. In Proceedings of the 2005 symposium on Interactive 3D graphics and games. 9–16. Google ScholarDigital Library
17. Meekyoung Kim, Gerard Pons-Moll, Sergi Pujades, Seungbae Bang, Jinwook Kim, Michael J Black, and Sung-Hee Lee. 2017. Data-driven physics for human soft tissue animation. ACM Transactions on Graphics (TOG) 36, 4 (2017), 54:1–54:12. Google ScholarDigital Library
18. Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907 (2016).Google Scholar
19. Binh Huy Le and Zhigang Deng. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Transactions on Graphics (TOG) 33, 4 (2014), 84:1–84:10. Google ScholarDigital Library
20. Binh Huy Le and Jessica K Hodgins. 2016. Real-time skeletal skinning with optimized centers of rotation. ACM Transactions on Graphics (TOG) 35, 4 (2016), 37. Google ScholarDigital Library
21. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2015. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics (TOG) 34, 6 (2015), 248:1–248:16. Google ScholarDigital Library
22. Ran Luo, Tianjia Shao, Huamin Wang, Weiwei Xu, Kun Zhou, and Yin Yang. 2018. NNWarp: Neural Network-based Nonlinear Deformation. IEEE Transactions on Visualization and Computer Graphics 24, 11 (2018).Google Scholar
23. Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. 2015. Geodesic convolutional neural networks on riemannian manifolds. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 37–45. Google ScholarDigital Library
24. Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. 2017. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. 3.Google ScholarCross Ref
25. Tomohiko Mukai and Shigeru Kuriyama. 2016. Efficient dynamic skinning with low-rank helper bone controllers. ACM Transactions on Graphics (TOG) 35, 4 (2016), 36:1–36:11. Google ScholarDigital Library
26. Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. 77–85.Google Scholar
27. Yi-Ling Qiao, Lin Gao, Yu-Kun Lai, and Shihong Xia. 2018. Learning Bidirectional LSTM Networks for Synthesizing 3D Mesh Animation Sequences. CoRR abs/1810.02042 (2018).Google Scholar
28. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic biomechanical simulation and control of human swimming. ACM Transactions on Graphics (TOG) 34, 1 (2014), 10:1–10:15. Google ScholarDigital Library
29. Qingyang Tan, Lin Gao, Yu-Kun Lai, Jie Yang, and Shihong Xia. 2018. Mesh-Based Autoencoders for Localized Deformation Component Analysis. In AAAI.Google Scholar
30. Kiran Koshy Thekumparampil, Chong Wang, Sewoong Oh, and Li-Jia Li. 2018. Attention-based Graph Neural Network for Semi-supervised Learning. CoRR abs/1803.03735 (2018).Google Scholar
31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008. Google ScholarDigital Library
32. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. International Conference on Learning Representations (2018).Google Scholar
33. Rich Wareham and Joan Lasenby. 2008. Bone glow: An improved method for the assignment of weights for mesh deformation. In International Conference on Articulated Motion and Deformable Objects. 63–71. Google ScholarDigital Library
