“SFLSH: Shape-Dependent Soft-Flesh Avatars” by Ramón, Romero, Tapia and Otaduy
Conference:
Type(s):
Title:
- SFLSH: Shape-Dependent Soft-Flesh Avatars
Session/Category Title: Flesh & Bones
Presenter(s)/Author(s):
Abstract:
We present a multi-person soft-tissue avatar model. This model maps a body shape descriptor to heterogeneous geometric and mechanical parameters of a soft-tissue model across the body, effectively producing a shape-dependent parametric soft avatar model. The design of the model overcomes two major challenges, the potential redundancy of geometric and mechanical parameters, and the complexity to obtain abundant subject data, which together induce major risk of overfitting the resulting model. To overcome these challenges, we introduce a local shape-dependent regularization of the model. We demonstrate accurate results, on par with independent per-subject estimation, accurate interpolation within the range of body shapes of the training subjects, and good generalization to unseen body shapes. As a result, we obtain a parametric soft-flesh avatar model easy to integrate in many existing applications.
References:
[1]
Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape Completion and Animation of People. ACM Trans. Graph. 24, 3 (July 2005), 408–416.
[2]
Ziqian Bai, Timur Bagautdinov, Javier Romero, Michael Zollhöfer, Ping Tan, and Shunsuke Saito. 2022. AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling. In Computer Vision – ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 222–239.
[3]
Mary Ann Branch, Thomas F. Coleman, and Yuying Li. 1999. A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems. SIAM Journal on Scientific Computing 21, 1 (1999), 1–23. https://doi.org/10.1137/S1064827595289108
[4]
Dan Casas and Miguel A Otaduy. 2018. Learning nonlinear soft-tissue dynamics for interactive avatars. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 1 (2018), 10.
[5]
Ye Fan, Joshua Litven, and Dinesh K. Pai. 2014. Active Volumetric Musculoskeletal Systems. ACM Trans. Graph. 33, 4, Article 152 (2014), 9 pages.
[6]
S. Foti, B. Koo, D. Stoyanov, and M. J. Clarkson. 2022. 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 18709–18718.
[7]
Darcy Harrison, Ye Fan, and Dinesh K Pai Egor Larionov. 2018. Fitting close-to-body garments with 3D soft body avatars. In Proceedings of 3DBODY. TECH. Lugano, Switzerland, 184–189.
[8]
Mustafa Işık, Martin Rünz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, and Matthias Nießner. 2023. HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. ACM Transactions on Graphics (TOG) 42, 4 (2023), 1–12. https://doi.org/10.1145/3592415
[9]
Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-time Deformation. ACM Trans. Graph. 30, 4, Article 78 (July 2011), 8 pages.
[10]
C. Kane, J. E. Marsden, M. Ortiz, and M. West. 2000. Variational integrators and the Newmark algorithm for conservative and dissipative mechanical systems. Internat. J. Numer. Methods Engrg. 49, 10 (2000), 1295–1325.
[11]
Meekyoung Kim, Gerard Pons-Moll, Sergi Pujades, Seungbae Bang, Jinwook Kim, Michael J. Black, and Sung-Hee Lee. 2017. Data-Driven Physics for Human Soft Tissue Animation. ACM Trans. Graph. 36, 4, Article 54 (2017), 12 pages. https://doi.org/10.1145/3072959.3073685
[12]
Martin Komaritzan, Stephan Wenninger, and Mario Botsch. 2021. Inside Humans: Creating a Simple Layered Anatomical Model from Human Surface Scans. Frontiers in Virtual Reality 2 (2021).
[13]
Seunghwan Lee, Yifeng Jiang, and C. Karen Liu. 2023. Anatomically Detailed Simulation of Human Torso. ACM Trans. on Graphics 42 (2023).
[14]
Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable Muscle-Actuated Human Simulation and Control. ACM Trans. Graph. 38, 4, Article 73 (2019), 13 pages.
[15]
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. 2019. Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids. In Proceedings of the International Conference on Learning Representations.
[16]
Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-Driven Soft Body Characters. ACM Trans. Graph. 32, 6, Article 215 (Nov. 2013), 8 pages.
[17]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015), 16 pages.
[18]
Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011. Efficient Elasticity for Character Skinning with Contact and Collisions. ACM Trans. Graph. 30, 4, Article 37 (July 2011), 12 pages.
[19]
A. Nava, E. Mazza, M. Furrer, P. Villiger, and W.H. Reinhart. 2008. In vivo mechanical characterization of human liver. Medical Image Analysis 12, 2 (2008), 203–216.
[20]
Dinesh K. Pai, Austin Rothwell, Pearson Wyder-Hodge, Alistair Wick, Ye Fan, Egor Larionov, Darcy Harrison, Debanga Raj Neog, and Cole Shing. 2018. The Human Touch: Measuring Contact with Real Human Soft Tissues. ACM Trans. Graph. 37, 4 (2018), 58:1–58:12.
[21]
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. 2019. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). 10975–10985.
[22]
Fred Pighin and J. P. Lewis. 2007. Practical Least-Squares for Computer Graphics: Video Files Associated with This Course Are Available from the Citation Page. In ACM SIGGRAPH 2007 Courses (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 1–57.
[23]
Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J. Black. 2015. Dyna: A Model of Dynamic Human Shape in Motion. ACM Trans. Graph. 34, 4, Article 120 (July 2015), 14 pages.
[24]
Cristian Romero, Miguel A. Otaduy, Dan Casas, and Jesus Perez. 2020. Modeling and Estimation of Nonlinear Skin Mechanics for Animated Avatars. Computer Graphics Forum (Proc. Eurographics) 39, 2 (2020).
[25]
Igor Santesteban, Elena Garces, Miguel A. Otaduy, and Dan Casas. 2020. SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans. Computer Graphics Forum 39, 2 (2020), 65–75.
[26]
Breannan Smith, Fernando De Goes, and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37, 2, Article 12 (March 2018), 15 pages.
[27]
Javier Tapia, Cristian Romero, Jesus Perez, and Miguel A. Otaduy. 2021. Parametric Skeletons with Reduced Soft-Tissue Deformations. Computer Graphics Forum (2021).
[28]
Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear Subspace Design for Real-Time Shape Deformation. ACM Trans. Graph. 34, 4, Article 57 (2015).
[29]
Hongyi Xu and Jernej Barbič. 2016. Pose-Space Subspace Dynamics. ACM Trans. Graph. 35, 4, Article 35 (July 2016), 14 pages.
[30]
Mianlun Zheng, Yi Zhou, Duygu Ceylan, and Jernej Barbic. 2021. A Deep Emulator for Secondary Motion of 3D Characters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5932–5940.
[31]
Zerong Zheng, Xiaochen Zhao, Hongwen Zhang, Boning Liu, and Yebin Liu. 2023. AvatarReX: Real-Time Expressive Full-Body Avatars. ACM Trans. Graph. 42, 4, Article 158 (2023).
