“Learning contact corrections for handle-based subspace dynamics” by Romero, Casas, Rodríguez and Otaduy

This paper introduces a novel subspace method for the simulation of dynamic deformations. The method augments existing linear handle-based subspace formulations with nonlinear learning-based corrections parameterized by the same subspace. Together, they produce a compact nonlinear model that combines the fast dynamics and overall contact-based interaction of subspace methods, with the highly detailed deformations of learning-based methods. We propose a formulation of the model with nonlinear corrections applied on the local undeformed setting, and decoupling internal and external contact-driven corrections. We define a simple mapping of these corrections to the global setting, an efficient implementation for dynamic simulation, and a training pipeline to generate examples that efficiently cover the interaction space. Altogether, the method achieves unprecedented combination of speed and contact-driven deformation detail.

1. Steven S. An, Theodore Kim, and Doug L. James. 2008. Optimizing Cubature for Efficient Integration of Subspace Deformations. ACM Trans. Graph. 27, 5, Article 165 (2008).Google ScholarDigital Library
2. Stephen W. Bailey, Dave Otte, Paul Dilorenzo, and James F. O’Brien. 2018. Fast and Deep Deformation Approximations. ACM Trans. Graph. 37, 4, Article 119 (2018).Google ScholarDigital Library
3. Jernej Barbič and Doug L. James. 2005. Real-Time Subspace Integration for St. Venant-Kirchhoff Deformable Models. ACM Trans. Graph. 24, 3 (July 2005), 982–990.Google ScholarDigital Library
4. Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray kavukcuoglu. 2016. Interaction Networks for Learning about Objects, Relations and Physics. In Proceedings of the 30th International Conference on Neural Information Processing Systems. 4509–4517.Google Scholar
5. Bernd Bickel, Manuel Lang, Mario Botsch, Miguel A. Otaduy, and Markus Gross. 2008. Pose-Space Animation and Transfer of Facial Details. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 57–66.Google ScholarDigital Library
6. Christopher Brandt, Elmar Eisemann, and Klaus Hildebrandt. 2018. Hyper-Reduced Projective Dynamics. ACM Trans. Graph. 37, 4, Article 80 (2018).Google ScholarDigital Library
7. Dan Casas and Miguel A. Otaduy. 2018. Learning Nonlinear Soft-Tissue Dynamics for Interactive Avatars. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 10 (July 2018), 15 pages.Google Scholar
8. Michael B. Chang, Tomer Ullman, Antonio Torralba, and Joshua B. Tenenbaum. 2017. A Compositional Object-Based Approach to Learning Physical Dynamics. In Proceedings of the International Conference on Learning Representations.Google Scholar
9. Jiong Chen, Hujun Bao, Tianyu Wang, Mathieu Desbrun, and Jin Huang. 2018. Numerical Coarsening Using Discontinuous Shape Functions. ACM Trans. Graph. 37, 4, Article 120 (2018).Google ScholarDigital Library
10. Nuttapong Chentanez, Miles Macklin, Matthias Müller, Stefan Jeschke, and Tae-Yong Kim. 2020. Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network. Computer Graphics Forum 39, 8 (2020), 123–134.Google ScholarDigital Library
11. Edilson de Aguiar, Leonid Sigal, Adrien Treuille, and Jessica K. Hodgins. 2010. Stable Spaces for Real-Time Clothing. ACM Trans. Graph. 29, 4 (2010).Google ScholarDigital Library
12. Ye Fan, Joshua Litven, David I. W. Levin, and Dinesh K. Pai. 2013. Eulerian-on-lagrangian Simulation. ACM Trans. Graph. 32, 3 (2013).Google ScholarDigital Library
13. Lawson Fulton, Vismay Modi, David Duvenaud, David I. W. Levin, and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. Computer Graphics Forum 38, 2 (2019), 379–391.Google ScholarCross Ref
14. Theodore F Gast, Craig Schroeder, Alexey Stomakhin, Chenfanfu Jiang, and Joseph M Teran. 2015. Optimization integrator for large time steps. IEEE Transactions on Visualization and Computer Graphics (TVCG) 21, 10 (2015), 1103–1115.Google ScholarDigital Library
15. Benjamin Gilles, Guillaume Bousquet, Francois Faure, and Dinesh K. Pai. 2011. Frame-Based Elastic Models. ACM Trans. Graph. 30, 2 (2011).Google ScholarDigital Library
16. Samuel Greydanus, Misko Dzamba, and Jason Yosinski. 2019. Hamiltonian Neural Networks. In Advances in Neural Information Processing Systems, Vol. 32. 15379–15389.Google Scholar
17. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rig-Space Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012), 8 pages.Google ScholarDigital Library
18. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W. Sumner, Forrester Cole, Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace Clothing Simulation Using Adaptive Bases. ACM Trans. Graph. 33, 4 (2014).Google ScholarDigital Library
19. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W. Sumner, and Markus Gross. 2013. Efficient Simulation of Secondary Motion in Rig-Space. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 165–171.Google ScholarDigital Library
20. David Harmon and Denis Zorin. 2013. Subspace Integration with Local Deformations. ACM Trans. Graph. 32, 4, Article 107 (July 2013), 10 pages.Google ScholarDigital Library
21. Kris K. Hauser, Chen Shen, and James F. O’Brien. 2003. Interactive Deformation Using Modal Analysis with Constraints. In Proceedings of the Graphics Interface. 247–256.Google Scholar
22. Daniel Holden, Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai. 2019. Subspace Neural Physics: Fast Data-Driven Interactive Simulation. In Proceedings of the 18th Annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation.Google ScholarDigital Library
23. G. Irving, J. Teran, and R. Fedkiw. 2004. Invertible Finite Elements for Robust Simulation of Large Deformation. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 131–140.Google Scholar
24. Ladislav Kavan, Dan Gerszewski, Adam W. Bargteil, and Peter-Pike Sloan. 2011. Physics-Inspired Upsampling for Cloth Simulation in Games. ACM Trans. Graph. 30, 4 (2011).Google ScholarDigital Library
25. Lily Kharevych, Patrick Mullen, Houman Owhadi, and Mathieu Desbrun. 2009. Numerical Coarsening of Inhomogeneous Elastic Materials. ACM Trans. on Graphics 28, 3 (2009), 51:1–51:8.Google ScholarDigital Library
26. Doyub Kim, Woojong Koh, Rahul Narain, Kayvon Fatahalian, Adrien Treuille, and James F. O’Brien. 2013. Near-Exhaustive Precomputation of Secondary Cloth Effects. ACM Trans. Graph. 32, 4 (2013).Google ScholarDigital Library
27. Theodore Kim and Doug L. James. 2011. Physics-Based Character Skinning Using Multi-Domain Subspace Deformations. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 63–72.Google Scholar
28. Paul G. Kry, Doug L. James, and Dinesh K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 153–159.Google Scholar
29. P. Krysl, S. Lall, and J. E. Marsden. 2001. Dimensional model reduction in non-linear finite element dynamics of solids and structures. Internat. J. Numer. Methods Engrg. 51, 4 (2001), 479–504.Google ScholarCross Ref
30. J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH). ACM Press/Addison-Wesley Publishing Co., USA, 165–172.Google ScholarDigital Library
31. Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. 2019. Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids. In Proceedings of the International Conference on Learning Representations.Google Scholar
32. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015), 16 pages.Google ScholarDigital Library
33. R. Luo, T. Shao, H. Wang, W. Xu, X. Chen, K. Zhou, and Y. Yang. 2020. NNWarp: Neural Network-Based Nonlinear Deformation. IEEE Transactions on Visualization and Computer Graphics 26, 4 (2020), 1745–1759.Google Scholar
34. Richard Malgat, Benjamin Gilles, David I. W. Levin, Matthieu Nesme, and François Faure. 2015. Multifarious Hierarchies of Mechanical Models for Artist Assigned Levels-of-detail. In Proceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation. 27–36.Google ScholarDigital Library
35. Sebastian Martin, Bernhard Thomaszewski, Eitan Grinspun, and Markus Gross. 2011. Example-Based Elastic Materials. ACM Trans. Graph. 30, 4, Article 72 (2011).Google ScholarDigital Library
36. Matthias Müller and Markus Gross. 2004. Interactive Virtual Materials. In Proceedings of Graphics Interface. 239–246.Google Scholar
37. M. Müller, R. Keiser, A. Nealen, M. Pauly, M. Gross, and M. Alexa. 2004. Point Based Animation of Elastic, Plastic and Melting Objects. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 141–151.Google Scholar
38. Zherong Pan, Hujun Bao, and Jin Huang. 2015. Subspace Dynamic Simulation Using Rotation-Strain Coordinates. ACM Trans. Graph. 34, 6, Article 242 (Oct. 2015), 12 pages.Google ScholarDigital Library
39. Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.Google Scholar
40. Alex Pentland and John Williams. 1989. Good Vibrations: Modal Dynamics for Graphics and Animation. Computer Graphics 23, 3 (1989), 215–222.Google ScholarDigital Library
41. Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J. Black. 2015. Dyna: A Model of Dynamic Human Shape in Motion. ACM Trans. Graph. 34, 4, Article 120 (July 2015), 14 pages.Google ScholarDigital Library
42. Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin A. Riedmiller, Raia Hadsell, and Peter Battaglia. 2018. Graph Networks as Learnable Physics Engines for Inference and Control. In ICML. 4467–4476.Google Scholar
43. Igor Santesteban, Elena Garces, Miguel A. Otaduy, and Dan Casas. 2020. SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans. Computer Graphics Forum 39, 2 (2020), 65–75.Google ScholarCross Ref
44. Igor Santesteban, Miguel A. Otaduy, and Dan Casas. 2019. Learning-Based Animation of Clothing for Virtual Try-On. Computer Graphics Forum 38, 2 (2019), 355–366.Google Scholar
45. Christian Schumacher, Bernd Bickel, Jan Rys, Steve Marschner, Chiara Daraio, and Markus Gross. 2015. Microstructures to Control Elasticity in 3D Printing. ACM Trans. Graph. 34, 4 (2015), 136:1–136:13.Google ScholarDigital Library
46. Breannan Smith, Fernando De Goes, and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37, 2, Article 12 (March 2018), 15 pages.Google ScholarDigital Library
47. Steven L. Song, Weiqi Shi, and Michael Reed. 2020. Accurate Face Rig Approximation with Deep Differential Subspace Reconstruction. ACM Trans. Graph. 39, 4 (2020). Google ScholarDigital Library
48. Javier Tapia, Cristian Romero, Jesús Pérez, and Miguel A. Otaduy. 2021. Parametric Skeletons with Reduced Soft-Tissue Deformations. Computer Graphics Forum (2021).Google Scholar
49. Camillo J. Taylor and David J. Kriegman. 1994. Minimization on the Lie Group SO(3) and Related Manifolds. Technical Report. Yale University.Google Scholar
50. Yun Teng, Mark Meyer, Tony DeRose, and Theodore Kim. 2015. Subspace Condensation: Full Space Adaptivity for Subspace Deformations. ACM Trans. Graph. 34, 4, Article 76 (July 2015), 9 pages.Google ScholarDigital Library
51. Yun Teng, Miguel A. Otaduy, and Theodore Kim. 2014. Simulating Articulated Subspace Self-Contact. ACM Trans. Graph. 33, 4 (2014).Google ScholarDigital Library
52. Rosell Torres, Alejandro Rodríguez, José M. Espadero, and Miguel A. Otaduy. 2016. High-resolution Interaction with Corotational Coarsening Models. ACM Trans. Graph. 35, 6 (2016), 211:1–211:11.Google ScholarDigital Library
53. Mickeal Verschoor, Dan Casas, and Miguel A. Otaduy. 2020. Tactile Rendering Based on Skin Stress Optimization. ACM Trans. Graph. 39, 4, Article 90 (2020).Google ScholarDigital Library
54. Christoph von Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2013. An Efficient Construction of Reduced Deformable Objects. ACM Trans. Graph. 32, 6, Article 213 (2013).Google ScholarDigital Library
55. Huamin Wang, Florian Hecht, Ravi Ramamoorthi, and James F. O’Brien. 2010. Example-Based Wrinkle Synthesis for Clothing Animation. ACM Trans. Graph. 29, 4, Article 107 (July 2010).Google ScholarDigital Library
56. Robert Y. Wang, Kari Pulli, and Jovan Popović. 2007. Real-Time Enveloping with Rotational Regression. In ACM SIGGRAPH 2007 Papers (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA.Google Scholar
57. Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear Subspace Design for Real-Time Shape Deformation. ACM Trans. Graph. 34, 4, Article 57 (2015).Google ScholarDigital Library
58. S. Wiewel, M. Becher, and N. Thuerey. 2019. Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow. Computer Graphics Forum 38, 2 (2019), 71–82.Google ScholarCross Ref
59. Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T Freeman, and Joshua B Tenenbaum. 2016. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In Proceedings of the 30th International Conference on Neural Information Processing Systems. 82–90.Google ScholarDigital Library
60. Hongyi Xu and Jernej Barbič. 2016. Pose-Space Subspace Dynamics. ACM Trans. Graph. 35, 4, Article 35 (July 2016), 14 pages.Google ScholarDigital Library
61. Hongyi Xu, Funshing Sin, Yufeng Zhu, and Jernej Barbič. 2015. Nonlinear Material Design Using Principal Stretches. ACM Trans. Graph. 34, 4, Article 75 (2015).Google ScholarDigital Library
62. J. S. Zurdo, J. P. Brito, and M. A. Otaduy. 2013. Animating Wrinkles by Example on Non-Skinned Cloth. IEEE Transactions on Visualization & Computer Graphics 19, 01 (2013), 149–158.Google ScholarDigital Library