“High-order differentiable autoencoder for nonlinear model reduction” by Shen, Yang, Shao, Wang, Jiang, et al. …

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton’s method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., 10+ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable weight-varying Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.

1. Lars V Ahlfors. 1973. Complex Analysis. 1979.Google Scholar
2. Steven S An, Theodore Kim, and Doug L James. 2008. Optimizing cubature for efficient integration of subspace deformations. ACM transactions on graphics (TOG) 27, 5 (2008), 1–10.Google Scholar
3. Johannes Ballé, Valero Laparra, and Eero P Simoncelli. 2016. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704 (2016).Google Scholar
4. Jernej Barbič, Marco da Silva, and Jovan Popović. 2009. Deformable object animation using reduced optimal control. In ACM SIGGRAPH 2009 papers. 1–9.Google ScholarDigital Library
5. Jernej Barbič, Funshing Sin, and Eitan Grinspun. 2012. Interactive editing of deformable simulations. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1–8.Google ScholarDigital Library
6. Jernej Barbič and Yili Zhao. 2011. Real-time large-deformation substructuring. ACM transactions on graphics (TOG) 30, 4 (2011), 1–8.Google Scholar
7. Jernej Barbič and Doug L James. 2005. Real-time subspace integration for St. Venant-Kirchhoff deformable models. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 982–990.Google ScholarDigital Library
8. Atilim Günes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2017. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research 18, 1 (2017), 5595–5637.Google ScholarDigital Library
9. Hervé Bourlard and Yves Kamp. 1988. Auto-association by multilayer perceptrons and singular value decomposition. Biological cybernetics 59, 4–5 (1988), 291–294.Google Scholar
10. Alain J Brizard. 2014. Introduction To Lagrangian Mechanics, An. World Scientific Publishing Company.Google Scholar
11. H Martin Bücker, George Corliss, Paul Hovland, Uwe Naumann, and Boyana Norris. 2006. Automatic differentiation: applications, theory, and implementations. Vol. 50. Springer Science & Business Media.Google Scholar
12. Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popović. 2002. Interactive skeleton-driven dynamic deformations. In ACM Trans. Graph. (TOG), Vol. 21. ACM, 586–593.Google ScholarDigital Library
13. Min Gyu Choi and Hyeong-Seok Ko. 2005. Modal warping: Real-time simulation of large rotational deformation and manipulation. IEEE Trans. on Visualization and Computer Graphics 11, 1 (2005), 91–101.Google ScholarDigital Library
14. Marco Fratarcangeli, Valentina Tibaldo, and Fabio Pellacini. 2016. Vivace: A practical gauss-seidel method for stable soft body dynamics. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1–9.Google ScholarDigital Library
15. Lawson Fulton, Vismay Modi, David Duvenaud, David IW Levin, and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. In Computer graphics forum, Vol. 38. Wiley Online Library, 379–391.Google Scholar
16. Benjamin Gilles, Guillaume Bousquet, Francois Faure, and Dinesh K Pai. 2011. Frame-based elastic models. ACM Trans. Graph. (TOG) 30, 2 (2011), 15.Google ScholarDigital Library
17. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. 6.2. 2.3 softmax units for multinoulli output distributions. Deep learning (2016), 180–184.Google ScholarDigital Library
18. Gaël Guennebaud, Benoit Jacob, et al. 2010. Eigen. @URl: http://eigen.tuxfamily.org (2010).Google Scholar
19. Brian Guenter. 2007. Efficient symbolic differentiation for graphics applications. In ACM SIGGRAPH 2007 papers. 108–es.Google ScholarDigital Library
20. Amirhossein Habibian, Ties van Rozendaal, Jakub M Tomczak, and Taco S Cohen. 2019. Video compression with rate-distortion autoencoders. In Proceedings of the IEEE International Conference on Computer Vision. 7033–7042.Google ScholarCross Ref
21. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W Sumner, Forrester Cole, Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace clothing simulation using adaptive bases. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–9.Google ScholarDigital Library
22. David Harmon and Denis Zorin. 2013. Subspace integration with local deformations. ACM Transactions on Graphics(TOG) 32, 4 (2013), 1–10.Google ScholarDigital Library
23. Kris K Hauser, Chen Shen, and James F O’Brien. 2003. Interactive Deformation Using Modal Analysis with Constraints.. In Graphics Interface, Vol. 3. 16–17.Google Scholar
24. Florian Hecht, Yeon Jin Lee, Jonathan R Shewchuk, and James F O’Brien. 2012. Updated sparse cholesky factors for corotational elastodynamics. ACM Trans. Graph. (TOG) 31, 5 (2012), 123.Google ScholarDigital Library
25. Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural networks for perception. Elsevier, 65–93.Google ScholarDigital Library
26. Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science 313, 5786 (2006), 504–507.Google Scholar
27. Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. 2001. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.Google Scholar
28. Theodore Kim and John Delaney. 2013. Subspace fluid re-simulation. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–9.Google ScholarDigital Library
29. Theodore Kim and Doug L James. 2009. Skipping steps in deformable simulation with online model reduction. In ACM Trans. Graph. (TOG), Vol. 28. ACM, 123.Google ScholarDigital Library
30. Theodore Kim and Doug L James. 2012. Physics-based character skinning using multidomain subspace deformations. IEEE transactions on visualization and computer graphics 18, 8 (2012), 1228–1240.Google ScholarDigital Library
31. Dana A Knoll and David E Keyes. 2004. Jacobian-free Newton-Krylov methods: a survey of approaches and applications. J. Comput. Phys. 193, 2 (2004), 357–397.Google ScholarDigital Library
32. L’ubor Ladickỳ, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–9.Google ScholarDigital Library
33. Lei Lan, Ran Luo, Marco Fratarcangeli, Weiwei Xu, Huamin Wang, Xiaohu Guo, Junfeng Yao, and Yin Yang. 2020. Medial Elastics: Efficient and Collision-Ready Deformation via Medial Axis Transform. ACM Transactions on Graphics (TOG) 39, 3 (2020), 1–17.Google ScholarDigital Library
34. Gregory Lantoine, Ryan P Russell, and Thierry Dargent. 2012. Using multicomplex variables for automatic computation of high-order derivatives. ACM Transactions on Mathematical Software (TOMS) 38, 3 (2012), 1–21.Google ScholarDigital Library
35. Tiantian Liu, Adam W. Bargteil, James F. O’Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-spring Systems. ACM Trans. Graph. (TOG) 32, 6 (2013), 214:1–214:7.Google ScholarDigital Library
36. Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. 2017. Quasi-newton methods for real-time simulation of hyperelastic materials. ACM Transactions on Graphics (TOG) 36, 3 (2017), 1–16.Google ScholarDigital Library
37. RanLuo, Weiwei Xu, Tianjia Shao, Hongyi Xu, and Yin Yang. 2019. Accelerated complex-step finite difference for expedient deformable simulation. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–16.Google ScholarDigital Library
38. Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015).Google Scholar
39. Sebastian Martin, Peter Kaufmann, Mario Botsch, Eitan Grinspun, and Markus Gross. 2010. Unified simulation of elastic rods, shells, and solids. In ACM Trans. Graph. (TOG), Vol. 29. ACM, 39.Google ScholarDigital Library
40. Joaquim RRA Martins, Peter Sturdza, and Juan J Alonso. 2003. The complex-step derivative approximation. ACM Transactions on Mathematical Software (TOMS) 29, 3 (2003), 245–262.Google ScholarDigital Library
41. Rajaditya Mukherjee, Xiaofeng Wu, and Huamin Wang. 2016. Incremental deformation subspace reconstruction. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 169–178.Google Scholar
42. Matthias Müller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. 2005. Meshless deformations based on shape matching. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 471–478.Google ScholarDigital Library
43. Przemyslaw Musialski, Christian Hafner, Florian Rist, Michael Birsak, Michael Wimmer, and Leif Kobbelt. 2016. Non-linear shape optimization using local subspace projections. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–13.Google ScholarDigital Library
44. Vinod Nair and Geoffrey E Hinton. 2009. 3D object recognition with deep belief nets. Advances in neural information processing systems 22 (2009), 1339–1347.Google Scholar
45. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.Google Scholar
46. HM Nasir. 2013. A new class of multicomplex algebra with applications. Mathematical Sciences International Research Journals 2, 2 (2013), 163–168.Google Scholar
47. CUDA Nvidia. 2008. Cublas library. NVIDIA Corporation, Santa Clara, California 15, 27 (2008), 31.Google Scholar
48. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019).Google Scholar
49. Alex Pentland and John Williams. 1989. Good vibrations: Modal dynamics for graphics and animation. In Proceedings of the 16th annual conference on Computer graphics and interactive techniques. 215–222.Google ScholarDigital Library
50. Eric Pesheck, Nicolas Boivin, Christophe Pierre, and Steven W Shaw. 2001. Nonlinear modal analysis of structural systems using multi-mode invariant manifolds. Nonlinear Dynamics 25, 1–3 (2001), 183–205.Google ScholarCross Ref
51. Salah Rifai, Grégoire Mesnil, Pascal Vincent, Xavier Muller, Yoshua Bengio, Yann Dauphin, and Xavier Glorot. 2011a. Higher order contractive auto-encoder. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 645–660.Google ScholarDigital Library
52. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. 2011b. Contractive auto-encoders: Explicit invariance during feature extraction. In Icml.Google Scholar
53. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211–252.Google Scholar
54. Sangriyadi Setio, Herlien D Setio, and Louis Jezequel. 1992. Modal analysis of nonlinear multi-degree-of-freedom structures. IJAEM 7, 2 (1992), 75–93.Google Scholar
55. Ahmed A Shabana. 2003. Dynamics of multibody systems. Cambridge university press.Google Scholar
56. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 conference on empirical methods in natural language processing. 151–161.Google Scholar
57. Rasmus Tamstorf, Toby Jones, and Stephen F McCormick. 2015. Smoothed aggregation multigrid for cloth simulation. ACM Trans. Graph. (TOG) 34, 6 (2015), 245.Google ScholarDigital Library
58. Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. 2020. Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv:2006.10739 (2020).Google Scholar
59. Adrien Treuille, Andrew Lewis, and Zoran Popović. 2006. Model reduction for real-time fluids. ACM Transactions on Graphics (TOG) 25, 3 (2006), 826–834.Google ScholarDigital Library
60. Christoph Von-Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2015. Real-time nonlinear shape interpolation. ACM Transactions on Graphics (TOG) 34, 3 (2015), 1–10.Google ScholarDigital Library
61. Huamin Wang, James F O’Brien, and Ravi Ramamoorthi. 2011. Data-driven elastic models for cloth: modeling and measurement. ACM transactions on graphics (TOG) 30, 4 (2011), 1–12.Google ScholarDigital Library
62. Huamin Wang and Yin Yang. 2016. Descent methods for elastic body simulation on the GPU. ACM Trans. Graph. (TOG) 35, 6 (2016), 212.Google ScholarDigital Library
63. Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear subspace design for real-time shape deformation. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1–11.Google ScholarDigital Library
64. Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 71–82.Google Scholar
65. Stephen Wolfram et al. 1999. The MATHEMATICA® book, version 4. Cambridge university press.Google Scholar
66. Xiaofeng Wu, Rajaditya Mukherjee, and Huamin Wang. 2015. A unified approach for subspace simulation of deformable bodies in multiple domains. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–9.Google ScholarDigital Library
67. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2020).Google Scholar
68. Hongyi Xu and Jernej Barbič. 2016. Pose-space subspace dynamics. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–14.Google ScholarDigital Library
69. Hongyi Xu, Yijing Li, Yong Chen, and Jernej Barbič. 2015. Interactive material design using model reduction. ACM Transactions on Graphics (TOG) 34, 2 (2015), 1–14.Google ScholarDigital Library
70. Yin Yang, Dingzeyu Li, Weiwei Xu, Yuan Tian, and Changxi Zheng. 2015. Expediting precomputation for reduced deformable simulation. ACM Trans. Graph. (TOG) 34, 6 (2015).Google ScholarDigital Library
71. Yin Yang, Weiwei Xu, Xiaohu Guo, Kun Zhou, and Baining Guo. 2013. Boundary-aware multidomain subspace deformation. IEEE transactions on visualization and computer graphics 19, 10 (2013), 1633–1645.Google ScholarCross Ref
72. Nianyin Zeng, Hong Zhang, Baoye Song, Weibo Liu, Yurong Li, and Abdullah M Dobaie. 2018. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273 (2018), 643–649.Google ScholarCross Ref
73. Jiayi Eris Zhang, Seungbae Bang, David I.W. Levin, and Alec Jacobson. 2020. Complementary Dynamics. ACM Transactions on Graphics (2020).Google Scholar
74. Yongning Zhu, Eftychios Sifakis, Joseph Teran, and Achi Brandt. 2010. An efficient multigrid method for the simulation of high-resolution elastic solids. ACM Trans. Graph. (TOG) 29, 2 (2010), 16.Google ScholarDigital Library
75. Olgierd Cecil Zienkiewicz, Robert Leroy Taylor, Olgierd Cecil Zienkiewicz, and Robert Lee Taylor. 1977. The finite element method. Vol. 36. McGraw-hill London.Google Scholar