“High-order differentiable autoencoder for nonlinear model reduction” by Shen, Yang, Shao, Wang, Jiang, et al. …

  • ©Siyuan Shen, Yin Yang, Tianjia Shao, He Wang, Chenfanfu Jiang, Lei Lan, and Kun Zhou




    High-order differentiable autoencoder for nonlinear model reduction



    This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton’s method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., 10+ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable weight-varying Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.


    1. Lars V Ahlfors. 1973. Complex Analysis. 1979.Google Scholar
    2. Steven S An, Theodore Kim, and Doug L James. 2008. Optimizing cubature for efficient integration of subspace deformations. ACM transactions on graphics (TOG) 27, 5 (2008), 1–10.Google Scholar
    3. Johannes Ballé, Valero Laparra, and Eero P Simoncelli. 2016. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704 (2016).Google Scholar
    4. Jernej Barbič, Marco da Silva, and Jovan Popović. 2009. Deformable object animation using reduced optimal control. In ACM SIGGRAPH 2009 papers. 1–9.Google ScholarDigital Library
    5. Jernej Barbič, Funshing Sin, and Eitan Grinspun. 2012. Interactive editing of deformable simulations. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1–8.Google ScholarDigital Library
    6. Jernej Barbič and Yili Zhao. 2011. Real-time large-deformation substructuring. ACM transactions on graphics (TOG) 30, 4 (2011), 1–8.Google Scholar
    7. Jernej Barbič and Doug L James. 2005. Real-time subspace integration for St. Venant-Kirchhoff deformable models. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 982–990.Google ScholarDigital Library
    8. Atilim Günes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2017. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research 18, 1 (2017), 5595–5637.Google ScholarDigital Library
    9. Hervé Bourlard and Yves Kamp. 1988. Auto-association by multilayer perceptrons and singular value decomposition. Biological cybernetics 59, 4–5 (1988), 291–294.Google Scholar
    10. Alain J Brizard. 2014. Introduction To Lagrangian Mechanics, An. World Scientific Publishing Company.Google Scholar
    11. H Martin Bücker, George Corliss, Paul Hovland, Uwe Naumann, and Boyana Norris. 2006. Automatic differentiation: applications, theory, and implementations. Vol. 50. Springer Science & Business Media.Google Scholar
    12. Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popović. 2002. Interactive skeleton-driven dynamic deformations. In ACM Trans. Graph. (TOG), Vol. 21. ACM, 586–593.Google ScholarDigital Library
    13. Min Gyu Choi and Hyeong-Seok Ko. 2005. Modal warping: Real-time simulation of large rotational deformation and manipulation. IEEE Trans. on Visualization and Computer Graphics 11, 1 (2005), 91–101.Google ScholarDigital Library
    14. Marco Fratarcangeli, Valentina Tibaldo, and Fabio Pellacini. 2016. Vivace: A practical gauss-seidel method for stable soft body dynamics. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1–9.Google ScholarDigital Library
    15. Lawson Fulton, Vismay Modi, David Duvenaud, David IW Levin, and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. In Computer graphics forum, Vol. 38. Wiley Online Library, 379–391.Google Scholar
    16. Benjamin Gilles, Guillaume Bousquet, Francois Faure, and Dinesh K Pai. 2011. Frame-based elastic models. ACM Trans. Graph. (TOG) 30, 2 (2011), 15.Google ScholarDigital Library
    17. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. 6.2. 2.3 softmax units for multinoulli output distributions. Deep learning (2016), 180–184.Google ScholarDigital Library
    18. Gaël Guennebaud, Benoit Jacob, et al. 2010. Eigen. @URl: http://eigen.tuxfamily.org (2010).Google Scholar
    19. Brian Guenter. 2007. Efficient symbolic differentiation for graphics applications. In ACM SIGGRAPH 2007 papers. 108–es.Google ScholarDigital Library
    20. Amirhossein Habibian, Ties van Rozendaal, Jakub M Tomczak, and Taco S Cohen. 2019. Video compression with rate-distortion autoencoders. In Proceedings of the IEEE International Conference on Computer Vision. 7033–7042.Google ScholarCross Ref
    21. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W Sumner, Forrester Cole, Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace clothing simulation using adaptive bases. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–9.Google ScholarDigital Library
    22. David Harmon and Denis Zorin. 2013. Subspace integration with local deformations. ACM Transactions on Graphics(TOG) 32, 4 (2013), 1–10.Google ScholarDigital Library
    23. Kris K Hauser, Chen Shen, and James F O’Brien. 2003. Interactive Deformation Using Modal Analysis with Constraints.. In Graphics Interface, Vol. 3. 16–17.Google Scholar
    24. Florian Hecht, Yeon Jin Lee, Jonathan R Shewchuk, and James F O’Brien. 2012. Updated sparse cholesky factors for corotational elastodynamics. ACM Trans. Graph. (TOG) 31, 5 (2012), 123.Google ScholarDigital Library
    25. Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural networks for perception. Elsevier, 65–93.Google ScholarDigital Library
    26. Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science 313, 5786 (2006), 504–507.Google Scholar
    27. Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. 2001. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.Google Scholar
    28. Theodore Kim and John Delaney. 2013. Subspace fluid re-simulation. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–9.Google ScholarDigital Library
    29. Theodore Kim and Doug L James. 2009. Skipping steps in deformable simulation with online model reduction. In ACM Trans. Graph. (TOG), Vol. 28. ACM, 123.Google ScholarDigital Library
    30. Theodore Kim and Doug L James. 2012. Physics-based character skinning using multidomain subspace deformations. IEEE transactions on visualization and computer graphics 18, 8 (2012), 1228–1240.Google ScholarDigital Library
    31. Dana A Knoll and David E Keyes. 2004. Jacobian-free Newton-Krylov methods: a survey of approaches and applications. J. Comput. Phys. 193, 2 (2004), 357–397.Google ScholarDigital Library
    32. L’ubor Ladickỳ, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–9.Google ScholarDigital Library
    33. Lei Lan, Ran Luo, Marco Fratarcangeli, Weiwei Xu, Huamin Wang, Xiaohu Guo, Junfeng Yao, and Yin Yang. 2020. Medial Elastics: Efficient and Collision-Ready Deformation via Medial Axis Transform. ACM Transactions on Graphics (TOG) 39, 3 (2020), 1–17.Google ScholarDigital Library
    34. Gregory Lantoine, Ryan P Russell, and Thierry Dargent. 2012. Using multicomplex variables for automatic computation of high-order derivatives. ACM Transactions on Mathematical Software (TOMS) 38, 3 (2012), 1–21.Google ScholarDigital Library
    35. Tiantian Liu, Adam W. Bargteil, James F. O’Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-spring Systems. ACM Trans. Graph. (TOG) 32, 6 (2013), 214:1–214:7.Google ScholarDigital Library
    36. Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. 2017. Quasi-newton methods for real-time simulation of hyperelastic materials. ACM Transactions on Graphics (TOG) 36, 3 (2017), 1–16.Google ScholarDigital Library
    37. RanLuo, Weiwei Xu, Tianjia Shao, Hongyi Xu, and Yin Yang. 2019. Accelerated complex-step finite difference for expedient deformable simulation. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–16.Google ScholarDigital Library
    38. Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015).Google Scholar
    39. Sebastian Martin, Peter Kaufmann, Mario Botsch, Eitan Grinspun, and Markus Gross. 2010. Unified simulation of elastic rods, shells, and solids. In ACM Trans. Graph. (TOG), Vol. 29. ACM, 39.Google ScholarDigital Library
    40. Joaquim RRA Martins, Peter Sturdza, and Juan J Alonso. 2003. The complex-step derivative approximation. ACM Transactions on Mathematical Software (TOMS) 29, 3 (2003), 245–262.Google ScholarDigital Library
    41. Rajaditya Mukherjee, Xiaofeng Wu, and Huamin Wang. 2016. Incremental deformation subspace reconstruction. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 169–178.Google Scholar
    42. Matthias Müller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. 2005. Meshless deformations based on shape matching. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 471–478.Google ScholarDigital Library
    43. Przemyslaw Musialski, Christian Hafner, Florian Rist, Michael Birsak, Michael Wimmer, and Leif Kobbelt. 2016. Non-linear shape optimization using local subspace projections. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–13.Google ScholarDigital Library
    44. Vinod Nair and Geoffrey E Hinton. 2009. 3D object recognition with deep belief nets. Advances in neural information processing systems 22 (2009), 1339–1347.Google Scholar
    45. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.Google Scholar
    46. HM Nasir. 2013. A new class of multicomplex algebra with applications. Mathematical Sciences International Research Journals 2, 2 (2013), 163–168.Google Scholar
    47. CUDA Nvidia. 2008. Cublas library. NVIDIA Corporation, Santa Clara, California 15, 27 (2008), 31.Google Scholar
    48. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019).Google Scholar
    49. Alex Pentland and John Williams. 1989. Good vibrations: Modal dynamics for graphics and animation. In Proceedings of the 16th annual conference on Computer graphics and interactive techniques. 215–222.Google ScholarDigital Library
    50. Eric Pesheck, Nicolas Boivin, Christophe Pierre, and Steven W Shaw. 2001. Nonlinear modal analysis of structural systems using multi-mode invariant manifolds. Nonlinear Dynamics 25, 1–3 (2001), 183–205.Google ScholarCross Ref
    51. Salah Rifai, Grégoire Mesnil, Pascal Vincent, Xavier Muller, Yoshua Bengio, Yann Dauphin, and Xavier Glorot. 2011a. Higher order contractive auto-encoder. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 645–660.Google ScholarDigital Library
    52. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. 2011b. Contractive auto-encoders: Explicit invariance during feature extraction. In Icml.Google Scholar
    53. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211–252.Google Scholar
    54. Sangriyadi Setio, Herlien D Setio, and Louis Jezequel. 1992. Modal analysis of nonlinear multi-degree-of-freedom structures. IJAEM 7, 2 (1992), 75–93.Google Scholar
    55. Ahmed A Shabana. 2003. Dynamics of multibody systems. Cambridge university press.Google Scholar
    56. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 conference on empirical methods in natural language processing. 151–161.Google Scholar
    57. Rasmus Tamstorf, Toby Jones, and Stephen F McCormick. 2015. Smoothed aggregation multigrid for cloth simulation. ACM Trans. Graph. (TOG) 34, 6 (2015), 245.Google ScholarDigital Library
    58. Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. 2020. Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv:2006.10739 (2020).Google Scholar
    59. Adrien Treuille, Andrew Lewis, and Zoran Popović. 2006. Model reduction for real-time fluids. ACM Transactions on Graphics (TOG) 25, 3 (2006), 826–834.Google ScholarDigital Library
    60. Christoph Von-Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2015. Real-time nonlinear shape interpolation. ACM Transactions on Graphics (TOG) 34, 3 (2015), 1–10.Google ScholarDigital Library
    61. Huamin Wang, James F O’Brien, and Ravi Ramamoorthi. 2011. Data-driven elastic models for cloth: modeling and measurement. ACM transactions on graphics (TOG) 30, 4 (2011), 1–12.Google ScholarDigital Library
    62. Huamin Wang and Yin Yang. 2016. Descent methods for elastic body simulation on the GPU. ACM Trans. Graph. (TOG) 35, 6 (2016), 212.Google ScholarDigital Library
    63. Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear subspace design for real-time shape deformation. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1–11.Google ScholarDigital Library
    64. Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 71–82.Google Scholar
    65. Stephen Wolfram et al. 1999. The MATHEMATICA® book, version 4. Cambridge university press.Google Scholar
    66. Xiaofeng Wu, Rajaditya Mukherjee, and Huamin Wang. 2015. A unified approach for subspace simulation of deformable bodies in multiple domains. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–9.Google ScholarDigital Library
    67. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2020).Google Scholar
    68. Hongyi Xu and Jernej Barbič. 2016. Pose-space subspace dynamics. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–14.Google ScholarDigital Library
    69. Hongyi Xu, Yijing Li, Yong Chen, and Jernej Barbič. 2015. Interactive material design using model reduction. ACM Transactions on Graphics (TOG) 34, 2 (2015), 1–14.Google ScholarDigital Library
    70. Yin Yang, Dingzeyu Li, Weiwei Xu, Yuan Tian, and Changxi Zheng. 2015. Expediting precomputation for reduced deformable simulation. ACM Trans. Graph. (TOG) 34, 6 (2015).Google ScholarDigital Library
    71. Yin Yang, Weiwei Xu, Xiaohu Guo, Kun Zhou, and Baining Guo. 2013. Boundary-aware multidomain subspace deformation. IEEE transactions on visualization and computer graphics 19, 10 (2013), 1633–1645.Google ScholarCross Ref
    72. Nianyin Zeng, Hong Zhang, Baoye Song, Weibo Liu, Yurong Li, and Abdullah M Dobaie. 2018. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273 (2018), 643–649.Google ScholarCross Ref
    73. Jiayi Eris Zhang, Seungbae Bang, David I.W. Levin, and Alec Jacobson. 2020. Complementary Dynamics. ACM Transactions on Graphics (2020).Google Scholar
    74. Yongning Zhu, Eftychios Sifakis, Joseph Teran, and Achi Brandt. 2010. An efficient multigrid method for the simulation of high-resolution elastic solids. ACM Trans. Graph. (TOG) 29, 2 (2010), 16.Google ScholarDigital Library
    75. Olgierd Cecil Zienkiewicz, Robert Leroy Taylor, Olgierd Cecil Zienkiewicz, and Robert Lee Taylor. 1977. The finite element method. Vol. 36. McGraw-hill London.Google Scholar

ACM Digital Library Publication:

Overview Page: