“Langevin monte carlo rendering with gradient-based adaptation” by Luan, Zhao, Bala and Gkioulekas
Conference:
Type:
Title:
- Langevin monte carlo rendering with gradient-based adaptation
Session/Category Title: Differentiable Rendering and Applications
Presenter(s)/Author(s):
Abstract:
We introduce a suite of Langevin Monte Carlo algorithms for efficient photorealistic rendering of scenes with complex light transport effects, such as caustics, interreflections, and occlusions. Our algorithms operate in primary sample space, and use the Metropolis-adjusted Langevin algorithm (MALA) to generate new samples. Drawing inspiration from state-of-the-art stochastic gradient descent procedures, we combine MALA with adaptive preconditioning and momentum schemes that re-use previously-computed first-order gradients, either in an online or in a cache-driven fashion. This combination allows MALA to adapt to the local geometry of the primary sample space, without the computational overhead associated with previous Hessian-based adaptation algorithms. We use the theory of controlled Markov chain Monte Carlo to ensure that these combinations remain ergodic, and are therefore suitable for unbiased Monte Carlo rendering. Through extensive experiments, we show that our algorithms, MALA with online and cache-driven adaptation, can successfully handle complex light transport in a large variety of scenes, leading to improved performance (on average more than 3× variance reduction at equal time, and 7× for motion blur) compared to state-of-the-art Markov chain Monte Carlo rendering algorithms.
References:
1. Luke Anderson, Tzu-Mao Li, Jaakko Lehtinen, and Frédo Durand. 2017. Aether: An Embedded Domain Specific Sampling Language for Monte Carlo Rendering. ACM TOG (July 2017).Google ScholarDigital Library
2. Christophe Andrieu and Johannes Thoms. 2008. A tutorial on adaptive MCMC. Statistics and computing (2008).Google Scholar
3. James Arvo. 1986. Backward ray tracing. ACM SIGGRAPH Course Notes (1986).Google Scholar
4. James Arvo. 1994. The irradiance Jacobian for partially occluded polyhedral sources. SIGGRAPH (1994).Google Scholar
5. Yves F Atchade. 2006. An adaptive version for the Metropolis adjusted Langevin algorithm with a truncated drift. MCAP (2006).Google Scholar
6. Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, and Matthias Nießner. 2019. Inverse Path Tracing for Joint Material and Lighting Estimation. CVPR (2019).Google Scholar
7. Bradley M Bell. 2003-2015. Cppad: A package for differentiation of C++ algorithms.Google Scholar
8. Michael Betancourt. 2013. A general metric for Riemannian manifold Hamiltonian Monte Carlo. International Conference on Geometric Science of Information (2013).Google ScholarCross Ref
9. Michael Betancourt. 2017. A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint arXiv:1701.02434 (2017).Google Scholar
10. Benedikt Bitterli, Wenzel Jakob, Jan Novák, and Wojciech Jarosz. 2018. Reversible jump metropolis light transport using inverse mappings. ACM TOG (2018).Google Scholar
11. Benedikt Bitterli and Wojciech Jarosz. 2019. Selectively metropolised Monte Carlo light transport simulation. ACM TOG (2019).Google Scholar
12. Jose Luis Blanco and Pranjal Kumar Rai. 2014. nanoflann: a C++ header-only fork of FLANN, a library for Nearest Neighbor (NN) with KD-trees.Google Scholar
13. Ken W Brodlie, AR Gourlay, and John Greenstadt. 1973. Rank-one and rank-two corrections to positive definite matrices expressed in product form. IMA Journal of Applied Mathematics (1973).Google Scholar
14. Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011. Handbook of markov chain monte carlo. CRC press.Google Scholar
15. Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2018. Inverse Transport Networks. arXiv preprint arXiv:1809.10820 (2018).Google Scholar
16. Changyou Chen, David Carlson, Zhe Gan, Chunyuan Li, and Lawrence Carin. 2016. Bridging the gap between stochastic gradient MCMC and stochastic optimization. Artificial Intelligence and Statistics (2016).Google Scholar
17. Min Chen and James Arvo. 2000. Theory and application of specular path perturbation. ACM TOG (2000).Google Scholar
18. Tianqi Chen, Emily Fox, and Carlos Guestrin. 2014. Stochastic gradient hamiltonian monte carlo. ICML (2014).Google Scholar
19. Simon Duane, Anthony D Kennedy, Brian J Pendleton, and Duncan Roweth. 1987. Hybrid monte carlo. Physics letters B (1987).Google Scholar
20. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. JMLR (2011).Google ScholarDigital Library
21. Alain Durmus, Umut Simsekli, Eric Moulines, Roland Badeau, and Gaël Richard. 2016. Stochastic gradient Richardson-Romberg Markov chain Monte Carlo. NeurIPS (2016).Google Scholar
22. Philip Dutre, Philippe Bekaert, and Kavita Bala. 2006. Advanced global illumination. AK Peters/CRC Press.Google Scholar
23. Philip Dutré, Eric P. Lafortune, and Yves D. Willems. 1993. Monte Carlo Light Tracing with Direct Computation of Pixel Intensities. CGVT (1993).Google Scholar
24. Mark Girolami and Ben Calderhead. 2011. Riemann manifold langevin and hamiltonian monte carlo methods. JRSS (2011).Google Scholar
25. Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An evaluation of computational imaging techniques for heterogeneous inverse scattering. ECCV (2016).Google Scholar
26. Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse volume rendering with material dictionaries. ACM TOG (2013).Google Scholar
27. Andreas Griewank and Andrea Walther. 2008. Evaluating derivatives: principles and techniques of algorithmic differentiation. Siam.Google Scholar
28. Toshiya Hachisuka and Henrik Wann Jensen. 2009. Stochastic progressive photon mapping. ACM TOG (2009).Google Scholar
29. Toshiya Hachisuka and Henrik Wann Jensen. 2011. Robust adaptive photon tracing using photon path visibility. ACM TOG (2011).Google Scholar
30. Toshiya Hachisuka, Anton S Kaplanyan, and Carsten Dachsbacher. 2014. Multiplexed metropolis light transport. ACM TOG (2014).Google Scholar
31. Johannes Hanika, Anton Kaplanyan, and Carsten Dachsbacher. 2015. Improved half vector space light transport. In CGF.Google Scholar
32. W Keith Hastings. 1970. Monte Carlo sampling methods using Markov chains and their applications. (1970).Google Scholar
33. Nicolas Holzschuch and François X Sillion. 1995. Accurate computation of the radiosity gradient with constant and linear emitters. EGSR (1995).Google Scholar
34. Homan Igehy. 1999. Tracing ray differentials. SIGGRAPH (1999).Google Scholar
35. Wenzel Jakob. 2019. Enoki: structured vectorization and differentiation on modern processor architectures. https://github.com/mitsuba-renderer/enoki.Google Scholar
36. Wenzel Jakob and Steve Marschner. 2012. Manifold exploration: a Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM TOG (2012).Google ScholarDigital Library
37. Wojciech Jarosz, Craig Donner, Matthias Zwicker, and Henrik Wann Jensen. 2008a. Radiance caching for participating media. ACM TOG (2008).Google Scholar
38. Wojciech Jarosz, Derek Nowrouzezahrai, Robert Thomas, Peter-Pike Sloan, and Matthias Zwicker. 2011. Progressive Photon Beams. ACM TOG (2011).Google Scholar
39. Wojciech Jarosz, Volker Schönefeld, Leif Kobbelt, and Henrik Wann Jensen. 2012. Theory, Analysis and Applications of 2D Global Illumination. ACM TOG (2012).Google Scholar
40. Wojciech Jarosz, Matthias Zwicker, and Henrik Wann Jensen. 2008b. Irradiance Gradients in the Presence of Participating Media and Occlusions. EGSR (2008).Google Scholar
41. Henrik Wann Jensen. 1995. Importance driven path tracing using the photon map. Rendering Techniques (1995).Google Scholar
42. Henrik Wann Jensen. 2001. Realistic image synthesis using photon mapping. AK Peters/CRC Press.Google Scholar
43. James T Kajiya. 1986. The rendering equation. SIGGRAPH (1986).Google Scholar
44. Anton S Kaplanyan, Johannes Hanika, and Carsten Dachsbacher. 2014. The natural-constraint representation of the path space for efficient light transport simulation. ACM TOG (2014).Google Scholar
45. Csaba Kelemen, László Szirmay-Kalos, György Antal, and Ferenc Csonka. 2002. A simple and robust mutation strategy for the metropolis light transport algorithm. CGF (2002).Google Scholar
46. Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2015. Matching Real Fabrics with Micro-Appearance Models. ACM TOG (2015).Google Scholar
47. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
48. Jaroslav Krivánek, Kadi Bouatouch, Sumanta N Pattanaik, and Jiri Zara. 2006. Making Radiance and Irradiance Caching Practical: Adaptive Caching and Neighbor Clamping. Rendering Techniques (2006).Google Scholar
49. Jaroslav Krivanek, Pascal Gautron, Sumanta Pattanaik, and Kadi Bouatouch. 2005. Radiance caching for efficient global illumination computation. IEEE TVCG (2005).Google ScholarDigital Library
50. Jaroslav Křivánek, Iliyan Georgiev, Toshiya Hachisuka, Petr Vévoda, Martin Šik, Derek Nowrouzezahrai, and Wojciech Jarosz. 2014. Unifying Points, Beams, and Paths in Volumetric Light Transport Simulation. ACM TOG (2014).Google Scholar
51. Eric P. Lafortune and Yves D. Willems. 1993. Bi-directional path tracing. Compugraphics (1993).Google Scholar
52. Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-domain metropolis light transport. ACM TOG (2013).Google Scholar
53. Don Lemons and Anthony Gythiel. 1997. Paul Langevin’s 1908 paper “On the Theory of Brownian Motion”. American Journal of Physics (1997).Google Scholar
54. Tzu-Mao Li. 2015. dpt. https://github.com/BachiLi/dpt/.Google Scholar
55. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable monte carlo ray tracing through edge sampling. ACM TOG (2018).Google Scholar
56. Tzu-Mao Li, Jaakko Lehtinen, Ravi Ramamoorthi, Wenzel Jakob, and Frédo Durand. 2015. Anisotropic Gaussian Mutations for Metropolis Light Transport through Hessian-Hamiltonian Dynamics. ACM TOG (2015).Google Scholar
57. Samuel Livingstone and Giacomo Zanella. 2019. On the robustness of gradient-based MCMC algorithms. arXiv preprint arXiv:1908.11812 (2019).Google Scholar
58. Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2020. Project Website. https://research.cs.cornell.edu/langevin-mcmc.Google Scholar
59. Yi-An Ma, Tianqi Chen, and Emily Fox. 2015. A complete recipe for stochastic gradient MCMC. NeurIPS (2015).Google Scholar
60. Julio Marco, Adrian Jarabo, Wojciech Jarosz, and Diego Gutierrez. 2018. Second-Order Occlusion-Aware Volumetric Radiance Caching. ACM TOG (2018).Google Scholar
61. Gisiro Maruyama. 1955. Continuous Markov processes and stochastic equations. Rendiconti del Circolo Matematico di Palermo (1955).Google Scholar
62. Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics (1953).Google ScholarCross Ref
63. Thomas Müller, Markus Gross, and Jan Novák. 2017. Practical path guiding for efficient light-transport simulation. CGF (2017).Google Scholar
64. Thomas Müller, Brian Mcwilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. 2019. Neural Importance Sampling. ACM TOG (2019).Google Scholar
65. Radford M Neal et al. 2011. MCMC using Hamiltonian dynamics. Handbook of markov chain monte carlo (2011).Google Scholar
66. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: a retargetable forward and inverse renderer. ACM TOG (2019).Google ScholarDigital Library
67. Jorge Nocedal and Stephen Wright. 2006. Numerical optimization. Springer.Google Scholar
68. Bernt Øksendal. 2003. Stochastic differential equations. In Stochastic differential equations. Springer.Google Scholar
69. Hisanari Otsu, Johannes Hanika, Toshiya Hachisuka, and Carsten Dachsbacher. 2018. Geometry-Aware Metropolis Light Transport. ACM TOG (2018).Google Scholar
70. Hisanari Otsu, Anton S. Kaplanyan, Johannes Hanika, Carsten Dachsbacher, and Toshiya Hachisuka. 2017. Fusing State Spaces for Markov Chain Monte Carlo Rendering. ACM TOG (2017).Google Scholar
71. Jacopo Pantaleoni. 2017. Charted Metropolis Light Transport. ACM TOG (2017).Google Scholar
72. Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically based rendering: From theory to implementation. Morgan Kaufmann.Google Scholar
73. Boris T Polyak. 1964. Some methods of speeding up the convergence of iteration methods. U. S. S. R. Comput. Math. and Math. Phys. (1964).Google Scholar
74. Ravi Ramamoorthi, Dhruv Mahajan, and Peter Belhumeur. 2007. A First-Order Analysis of Lighting, Shading, and Shadows. ACM TOG (2007).Google Scholar
75. Florian Reibold, Johannes Hanika, Alisa Jung, and Carsten Dachsbacher. 2019. Selective Guided Sampling with Complete Light Transport Paths. ACM TOG (2019).Google Scholar
76. Gareth O Roberts and Jeffrey S Rosenthal. 2007. Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. Journal of applied probability (2007).Google Scholar
77. Gareth O Roberts and Jeffrey S Rosenthal. 2009. Examples of adaptive MCMC. Journal of Computational and Graphical Statistics (2009).Google Scholar
78. Gareth O Roberts and Osnat Stramer. 2002. Langevin diffusions and Metropolis-Hastings algorithms. Methodology and computing in applied probability (2002).Google Scholar
79. Gareth O Roberts and Richard L Tweedie. 1996. Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli (1996).Google Scholar
80. Jorge Schwarzhaupt, Henrik Wann Jensen, and Wojciech Jarosz. 2012. Practical Hessian-Based Error Control for Irradiance Caching. ACM TOG (2012).Google Scholar
81. Martin Šik, Hisanari Otsu, Toshiya Hachisuka, and Jaroslav Křivánek. 2016. Robust light transport simulation via metropolised bidirectional estimators. ACM TOG (2016).Google Scholar
82. Umut Simsekli, Roland Badeau, Taylan Cemgil, and Gaël Richard. 2016. Stochastic quasi-newton langevin monte carlo. ICML (2016).Google Scholar
83. Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the importance of initialization and momentum in deep learning. ICML (2013).Google Scholar
84. Frank Suykens and Yves D Willems. 2001. Path differentials and applications. Rendering Techniques (2001).Google Scholar
85. Chia-Yin Tsai, Aswin C Sankaranarayanan, and Ioannis Gkioulekas. 2019. Beyond Volumetric Albedo-A Surface Optimization Framework for Non-Line-Of-Sight Imaging. CVPR (2019).Google Scholar
86. Eric Veach. 1997. Robust Monte Carlo methods for light transport simulation. Stanford University PhD thesis.Google ScholarDigital Library
87. Eric Veach and Leonidas Guibas. 1995. Bidirectional estimators for light transport. Photorealistic Rendering Techniques (1995).Google Scholar
88. Eric Veach and Leonidas J Guibas. 1997. Metropolis light transport. SIGGRAPH (1997).Google Scholar
89. Jiří Vorba, Ondřej Karlík, Martin Šik, Tobias Ritschel, and Jaroslav Křivánek. 2014. On-Line Learning of Parametric Mixture Models for Light Transport Simulation. ACM TOG (2014).Google Scholar
90. Gregory J Ward and Paul S Heckbert. 1992. Irradiance gradients. Technical Report.Google Scholar
91. Gregory J Ward, Francis M Rubinstein, and Robert D Clear. 1988. A ray tracing solution for diffuse interreflection. SIGGRAPH (1988).Google Scholar
92. Max Welling and Yee W Teh. 2011. Bayesian learning via stochastic gradient Langevin dynamics. ICML (2011).Google Scholar
93. Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A differential theory of radiative transfer. ACM TOG (2019).Google Scholar
94. Yichuan Zhang and Charles A. Sutton. 2011. Quasi-Newton Methods for Markov Chain Monte Carlo. NeurIPS (2011).Google Scholar
95. Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling scattering parameters for rendering anisotropic media. ACM TOG (2016).Google Scholar
96. Quan Zheng and Matthias Zwicker. 2019. Learning to importance sample in primary sample space. CGF (2019).Google Scholar
