“Path-space differentiable rendering of participating media” by Zhang, Yu and Zhao

Physics-based differentiable rendering—which focuses on estimating derivatives of radiometric detector responses with respect to arbitrary scene parameters—has a diverse array of applications from solving analysis-by-synthesis problems to training machine-learning pipelines incorporating forward-rendering processes. Unfortunately, existing general-purpose differentiable rendering techniques lack either the generality to handle volumetric light transport or the flexibility to devise Monte Carlo estimators capable of handling complex geometries and light transport effects.In this paper, we bridge this gap by showing how generalized path integrals can be differentiated with respect to arbitrary scene parameters. Specifically, we establish the mathematical formulation of generalized differential path integrals that capture both interfacial and volumetric light transport. Our formulation allows the development of advanced differentiable rendering algorithms capable of efficiently handling challenging geometric discontinuities and light transport phenomena such as volumetric caustics.We validate our method by comparing our derivative estimates to those generated using the finite differences. Further, to demonstrate the effectiveness of our technique, we compare both differentiable rendering and inverse rendering performance with state-of-the-art methods.

1. Sai Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased warped-area sampling for differentiable rendering. ACM Trans. Graph. 39, 6 (2020), 245:1–245:18.Google ScholarDigital Library
2. Eva Cerezo, Frederic Pérez, Xavier Pueyo, Francisco J Seron, and François X Sillion. 2005. A survey on participating media rendering techniques. The Visual Computer 21, 5 (2005), 303–328.Google ScholarDigital Library
3. Subrahmanyan Chandrasekhar. 1960. Radiative transfer. Courier Corporation.Google Scholar
4. Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2020. Towards Learning-based Inverse Subsurface Scattering. In 2020 IEEE International Conference on Computational Photography (ICCP). IEEE, 1–12.Google Scholar
5. Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse volume rendering with material dictionaries. ACM Trans. Graph. 32, 6 (2013), 162:1–162:13.Google ScholarDigital Library
6. John Michael Hammersley and JG Mauldon. 1956. General principles of antithetic variates. In Mathematical proceedings of the Cambridge philosophical society, Vol. 52. Cambridge University Press, 476–481.Google Scholar
7. Wenzel Jakob and Steve Marschner. 2012a. Manifold Exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Trans.Graph. 31, 4 (2012), 58:1–58:13 pages.Google ScholarDigital Library
8. Wenzel Jakob and Steve Marschner. 2012b. Manifold exploration: A Markov chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Trans. Graph. 31, 4 (2012), 58:1–58:13.Google ScholarDigital Library
9. James T. Kajiya. 1986. The Rendering Equation. In SIGGRAPH ’86. 143–150.Google Scholar
10. James T. Kajiya and Brian P Von Herzen. 1984. Ray tracing volume densities. SIGGRAPH Comput. Graph. 18, 3 (1984), 165–174.Google ScholarDigital Library
11. Csaba Kelemen, László Szirmay-Kalos, György Antal, and Ferenc Csonka. 2002. A simple and robust mutation strategy for the metropolis light transport algorithm. In Computer Graphics Forum, Vol. 21. Wiley Online Library, 531–540.Google Scholar
12. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
13. Eric P Lafortune and Yves D Willems. 1996. Rendering participating media with bidirectional path tracing. In Rendering techniques’ 96. Springer, 91–100.Google ScholarDigital Library
14. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 222:1–222:11.Google ScholarDigital Library
15. Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing discontinuous integrands for differentiable rendering. ACM Trans. Graph. 38, 6 (2019), 228:1–228:14.Google ScholarDigital Library
16. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A retargetable forward and inverse renderer. ACM Trans. Graph. 38, 6 (2019), 203:1–203:17.Google ScholarDigital Library
17. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo methods for volumetric light transport simulation. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 551–576.Google Scholar
18. Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. In Rendering Techniques 2000. Springer, 11–22.Google ScholarCross Ref
19. Osborne Reynolds. 1903. Papers on mechanical and physical subjects: the sub-mechanics of the universe. Vol. 3. The University Press.Google Scholar
20. Hang Si. 2015. TetGen, a Delaunay-based quality tetrahedral mesh generator. ACM Transactions on Mathematical Software (TOMS) 41, 2 (2015), 1–36.Google ScholarDigital Library
21. Jerome Spanier and Ely M Gelbard. 1969. Monte Carlo principles and neutron transport problems. The Addison-Wesley Publishing Company.Google Scholar
22. Denis Sumin, Tobias Rittig, Vahid Babaei, Thomas Nindel, Alexander Wilkie, Piotr Didyk, Bernd Bickel, Jaroslav Křivánek, Karol Myszkowski, and Tim Weyrich. 2019. Geometry-aware scattering compensation for 3D printing. ACM Trans. Graph. 38, 4 (2019), 111:1–111:14.Google ScholarDigital Library
23. Jean-Marc Thiery, Pooran Memari, and Tamy Boubekeur. 2018. Mean value coordinates for quad cages in 3D. ACM Trans. Graph. 37, 6 (2018), 229:1–229:14.Google ScholarDigital Library
24. Eric Veach. 1997. Robust Monte Carlo methods for light transport simulation. Vol. 1610. Stanford University PhD thesis.Google ScholarDigital Library
25. Eric Veach and Leonidas Guibas. 1995. Bidirectional estimators for light transport. In Photorealistic Rendering Techniques. Springer, 145–167.Google Scholar
26. Eric Veach and Leonidas J. Guibas. 1997. Metropolis light transport. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., 65–76.Google Scholar
27. E Woodcock, T Murphy, P Hemmings, and S Longworth. 1965. Techniques used in the GEM code for Monte Carlo neutronics calculations in reactors and other systems of complex geometry. In Proc. Conf. Applications of Computing Methods to Reactor Problems, Vol. 557.Google Scholar
28. Cheng Zhang, Zhao Dong, Michael Doggett, and Shuang Zhao. 2021. Antithetic sampling for Monte Carlo differentiable rendering. ACM Trans. Graph. 40, 4 (2021), 77:1–77:12.Google ScholarDigital Library
29. Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-space differentiable rendering. ACM Trans. Graph. 39, 4 (2020), 143:1–143:19.Google ScholarDigital Library
30. Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A differential theory of radiative transfer. ACM Trans. Graph. 38, 6 (2019), 227:1–227:16.Google ScholarDigital Library
31. Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling scattering parameters for rendering anisotropic media. ACM Trans. Graph. 35, 6 (2016), 166:1–166:11.Google ScholarDigital Library