“Conditional Mixture Path Guiding for Differentiable Rendering”
Conference:
Type(s):
Title:
- Conditional Mixture Path Guiding for Differentiable Rendering
Presenter(s)/Author(s):
Abstract:
This work develops a path guiding based on a distribution mixture that improves the performance of differentiable rendering processes. It demonstrates why the mixture is required, how to obtain this distribution mixture, how to update each distribution at each iteration, and how to handle negative gradients.
References:
[1]
Dejan Azinovic, Tzu-Mao Li, Anton Kaplanyan, and Matthias Niessner. 2019. Inverse Path Tracing for Joint Material and Lighting Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[2]
Martin Balint, Karol Myszkowski, Hans-Peter Seidel, and Gurprit Singh. 2023. Joint Sampling and Optimisation for Inverse Rendering. In SIGGRAPH Asia 2023 Conference Papers (SA ’23). Association for Computing Machinery, New York, NY, USA, Article 29, 10 pages.
[3]
Sai Praveen Bangaru, Michael Gharbi, Fujun Luan, Tzu-Mao Li, Kalyan Sunkavalli, Milos Hasan, Sai Bi, Zexiang Xu, Gilbert Bernstein, and Fredo Durand. 2022. Differentiable Rendering of Neural SDFs through Reparameterization. In SIGGRAPH Asia 2022 Conference Papers (SA ’22). Association for Computing Machinery, New York, NY, USA.
[4]
Sai Praveen Bangaru, Tzu-Mao Li, and Fr?do Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Transactions on Graphics 39, 6 (Nov. 2020), 245:1–245:18.
[5]
Yash Belhe, Bing Xu, Sai Praveen Bangaru, Ravi Ramamoorthi, and Tzu-Mao Li. 2024. Importance Sampling BRDF Derivatives. ACM Trans. Graph. (feb 2024). Just Accepted.
[6]
Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting. ACM Transactions on Graphics 39, 4 (Aug. 2020).
[7]
Wesley Chang, Venkataram Sivaram, Derek Nowrouzezahrai, Toshiya Hachisuka, Ravi Ramamoorthi, and Tzu-Mao Li. 2023. Parameter-space ReSTIR for Differentiable and Inverse Rendering. In ACM SIGGRAPH 2023 Conference Proceedings (SIGGRAPH ’23). Association for Computing Machinery, New York, NY, USA, Article 18, 10 pages.
[8]
Ana Dodik, Marios Papas, Cengiz ?ztireli, and Thomas M?ller. 2022. Path Guiding Using Spatio-Directional Mixture Models. Computer Graphics Forum 41, 1 (Feb. 2022), 172–189.
[9]
Honghao Dong, Guoping Wang, and Sheng Li. 2023. Neural Parametric Mixtures for Path Guiding. In ACM SIGGRAPH 2023 Conference Proceedings (SIGGRAPH ’23). Association for Computing Machinery, New York, NY, USA, Article 29, 10 pages.
[10]
Jerry Jinfeng Guo, Pablo Bauszat, Jacco Bikker, and Elmar Eisemann. 2018. Primary sample space path guiding. In Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations (Karlsruhe, Germany) (SR ’18). Eurographics Association, Goslar, DEU, 73–82.
[11]
Ayoub El Hanchi and David A. Stephens. 2021. Stochastic Reweighted Gradient Descent. In International Conference on Machine Learning. https://api.semanticscholar.org/CorpusID:232320705
[12]
Sebastian Herholz, Oskar Elek, Ji?? Vorba, Hendrik Lensch, and Jaroslav K?iv?nek. 2016. Product Importance Sampling for Light Transport Path Guiding. Computer Graphics Forum 35, 4 (2016), 67–77.
[13]
Wenzel Jakob, S?bastien Speierer, Nicolas Roussel, and Delio Vicini. 2022. DR.JIT: A Just-in-Time Compiler for Differentiable Rendering. ACM Transactions on Graphics 41, 4 (July 2022), 124:1–124:19.
[14]
James T. Kajiya. 1986. The Rendering Equation. Proceedings of the 13th annual conference on Computer graphics and interactive techniques – SIGGRAPH ’86 (1986), 143–150.
[15]
Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. 2020. Differentiable Rendering: A Survey. arXiv:2006.12057 [cs]
[16]
Ivo Kondapaneni, Petr Vevoda, Pascal Grittmann, Tom?? Sk?ivan, Philipp Slusallek, and Jaroslav K?iv?nek. 2019. Optimal Multiple Importance Sampling. ACM Transactions on Graphics 38, 4 (Aug. 2019), 1–14.
[17]
Tzu-Mao Li. 2022. Differentiable Visual Computing: Challenges and Opportunities. IEEE Computer Graphics and Applications 42, 2 (March 2022), 101–109.
[18]
Tzu-Mao Li, Miika Aittala, Fr?do Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Transactions on Graphics 37, 6 (Dec. 2018), 222:1–222:11.
[19]
F. Llorente, L. Martino, J. Read, and D. Delgado. 2021. A Survey of Monte Carlo Methods for Noisy and Costly Densities with Application to Reinforcement Learning. arXiv:2108.00490 [cs, stat]
[20]
Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing Discontinuous Integrands for Differentiable Rendering. ACM Transactions on Graphics 38, 6 (Nov. 2019), 228:1–228:14.
[21]
Fujun Luan, Shuang Zhao, Kavita Bala, and Zhao Dong. 2021. Unified Shape and SVBRDF Recovery Using Differentiable Monte Carlo Rendering. Computer Graphics Forum 40, 4 (2021), 101–113.
[22]
Thomas M?ller. 2019. “Practical Path Guiding” in Production. In ACM SIGGRAPH Courses: Path Guiding in Production, Chapter 10 (Los Angeles, California). ACM, New York, NY, USA, 18:35–18:48.
[23]
Thomas M?ller, Markus Gross, and Jan Nov?k. 2017. Practical Path Guiding for Efficient Light-Transport Simulation. Computer Graphics Forum 36, 4 (July 2017), 91–100.
[24]
Thomas M?ller, Brian Mcwilliams, Fabrice Rousselle, Markus Gross, and Jan Nov?k. 2019. Neural Importance Sampling. ACM Transactions on Graphics 38, 5 (Oct. 2019), 1–19.
[25]
Baptiste Nicolet, Fabrice Rousselle, Jan Novak, Alexander Keller, Wenzel Jakob, and Thomas M?ller. 2023. Recursive Control Variates for Inverse Rendering. ACM Trans. Graph. 42, 4, Article 62 (jul 2023), 13 pages.
[26]
Merlin Nimier-David, Thomas M?ller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased Inverse Volume Rendering with Differential Trackers. ACM Transactions on Graphics 41, 4 (July 2022), 1–20.
[27]
Merlin Nimier-David, S?bastien Speierer, Beno?t Ruiz, and Wenzel Jakob. 2020. Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering. ACM Transactions on Graphics 39, 4 (Aug. 2020), 146:146:1–146:146:15.
[28]
Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A Retargetable Forward and Inverse Renderer. Transactions on Graphics (Proceedings of SIGGRAPH Asia) 38, 6 (Dec. 2019).
[29]
Art Owen and Yi Zhou. 2000. Safe and Effective Importance Sampling. J. Amer. Statist. Assoc. 95, 449 (March 2000), 135–143.
[30]
Alexander Rath, Pascal Grittmann, Sebastian Herholz, Petr V?voda, Philipp Slusallek, and Jaroslav K?iv?nek. 2020. Variance-Aware Path Guiding. ACM Transactions on Graphics 39, 4 (Aug. 2020).
[31]
Florian Reibold, Johannes Hanika, Alisa Jung, and Carsten Dachsbacher. 2018. Selective Guided Sampling with Complete Light Transport Paths. ACM Transactions on Graphics 37, 6 (Dec. 2018), 1–14.
[32]
Lukas Ruppert, Sebastian Herholz, and Hendrik P. A. Lensch. 2020. Robust Fitting of Parallax-Aware Mixtures for Path Guiding. ACM Transactions on Graphics 39, 4 (Aug. 2020).
[33]
Vincent Sch??ler, Johannes Hanika, Alisa Jung, and Carsten Dachsbacher. 2022. Path Guiding with Vertex Triplet Distributions. Computer Graphics Forum 41, 4 (July 2022), 1–15.
[34]
Sebastian U. Stich, Anant Raj, and Martin Jaggi. 2017. Safe Adaptive Importance Sampling. arXiv:1711.02637 [cs, math]
[35]
Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation.
[36]
Delio Vicini, S?bastien Speierer, and Wenzel Jakob. 2021. Path Replay Backpropagation: Differentiating Light Paths Using Constant Memory and Linear Time. ACM Transactions on Graphics 40, 4 (Aug. 2021), 1–14.
[37]
Delio Vicini, S?bastien Speierer, and Wenzel Jakob. 2022. Differentiable Signed Distance Function Rendering. ACM Transactions on Graphics 41, 4 (July 2022), 125:1–125:18.
[38]
Ji?? Vorba, Johannes Hanika, Sebastian Herholz, Thomas M?ller, Jaroslav K?iv?nek, and Alexander Keller. 2019. Path Guiding in Production. In ACM SIGGRAPH 2019 Courses. ACM, Los Angeles California, 1–77.
[39]
Ji?? Vorba, Ond?ej Karl?k, Martin ?ik, Tobias Ritschel, and Jaroslav K?iv?nek. 2014. On-Line Learning of Parametric Mixture Models for Light Transport Simulation. ACM Transactions on Graphics 33, 4 (July 2014), 1–11.
[40]
Yu-Chen Wang, Chris Wyman, Lifan Wu, and Shuang Zhao. 2023. Amortizing Samples in Physics-Based Inverse Rendering Using ReSTIR. ACM Trans. Graph. 42, 6, Article 214 (dec 2023), 17 pages.
[41]
Kai Yan, Christoph Lassner, Brian Budge, Zhao Dong, and Shuang Zhao. 2022. Efficient Estimation of Boundary Integrals for Path-Space Differentiable Rendering. ACM Transactions on Graphics 41, 4 (July 2022), 123:1–123:13.
[42]
Tizian Zeltner, S?bastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo estimators for differential light transport. ACM Trans. Graph. 40, 4, Article 78 (jul 2021), 16 pages.
[43]
Cheng Zhang, Zhao Dong, Michael Doggett, and Shuang Zhao. 2021a. Antithetic Sampling for Monte Carlo Differentiable Rendering. ACM Transactions on Graphics 40, 4 (July 2021), 77:1–77:12.
[44]
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-Space Differentiable Rendering. ACM Transactions on Graphics 39, 4 (Aug. 2020), 143:143:1–143:143:19.
[45]
Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A Differential Theory of Radiative Transfer. ACM Transactions on Graphics 38, 6 (Nov. 2019), 227:1–227:16.
[46]
Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021b. Path-Space Differentiable Rendering of Participating Media. ACM Transactions on Graphics 40, 4 (July 2021), 76:1–76:15.
[47]
Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective Sampling for Differentiable Rendering of Geometry. ACM Trans. Graph. 42, 6, Article 212 (dec 2023), 14 pages.
[48]
Shuang Zhao, Wenzel Jakob, and Tzu-Mao Li. 2020. Physics-Based Differentiable Rendering: From Theory to Implementation. In ACM SIGGRAPH 2020 Courses. ACM, Virtual Event USA, 1–30.
[49]
Quan Zheng and Matthias Zwicker. 2019. Learning to Importance Sample in Primary Sample Space. Computer Graphics Forum 38, 2 (May 2019), 169–179.
[50]
Yang Zhou, Lifan Wu, Ravi Ramamoorthi, and Ling-Qi Yan. 2021. Vectorization for Fast, Analytic, and Differentiable Visibility. ACM Transactions on Graphics 40, 3 (July 2021), 27:1–27:21.
[51]
Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. 2022. Learning-Based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing. In SIGGRAPH Asia 2022 Conference Papers (SA ’22). Association for Computing Machinery, New York, NY, USA.
[52]
Rong Zhu. 2016. Gradient-Based Sampling: An Adaptive Importance Sampling for Least-squares. arXiv:1803.00841 [cs, stat]
