“Efficient estimation of boundary integrals for path-space differentiable rendering” by Yan, Lassner, Budge, Dong and Zhao

Boundary integrals are unique to physics-based differentiable rendering and crucial for differentiating with respect to object geometry. Under the differential path integral framework—which has enabled the development of sophisticated differentiable rendering algorithms—the boundary components are themselves path integrals. Previously, although the mathematical formulation of boundary path integrals have been established, efficient estimation of these integrals remains challenging.In this paper, we introduce a new technique to efficiently estimate boundary path integrals. A key component of our technique is a primary-sample-space guiding step for importance sampling of boundary segments. Additionally, we show multiple importance sampling can be used to combine multiple guided samplings. Lastly, we introduce an optional edge sorting step to further improve the runtime performance. We evaluate the effectiveness of our method using several differentiable-rendering and inverse-rendering examples and provide comparisons with existing methods for reconstruction as well as gradient quality.

1. Michael Ashikhmin and Peter Shirley. 2000. An anisotropic phong BRDF model. Journal of graphics tools 5, 2 (2000), 25–32.Google ScholarDigital Library
2. Sai Praveen Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6 (2020), 245:1–245:18.Google ScholarDigital Library
3. Harry G Barrow, Jay M Tenenbaum, Robert C Bolles, and Helen C Wolf. 1977. Parametric correspondence and chamfer matching: Two new techniques for image matching. Technical Report. SRI INTERNATIONAL MENLO PARK CA ARTIFICIAL INTELLIGENCE CENTER.Google Scholar
4. Robert L Cook and Kenneth E. Torrance. 1982. A reflectance model for computer graphics. ACM Transactions on Graphics (ToG) 1, 1 (1982), 7–24.Google ScholarDigital Library
5. Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Appearance-from-motion: Recovering spatially varying surface reflectance under unknown lighting. ACM Trans. Graph. 33, 6 (2014), 1–12.Google ScholarDigital Library
6. Jerry Guo, Pablo Bauszat, Jacco Bikker, and Elmar Eisemann. 2018. Primary sample space path guiding. In Eurographics Symposium on Rendering, Vol. 2018. 73–82.Google Scholar
7. Eric Heitz and Eugene d’Eon. 2014. Importance sampling microfacet-based BSDFs using the distribution of visible normals. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 103–112.Google Scholar
8. Eric Heitz, Johannes Hanika, Eugene d’Eon, and Carsten Dachsbacher. 2016. Multiple-scattering microfacet BSDFs with the Smith model. ACM Trans. Graph. 35, 4 (2016), 58:1–58:14.Google ScholarDigital Library
9. Wenzel Jakob, Marco Tarini, Daniele Panozzo, and Olga Sorkine-Hornung. 2015. Instant Field-Aligned Meshes. ACM Trans. Graph. 34, 6 (2015), 189:1–189:15.Google ScholarDigital Library
10. Henrik Wann Jensen. 1995. Importance driven path tracing using the photon map. In Eurographics Workshop on Rendering Techniques. Springer, 326–335.Google ScholarCross Ref
11. Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs. 2014. Large scale multi-view stereopsis evaluation. In CVPR. 406–413.Google Scholar
12. Csaba Kelemen and Laszlo Szirmay-Kalos. 2001. A microfacet based coupled specular-matte BRDF model with importance sampling. In Eurographics short presentations, Vol. 2. 4.Google Scholar
13. Eric P Lafortune and Yves D Willems. 1995. A 5D tree to reduce the variance of Monte Carlo ray tracing. In Eurographics Workshop on Rendering Techniques. Springer, 11–20.Google ScholarCross Ref
14. Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular Primitives for High-Performance Differentiable Rendering. ACM Trans. Graph. 39, 6 (2020), 194:1–194:14.Google ScholarDigital Library
15. Joo Ho Lee, Adrian Jarabo, Daniel S. Jeon, Diego Gutierrez, and Min H. Kim. 2018. Practical multiple scattering for rough surfaces. ACM Trans. Graph. 37, 6 (2018), 275:1–275:12.Google ScholarDigital Library
16. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 222:1–222:11.Google ScholarDigital Library
17. Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing discontinuous integrands for differentiable rendering. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–14.Google ScholarDigital Library
18. Thomas Müller, Markus Gross, and Jan Novák. 2017. Practical path guiding for efficient light-transport simulation. In Computer Graphics Forum, Vol. 36. 91–100.Google ScholarDigital Library
19. Thomas Müller, Brian Mcwilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. 2019. Neural Importance Sampling. ACM Trans. Graph. 38, 5 (2019), 145:1–145:19.Google ScholarDigital Library
20. Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large Steps in Inverse Rendering of Geometry. ACM Trans. Graph. 40, 6 (2021), 248:1–248:13.Google ScholarDigital Library
21. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: a retargetable forward and inverse renderer. ACM Transactions on Graphics (TOG) 38, 6 (2019), 203.Google ScholarDigital Library
22. Michael Oren and Shree K Nayar. 1994. Generalization of Lambert’s reflectance model. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 239–246.Google ScholarDigital Library
23. Bui Tuong Phong. 1975. Illumination for computer generated pictures. Commun. ACM 18, 6 (1975), 311–317.Google ScholarDigital Library
24. Sylvia C Pont and Jan J Koenderink. 2002. Bidirectional reflectance distribution function of specular surfaces with hemispherical pits. JOSA A 19, 12 (2002), 2456–2466.Google ScholarCross Ref
25. Christophe Schlick. 1994. An inexpensive BRDF model for physically-based rendering. In Computer graphics forum, Vol. 13. Wiley Online Library, 233–246.Google Scholar
26. Bram van Ginneken, Marigo Stavridi, and Jan J Koenderink. 1998. Diffuse and specular reflectance from rough surfaces. Applied optics 37, 1 (1998), 130–139.Google Scholar
27. E. Veach. 1997. Robust Monte Carlo methods for light transport simulation. Vol. 1610. Stanford University PhD thesis.Google ScholarDigital Library
28. Jiří Vorba, Ondřej Karlík, Martin Šik, Tobias Ritschel, and Jaroslav Křivánek. 2014. On-Line Learning of Parametric Mixture Models for Light Transport Simulation. ACM Trans. Graph. 33, 4 (2014), 101:1–101:11.Google ScholarDigital Library
29. Bruce Walter, Stephen R Marschner, Hongsong Li, and Kenneth E Torrance. 2007. Microfacet models for refraction through rough surfaces. Rendering techniques 2007 (2007), 18th.Google Scholar
30. Gregory J Ward. 1992. Measuring and modeling anisotropic reflection. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques. 265–272.Google ScholarDigital Library
31. Feng Xie and Pat Hanrahan. 2018. Multiple scattering from distributions of specular V-grooves. ACM Trans. Graph. 37, 6 (2018), 276:1–2767:14.Google ScholarDigital Library
32. Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. ACM Trans. Graph. 40, 4 (2021), 78:1–78:16.Google ScholarDigital Library
33. Cheng Zhang, Zhao Dong, Michael Doggett, and Shuang Zhao. 2021a. Antithetic Sampling for Monte Carlo Differentiable Rendering. ACM Trans. Graph. 40, 4 (2021), 77:1–77:12.Google ScholarDigital Library
34. C. Zhang, B. Miller, K. Yan, I. Gkioulekas, and S. Zhao. 2020. Path-Space Differentiable Rendering. ACM Transactions on Graphics (SIGGRAPH 2020) 39, 4 (2020), 143:1–143:19.Google Scholar
35. Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shaung Zhao. 2019. A differential theory of radiative transfer. ACM Trans. Graph. 38, 6 (2019), 227:1–227:16.Google ScholarDigital Library
36. Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021b. Path-Space Differentiable Rendering of Participating Media. ACM Trans. Graph. 40, 4 (2021), 76:1–76:15.Google ScholarDigital Library
37. Quan Zheng and Matthias Zwicker. 2019. Learning to importance sample in primary sample space. In Computer Graphics Forum, Vol. 38. 169–179.Google ScholarCross Ref
38. Yang Zhou, Lifan Wu, Ravi Ramamoorthi, and Ling-Qi Yan. 2021. Vectorization for Fast, Analytic, and Differentiable Visibility. ACM Trans. Graph. 40, 3 (2021), 27:1–27:21.Google ScholarDigital Library