“Efficient Differentiation of Pixel Reconstruction Filters for Path-Space Differentiable Rendering” by Yu, Zhang, Nowrouzezahrai, Dong and Zhao
Conference:
Type(s):
Title:
- Efficient Differentiation of Pixel Reconstruction Filters for Path-Space Differentiable Rendering
Session/Category Title: Differentiable Rendering
Presenter(s)/Author(s):
Abstract:
Pixel reconstruction filters play an important role in physics-based rendering and have been thoroughly studied. In physics-based differentiable rendering, however, the proper treatment of pixel filters remains largely under-explored. We present a new technique to efficiently differentiate pixel reconstruction filters based on the path-space formulation. Specifically, we formulate the pixel boundary integral that models discontinuities in pixel filters and introduce new antithetic sampling methods that support differentiable path sampling methods, such as adjoint particle tracing and bidirectional path tracing. We demonstrate both the need and efficacy of antithetic sampling when estimating this integral, and we evaluate its effectiveness across several differentiable- and inverse-rendering settings.
References:
1. Mahdi M. Bagher, John M. Snyder, and Derek Nowrouzezahrai. 2016. A non-parametric factor microfacet model for isotropic BRDFs. ACM Trans. Graph. 35, 5 (2016), 159:1–159:16.
2. Mahdi Mohammad Bagher, Cyril Soler, Kartic Subr, Laurent Belcour, and Nicolas Holzschuch. 2012. Interactive rendering of acquired materials on dynamic geometry using frequency analysis. IEEE Transactions on Visualization and Computer Graphics 19, 5 (2012), 749–761.
3. Sai Praveen Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6 (2020), 245:1–245:18.
4. Harry G Barrow, Jay M Tenenbaum, Robert C Bolles, and Helen C Wolf. 1977. Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings: Image Understanding Workshop. 21–27.
5. Laurent Belcour, Kavita Bala, and Cyril Soler. 2014. A local frequency analysis of light scattering and absorption. ACM Trans. Graph. 33, 5 (2014), 163:1–163:17.
6. Laurent Belcour, Cyril Soler, Kartic Subr, Nicolas Holzschuch, and Fredo Durand. 2013. 5D covariance tracing for efficient defocus and motion blur. ACM Trans. Graph. 32, 3 (2013), 31:1–31:18.
7. Laurent Belcour, Ling-Qi Yan, Ravi Ramamoorthi, and Derek Nowrouzezahrai. 2017. Antialiasing complex global illumination effects in path-space. ACM Trans. Graph. 36, 1 (2017), 9:1–9:13.
8. Subrahmanyan Chandrasekhar. 1960. Radiative transfer. Courier Corporation.
9. Robert L Cook and Kenneth E. Torrance. 1982. A reflectance model for computer graphics. ACM Transactions on Graphics (ToG) 1, 1 (1982), 7–24.
10. Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Appearance-from-Motion: Recovering Spatially Varying Surface Reflectance under Unknown Lighting. ACM Trans. Graph. 33, 6 (2014), 193:1–193:12.
11. Frédo Durand, Nicolas Holzschuch, Cyril Soler, Eric Chan, and François X. Sillion. 2005. A frequency analysis of light transport. ACM Trans. Graph. 24, 3 (2005), 1115–1126.
12. Kevin Egan, Yu-Ting Tseng, Nicolas Holzschuch, Frédo Durand, and Ravi Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. In ACM SIGGRAPH 2009 Papers (SIGGRAPH ’09). Article 93, 13 pages.
13. Eric Heitz and Eugene d’Eon. 2014. Importance sampling microfacet-based BSDFs using the distribution of visible normals. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 103–112.
14. Eric Heitz, Johannes Hanika, Eugene d’Eon, and Carsten Dachsbacher. 2016. Multiple-scattering microfacet BSDFs with the Smith model. ACM Trans. Graph. 35, 4 (2016), 58:1–58:14.
15. Csaba Kelemen and Laszlo Szirmay-Kalos. 2001. A microfacet based coupled specular-matte BRDF model with importance sampling. In Eurographics short presentations, Vol. 2. 4.
16. Markus Kettunen, Eugene D’Eon, Jacopo Pantaleoni, and Jan Novák. 2021. An unbiased ray-marching transmittance estimator. ACM Trans. Graph. 40, 4 (2021), 137:1–137:20.
17. Joo Ho Lee, Adrian Jarabo, Daniel S. Jeon, Diego Gutierrez, and Min H. Kim. 2018. Practical multiple scattering for rough surfaces. ACM Trans. Graph. 37, 6 (2018), 275:1–275:12.
18. Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-Domain Metropolis Light Transport. ACM Trans. Graph. 32, 4 (2013), 95:1–95:12.
19. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 222:1–222:11.
20. Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing discontinuous integrands for differentiable rendering. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–14.
21. Soham Uday Mehta, Brandon Wang, and Ravi Ramamoorthi. 2012. Axis-aligned filtering for interactive sampled soft shadows. ACM Trans. Graph. 31, 6 (2012), 163:1–163:10.
22. William Moses and Valentin Churavy. 2020. Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 12472–12485.
23. Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large Steps in Inverse Rendering of Geometry. ACM Trans. Graph. 40, 6 (2021), 248:1–248:13.
24. Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. In Rendering Techniques 2000. Springer, 11–22.
25. Christophe Schlick. 1994. An inexpensive BRDF model for physically-based rendering. In Computer graphics forum, Vol. 13. Wiley Online Library, 233–246.
26. Cyril Soler, Kartic Subr, Frédo Durand, Nicolas Holzschuch, and François Sillion. 2009. Fourier depth of field. ACM Trans. Graph. 28, 2 (2009), 18:1–18:12.
27. Bram van Ginneken, Marigo Stavridi, and Jan J Koenderink. 1998. Diffuse and specular reflectance from rough surfaces. Applied optics 37, 1 (1998), 130–139.
28. Eric Veach. 1997. Robust Monte Carlo methods for light transport simulation. Vol. 1610. Stanford University PhD thesis.
29. Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2022. Differentiable Signed Distance Function Rendering. ACM Trans. Graph. 41, 4 (2022), 125:1–125:18.
30. Bruce Walter, Stephen R Marschner, Hongsong Li, and Kenneth E Torrance. 2007. Microfacet models for refraction through rough surfaces. Rendering techniques 2007 (2007), 18th.
31. Gregory J Ward. 1992. Measuring and modeling anisotropic reflection. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques. 265–272.
32. Feng Xie and Pat Hanrahan. 2018. Multiple scattering from distributions of specular V-grooves. ACM Trans. Graph. 37, 6 (2018), 276:1–2767:14.
33. Kai Yan, Christoph Lassner, Brian Budge, Zhao Dong, and Shuang Zhao. 2022. Efficient estimation of boundary integrals for path-space differentiable rendering. ACM Trans. Graph. 41, 4 (2022), 123:1–123:13.
34. Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo estimators for differential light transport. ACM Trans. Graph. 40, 4 (2021), 78:1–78:16.
35. Cheng Zhang, Zhao Dong, Michael Doggett, and Shuang Zhao. 2021a. Antithetic sampling for Monte Carlo differentiable rendering. ACM Trans. Graph. 40, 4 (2021), 77:1–77:12.
36. Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-space differentiable rendering. ACM Trans. Graph. 39, 4 (2020), 143:1–143:19.
37. Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A differential theory of radiative transfer. ACM Trans. Graph. 38, 6 (2019), 227:1–227:16.
38. Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021b. Path-space differentiable rendering of participating media. ACM Trans. Graph. 40, 4 (2021), 76:1–76:15.
39. Zhiming Zhou, Guojun Chen, Yue Dong, David Wipf, Yong Yu, John Snyder, and Xin Tong. 2016. Sparse-as-Possible SVBRDF Acquisition. ACM Trans. Graph. 35, 6 (2016), 189:1–189:12.
40. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer graphics forum 34, 2 (2015), 667–681.
