“Importance Sampling BRDF Derivatives”
Conference:
Type(s):
Title:
- Importance Sampling BRDF Derivatives
Presenter(s)/Author(s):
Abstract:
We propose the first set of techniques to efficiently importance sample the derivatives of several popular analytical BRDF models (e.g. anisotropic GGX/ Beckmann/ Ashikhmin-Shirley, Hanrahan-Kreuger, Oren-Nayar etc.). Our estimators are practical and provide significant variance improvement in all settings from direct illumination to global illumination effects like caustics.
References:
[1]
James Arvo. 1994. The Irradiance Jacobian for partially occluded polyhedral sources. In Proceedings of the SIGGRAPH Conference. 343?350.
[2]
James Arvo and David Kirk. 1990. Particle transport and image synthesis. Comput. Graph. (Proc. SIGGRAPH) 24, 4 (1990), 63?66.
[3]
Michael Ashikhmin and Peter Shirley. 2001. An anisotropic phong BRDF model. J. Graph. Tools 5, 2 (2001), 25?32.
[4]
Dejan Azinovi?, Tzu-Mao Li, Anton Kaplanyan, and Matthias Nie?ner. 2019. Inverse path tracing for joint material and lighting estimation. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 2447?2456.
[5]
Sai Praveen Bangaru, Tzu-Mao Li, and Fr?do Durand. 2020. Unbiased warped-area sampling for differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 39, 6 (2020), 245:1?245:18.
[6]
Sai Praveen Bangaru, Lifan Wu, Tzu-Mao Li, Jacob Munkberg, Gilbert Bernstein, Jonathan Ragan-Kelley, Fr?do Durand, Aaron Lefohn, and Yong He. 2023. SLANG.D: Fast, modular, and differentiable shader programming. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 42, 6, Article 264 (2023).
[7]
Petr Beckmann and Andre Spizzichino. 1987. The Scattering of Electromagnetic Waves from Rough Surfaces. Artech House Publishers, Norwood, MA.
[8]
James F. Blinn. 1977. Models of light reflection for computer synthesized pictures. Comput. Graph. (Proc. SIGGRAPH) 11, 2 (1977), 192?198.
[9]
Brent Burley. 2012. Physically based shading at Disney. In SIGGRAPH Course Notes. Practical Physically Based Shading in Film and Game Production.1?7.
[10]
Brent Burley. 2015. Extending the Disney BRDF to a BSDF with integrated subsurface scattering. In SIGGRAPH Course Notes. Physically Based Shading in Theory and Practice.1?9.
[11]
Wesley Chang, Venkataram Sivaram, Derek Nowrouzezahrai, Toshiya Hachisuka, Ravi Ramamoorthi, and Tzu-Mao Li. 2023. Parameter-space ReSTIR for differentiable and inverse rendering. In Proceedings of the SIGGRAPH Conference. Article 18, 10 pages.
[12]
Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2020. Towards learning-based inverse subsurface scattering. In Proceedings of the International Conference on Computational Photography (ICCP?20).
[13]
Michael Cohen and John Wallace. 1993. Radiosity and Realistic Image Synthesis. Morgan Kaufmann.
[14]
Martin de La Gorce, David J Fleet, and Nikos Paragios. 2011. Model-based 3D hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33, 9 (2011), 1793?1805.
[15]
Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. 2018. Single-image SVBRDF capture with a rendering-aware deep network. ACM Trans. Graph. (Proc. SIGGRAPH) 37, 4 (2018), 128:1?128:15.
[16]
Jonathan Dupuy and Wenzel Jakob. 2018. An adaptive parameterization for efficient material acquisition and rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 274:1?274:18.
[17]
Jiahui Fan, Beibei Wang, Milos Hasan, Jian Yang, and Ling-Qi Yan. 2022. Neural layered BRDFs. In Proceedings of the SIGGRAPH Conference. Article 4, 8 pages.
[18]
Iliyan Georgiev, Jamie Portsmouth, Zap Andersson, Adrien Herubel, Alan King, Shinji Ogaki, and Frederic Servant. 2019. Autodesk standard surface. https://autodesk.github.io/standard-surface/
[19]
Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse volume rendering with material dictionaries. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 32, 6 (2013), 162:1?162:13.
[20]
Pascal Grittmann, ?mercan Yazici, Iliyan Georgiev, and Philipp Slusallek. 2022. Efficiency-aware multiple importance sampling for bidirectional rendering algorithms. ACM Trans. Graph. (Proc. SIGGRAPH) 41, 4, Article 80 (2022).
[21]
Pat Hanrahan and Wolfgang Krueger. 1993. Reflection from layered surfaces due to subsurface scattering. In Proceedings of the SIGGRAPH. 165?174.
[22]
Hera Y. He and Art B. Owen. 2014. Optimal mixture weights in multiple importance sampling. Retrieved from https://arXiv:1411.3954
[23]
Eric Heitz. 2017. A Simpler and Exact Sampling Routine for the GGX Distribution of Visible Normals. Research Report. Unity Technologies.
[24]
Eric Heitz. 2018. Sampling the GGX distribution of visible normals. J. Comput. Graph. Techn. 7, 4 (2018), 1?13.
[25]
Eric Heitz and Eugene d?Eon. 2014. Importance sampling microfacet-based BSDFs using the distribution of visible normals. Comput. Graph. Forum (Proc. EGSR) 33, 4 (2014), 103?112.
[26]
L. G. Henyey and J. L. Greenstein. 1941. Diffuse radiation in the Galaxy. Astrophys. J. 93 (1941), 70?83.
[27]
Wenzel Jakob. 2014. An Improved Visible Normal Sampling Routine for the Beckmann Distribution. Technical Report. Cornell University.
[28]
Wenzel Jakob, S?bastien Speierer, Nicolas Roussel, and Delio Vicini. 2022. Dr.Jit: A just-in-time compiler for differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 41, 4 (2022).
[29]
James T. Kajiya. 1986. The rendering equation. Comput. Graph. (Proc. SIGGRAPH) 20, 4 (1986), 143?150.
[30]
Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Neural 3D mesh renderer. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 3907?3916.
[31]
Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2015. Matching real fabrics with micro-appearance models. ACM Trans. Graph. 35, 1 (2015), 1:1?1:26.
[32]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations.
[33]
Alexandr Kuznetsov, Krishna Mullia, Zexiang Xu, Milo? Ha?an, and Ravi Ramamoorthi. 2021. NeuMIP: Multi-resolution neural materials. ACM Trans. Graph. (Proc. SIGGRAPH) 40, 4, Article 175 (2021), 13 pages.
[34]
Alexandr Kuznetsov, Xuezheng Wang, Krishna Mullia, Fujun Luan, Zexiang Xu, Milo? Ha?an, and Ravi Ramamoorthi. 2022. Rendering neural materials on curved surfaces. In Proceedings of the SIGGRAPH Conference Proceedings. 9 pages.
[35]
Eric P. F. Lafortune, Sing-Choong Foo, Kenneth E. Torrance, and Donald P. Greenberg. 1997. Non-linear approximation of reflectance functions. In Proceedings of the SIGGRAPH. 117?126.
[36]
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular primitives for high-performance differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 39, 6 (2020).
[37]
Jason Lawrence, Szymon Rusinkiewicz, and Ravi Ramamoorthi. 2004. Efficient BRDF importance sampling using a factored representation. ACM Trans. Graph. (Proc. SIGGRAPH) (2004), 496?505.
[38]
Tzu-Mao Li, Miika Aittala, Fr?do Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1?222:11.
[39]
Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. 2020. A general differentiable mesh renderer for image-based 3D reasoning. IEEE Trans. Pattern Anal. Mach. Intell. 44, 1 (2020), 50?62.
[40]
Matthew M. Loper and Michael J. Black. 2014. OpenDR: An approximate differentiable renderer. In Proceedings of the European Conference on Computer Vision, Vol. 8695. 154?169.
[41]
Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing discontinuous integrands for differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (2019).
[42]
Joakim L?w, Joel Kronander, Anders Ynnerman, and Jonas Unger. 2012. BRDF models for accurate and efficient rendering of glossy surfaces. ACM Trans. Graph. 31, 1, Article 9 (2012), 14 pages.
[43]
Fujun Luan, Shuang Zhao, Kavita Bala, and Zhao Dong. 2021. Unified shape and SVBRDF recovery using differentiable Monte Carlo rendering. Comput. Graph. Forum (Proc. EGSR) 40, 4 (2021), 101?113.
[44]
Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003. A data-driven reflectance model. ACM Trans. Graph. (Proc. SIGGRAPH) 22, 3 (2003), 759?769.
[45]
M. Minnaert. 1941. The reciprocity principle in lunar photometry. Astrophys. J. 93 (1941), 403?410.
[46]
Don P. Mitchell. 1996. Consequences of stratified sampling in graphics. In Proceedings of the SIGGRAPH. 277?280.
[47]
Baptiste Nicolet, Fabrice Rousselle, Jan Nov?k, Alexander Keller, Wenzel Jakob, and Thomas M?ller. 2023. Recursive control variates for inverse rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 42, 4 (2023).
[48]
Merlin Nimier-David, Zhao Dong, Wenzel Jakob, and Anton Kaplanyan. 2021. Material and lighting reconstruction for complex indoor scenes with texture-space differentiable rendering. In Proceedings of the Eurographics Symposium on Rendering?DL-only Track.
[49]
Merlin Nimier-David, Thomas M?ller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased inverse volume rendering with differential trackers. ACM Trans. Graph. (Proc. SIGGRAPH) 41, 4 (2022), 44:1?44:20.
[50]
Merlin Nimier-David, S?bastien Speierer, Beno?t Ruiz, and Wenzel Jakob. 2020. Radiative backpropagation: An adjoint method for lightning-fast differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 39, 4 (2020).
[51]
Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A retargetable forward and inverse renderer. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (2019), 1?17.
[52]
Ko Nishino. 2009. Directional statistics BRDF model. In Proceedings of the International Conference on Computer Vision. 476?483.
[53]
Michael Oren and Shree K. Nayar. 1994. Generalization of Lambert?s reflectance model. In Proceedings of the SIGGRAPH. 239?246.
[54]
Art Owen and Yi Zhou. 2000. Safe and effective importance sampling. J. Amer. Statist. Assoc. 95, 449 (2000), 135?143.
[55]
Art B. Owen. 2013. Monte Carlo Theory, Methods and Examples.
[56]
Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically Based Rendering: From Theory to Implementation (3rd ed.). Morgan Kaufmann Publishers Inc. 1266 pages.
[57]
Bui Tuong Phong. 1975. Illumination for computer generated pictures. Commun. ACM 18, 6 (1975), 311?317.
[58]
Ravi Ramamoorthi, Dhruv Mahajan, and Peter Belhumeur. 2007. A first-order analysis of lighting, shading, and shadows. ACM Trans. Graph. 26, 1 (2007), 21 pages.
[59]
Iman Sadeghi, Oleg Bisker, Joachim De Deken, and Henrik Wann Jensen. 2013. A practical microcylinder appearance model for cloth rendering. ACM Trans. Graph. 32, 2, Article 14 (2013), 12 pages.
[60]
Mateu Sbert, Vlastimil Havran, and Laszlo Szirmay-Kalos. 2018. Multiple importance sampling revisited: Breaking the bounds. EURASIP J. Adv. Signal Process. 2018, 1 (2018), 1?15.
[61]
Cheng Sun, Guangyan Cai, Zhengqin Li, Kai Yan, Cheng Zhang, Carl Marshall, Jia-Bin Huang, Shuang Zhao, and Zhao Dong. 2023. Neural-PBIR reconstruction of shape, material, and illumination. In Proceedings of the International Conference on Computer Vision. 18046?18056.
[62]
Alejandro Sztrajman, Gilles Rainer, Tobias Ritschel, and Tim Weyrich. 2021. Neural BRDF representation and importance sampling. Comput. Graph. Forum 40, 6 (2021), 332?346.
[63]
Justin F. Talbot, David Cline, and Parris Egbert. 2005. Importance resampling for global illumination. Render. Techn. (Proc. EGSR), 139?146.
[64]
T. S. Trowbridge and Karl P. Reitz. 1975. Average irregularity representation of a rough surface for ray reflection. J. Opt. Soc. Am. 65, 5 (1975), 531?536.
[65]
Eric Veach and Leonidas J. Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the SIGGRAPH Conference. 419?428.
[66]
Delio Vicini, S?bastien Speierer, and Wenzel Jakob. 2021. Path replay backpropagation: Differentiating light paths using constant memory and linear time. ACM Trans. Graph. (Proc. SIGGRAPH) 40, 4 (2021), 108:1?108:14.
[67]
Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014. Embree: A kernel framework for efficient CPU ray tracing. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 4 (2014), 143:1?143:8.
[68]
Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet models for refraction through rough surfaces. Render. Techn. (Proc. EGSR) (2007), 195?206.
[69]
Yu-Chen Wang, Chris Wyman, Lifan Wu, and Shuang Zhao. 2023. Amortizing samples in physics-based inverse rendering using ReSTIR. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 42, 6 (2023).
[70]
Greg Ward and Paul Heckbert. 1992. Irradiance gradients. In Proceedings of the Eurographics Workshop on Rendering. 85?98.
[71]
Gregory J. Ward. 1992. Measuring and modeling anisotropic reflection. Comput. Graph. (Proc. SIGGRAPH) 26, 2 (1992), 265?272.
[72]
Lifan Wu, Guangyan Cai, Ravi Ramamoorthi, and Shuang Zhao. 2021. Differentiable time-gated rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 40, 6 (2021), 287:1?287:16.
[73]
Liwen Wu, Rui Zhu, Mustafa B. Yaldiz, Yinhao Zhu, Hong Cai, Janarbek Matai, Fatih Porikli, Tzu-Mao Li, Manmohan Chandraker, and Ravi Ramamoorthi. 2023. Factorized inverse path tracing for efficient and accurate material-lighting estimation. In Proceedings of the International Conference on Computer Vision. 3848?3858.
[74]
Jiankai Xing, Fujun Luan, Ling-Qi Yan, Xuejun Hu, Houde Qian, and Kun Xu. 2022. Differentiable rendering using RGBXY derivatives and optimal transport. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 41, 6, Article 189 (2022), 13 pages.
[75]
Bing Xu, Liwen Wu, Milos Hasan, Fujun Luan, Iliyan Georgiev, Zexiang Xu, and Ravi Ramamoorthi. 2023b. NeuSample: Importance sampling for neural materials. In Proceedings of the SIGGRAPH Conference. Article 41.
[76]
Peiyu Xu, Sai Bangaru, Tzu-Mao Li, and Shuang Zhao. 2023a. Warped-area reparameterization of differential path integrals. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 42, 6 (2023).
[77]
Kai Yan, Christoph Lassner, Brian Budge, Zhao Dong, and Shuang Zhao. 2022. Efficient estimation of boundary integrals for path-space differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 41, 4 (2022), 123:1?123:13.
[78]
Shinyoung Yi, Donggun Kim, Kiseok Choi, Adrian Jarabo, Diego Gutierrez, and Min H. Kim. 2021. Differentiable transient rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 40, 6 (2021).
[79]
Zihan Yu, Cheng Zhang, Derek Nowrouzezahrai, Zhao Dong, and Shuang Zhao. 2022. Efficient differentiation of pixel reconstruction filters for path-space differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 41, 6 (2022), 1?16.
[80]
Cem Yuksel. 2022. High-performance polynomial root finding for graphics. ACM Comput. Graph. Interact. Tech. (Proc. HPG) 5, 3 (2022), 7:1?7:15.
[81]
Tizian Zeltner, S?bastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo estimators for differential light transport. ACM Trans. Graph. (Proc. SIGGRAPH) 40, 4 (2021).
[82]
Cheng Zhang, Zhao Dong, Michael Doggett, and Shuang Zhao. 2021a. Antithetic sampling for Monte Carlo differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 40, 4 (2021), 77:1?77:12.
[83]
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-space differentiable rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 39, 4 (2020), 143:1?143:19.
[84]
Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A differential theory of radiative transfer. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (2019), 227:1?227:16.
[85]
Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021b. Path-space differentiable rendering of participating media. ACM Trans. Graph. (Proc. SIGGRAPH) 40, 4 (2021), 76:1?76:15.
[86]
Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023. Projective sampling for differentiable rendering of geometry. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 42, 6 (2023).
