“NeuSample: Importance Sampling for Neural Materials” by Xu, Wu, Hasan, Luan, Georgiev, et al. …
Conference:
Type:
Title:
- NeuSample: Importance Sampling for Neural Materials
Session/Category Title: Material Rendering
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Neural material representations have recently been proposed to augment the material appearance toolbox used in realistic rendering. These models are successful at tasks ranging from measured BTF compression, through efficient rendering of synthetic displaced materials with occlusions, to BSDF layering. However, importance sampling has been an after-thought in most neural material approaches, and has been handled by inefficient cosine-hemisphere sampling or mixing it with an additional simple analytic lobe. In this paper we fill that gap, by evaluating and comparing various pdf-learning approaches for sampling spatially varying neural materials, and proposing new variations of these approaches. We investigate three sampling approaches: analytic-lobe mixtures, normalizing flows, and histogram prediction. Within each type, we introduce improvements beyond previous work, and we extensively evaluate and compare these approaches in terms of sampling rate, wall-clock time, and final visual quality. Our versions of normalizing flows and histogram mixtures perform well and can be used in practical rendering systems, potentially facilitating the broader adoption of neural material models in production.
References:
1. Hendrik Baatz, Jonathan Granskog, Marios Papas, Fabrice Rousselle, and Jan Novák. 2021. NeRF-Tex: Neural Reflectance Field Textures. In Eurographics Symposium on Rendering – DL-only Track, Adrien Bousseau and Morgan McGuire (Eds.). The Eurographics Association.
2. Laurent Dinh, David Krueger, and Yoshua Bengio. 2014. NICE: Non-linear Independent Components Estimation. arxiv:1410.8516 [cs.LG]
3. Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016).
4. Hadi Mohaghegh Dolatabadi, Sarah Erfani, and Christopher Leckie. 2020. Invertible generative modeling using linear rational splines. In International Conference on Artificial Intelligence and Statistics. PMLR, 4236–4246.
5. Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. 2019. Neural spline flows. Advances in neural information processing systems 32 (2019).
6. Jiahui Fan, Beibei Wang, Milos Hasan, Jian Yang, and Ling-Qi Yan. 2022. Neural Layered BRDFs. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH ’22). Association for Computing Machinery, New York, NY, USA, Article 4, 8 pages. https://doi.org/10.1145/3528233.3530732
7. Chin-Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. 2018. Neural autoregressive flows. In International Conference on Machine Learning. PMLR, 2078–2087.
8. Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, and Delio Vicini. 2022. Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering. Transactions on Graphics (Proceedings of SIGGRAPH) 41, 4 (July 2022). https://doi.org/10.1145/3528223.3530099
9. Ivan Kobyzev, Simon J. D. Prince, and Marcus A. Brubaker. 2019. Normalizing Flows: An Introduction and Review of Current Methods. arxiv:1908.09257 [stat.ML]
10. Alexandr Kuznetsov, Krishna Mullia, Zexiang Xu, Miloš Hašan, and Ravi Ramamoorthi. 2021. NeuMIP: multi-resolution neural materials. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–13.
11. Alexandr Kuznetsov, Xuezheng Wang, Krishna Mullia, Fujun Luan, Zexiang Xu, Miloš Hašan, and Ravi Ramamoorthi. 2022. Rendering Neural Materials on Curved Surfaces. SIGGRAPH ’22 Conference Proceedings (2022), 9 pages. https://doi.org/10.1145/3528233.3530721
12. Thomas Müller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. 2019. Neural Importance Sampling. ACM Transactions on Graphics (TOG) 38, 5, Article 145 (2019), 19 pages.
13. Addy Ngan, Frédo Durand, and Wojciech Matusik. 2005. Experimental Analysis of BRDF Models.Rendering Techniques 2005, 16th (2005), 2.
14. George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. 2021. Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research 22, 1 (2021), 2617–2680.
15. George Papamakarios, Theo Pavlakou, and Iain Murray. 2017. Masked autoregressive flow for density estimation. Advances in neural information processing systems 30 (2017).
16. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
17. Gilles Rainer, Abhijeet Ghosh, Wenzel Jakob, and Tim Weyrich. 2020. Unified Neural Encoding of BTFs. Computer Graphics Forum (Proceedings of Eurographics) 39, 2 (2020), 167–178.
18. Gilles Rainer, Wenzel Jakob, Abhijeet Ghosh, and Tim Weyrich. 2019. Neural BTF Compression and Interpolation. Computer Graphics Forum (2019).
19. Alejandro Sztrajman, Gilles Rainer, Tobias Ritschel, and Tim Weyrich. 2021. Neural BRDF Representation and Importance Sampling. Computer Graphics Forum (2021). https://doi.org/10.1111/cgf.14335
20. Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation. Ph. D. Dissertation. Stanford University.
21. Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction Through Rough Surfaces(EGSR 07). 195–206.
22. Antoine Wehenkel and Gilles Louppe. 2019. Unconstrained monotonic neural networks. Advances in neural information processing systems 32 (2019).
23. Christina Winkler, Daniel Worrall, Emiel Hoogeboom, and Max Welling. 2019. Learning likelihoods with conditional normalizing flows. (2019). arxiv:1912.00042
24. Feng Xie, Anton Kaplanyan, Warren Hunt, and Pat Hanrahan. 2019. Multiple Scattering Using Machine Learning. In ACM SIGGRAPH 2019 Talks (Los Angeles, California) (SIGGRAPH ’19). Association for Computing Machinery, New York, NY, USA, Article 70, 2 pages. https://doi.org/10.1145/3306307.3328144
25. Junqiu Zhu, Yaoyi Bai, Zilin Xu, Steve Bako, Edgar Velázquez-Armendáriz, Lu Wang, Pradeep Sen, Miloš Hašan, and Ling-Qi Yan. 2021. Neural Complex Luminaires: Representation and Rendering. ACM Trans. Graph. 40, 4, Article 57 (jul 2021), 12 pages. https://doi.org/10.1145/3450626.3459798
