“Target-aware Image Denoising for Inverse Monte Carlo Rendering”
Conference:
Type(s):
Title:
- Target-aware Image Denoising for Inverse Monte Carlo Rendering
Presenter(s)/Author(s):
Abstract:
We present a novel image denoiser to improve the convergence of inverse rendering optimization, which infers scene parameters by matching a rendering image to a user-specified target image. We reformulate a regression-based denoiser using the target image to make the optimization with our denoising robust for various inverse rendering tasks.
References:
[1]
Attila T. ?fra. 2023. Intel? Open Image Denoise. https://www.openimagedenoise.org.
[2]
Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan Nov?K, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph. 36, 4, Article 97 (July 2017), 14 pages.
[3]
Martin Balint, Karol Myszkowski, Hans-Peter Seidel, and Gurprit Singh. 2023. Joint Sampling and Optimisation for Inverse Rendering. In SIGGRAPH Asia 2023 Conference Papers (Sydney, NSW, Australia) (SA ’23). Article 29, 10 pages.
[4]
Sai Praveen Bangaru, Tzu-Mao Li, and Fr?do Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6, Article 245 (Nov. 2020), 18 pages.
[5]
Benedikt Bitterli. 2016. Rendering resources. https://benedikt-bitterli.me/resources/.
[6]
Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, Jos? A. Iglesias-Guiti?n, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Nov?k. 2016. Nonlinearly Weighted First-order Regression for Denoising Monte Carlo Renderings. Computer Graphics Forum 35, 4 (2016), 107–117.
[7]
Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Trans. Graph. 39, 4, Article 148 (Aug. 2020), 17 pages.
[8]
Wesley Chang, Venkataram Sivaram, Derek Nowrouzezahrai, Toshiya Hachisuka, Ravi Ramamoorthi, and Tzu-Mao Li. 2023. Parameter-Space ReSTIR for Differentiable and Inverse Rendering. In ACM SIGGRAPH 2023 Conference Proceedings (Los Angeles, CA, USA) (SIGGRAPH ’23). Article 18, 10 pages.
[9]
Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. 2022. Shape, light, and material decomposition from images using Monte Carlo rendering and denoising. Advances in Neural Information Processing Systems 35 (2022), 22856–22869.
[10]
Wenzel Jakob, S?bastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. 2022. Mitsuba 3 renderer. https://mitsuba-renderer.org.
[11]
James T Kajiya. 1986. The rendering equation. In Proceedings of the 13th annual conference on Computer graphics and interactive techniques. 143–150.
[12]
Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. 2020. Differentiable Rendering: A Survey. CoRR abs/2006.12057 (2020). arXiv:2006.12057 https://arxiv.org/abs/2006.12057
[13]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[14]
Tzu-Mao Li, Miika Aittala, Fr?do Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. 37, 6, Article 222 (Dec. 2018), 11 pages.
[15]
Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012. SURE-based optimization for adaptive sampling and reconstruction. ACM Trans. Graph. 31, 6, Article 194 (Nov. 2012), 9 pages.
[16]
Daqi Lin, Markus Kettunen, Benedikt Bitterli, Jacopo Pantaleoni, Cem Yuksel, and Chris Wyman. 2022. Generalized Resampled Importance Sampling: Foundations of ReSTIR. ACM Trans. Graph. 41, 4, Article 75 (July 2022), 23 pages.
[17]
Clive Loader. 2006. Local regression and likelihood. Springer Science & Business Media.
[18]
Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing Discontinuous Integrands for Differentiable Rendering. ACM Trans. Graph. 38, 6, Article 228 (Nov. 2019), 14 pages.
[19]
Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive Rendering Based on Weighted Local Regression. ACM Trans. Graph. 33, 5, Article 170 (Sept. 2014), 14 pages.
[20]
Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive Polynomial Rendering. ACM Trans. Graph. 35, 4, Article 40 (jul 2016), 10 pages.
[21]
Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large steps in inverse rendering of geometry. ACM Trans. Graph. 40, 6, Article 248 (Dec. 2021), 13 pages.
[22]
Baptiste Nicolet, Fabrice Rousselle, Jan Novak, Alexander Keller, Wenzel Jakob, and Thomas M?ller. 2023. Recursive Control Variates for Inverse Rendering. ACM Trans. Graph. 42, 4, Article 62 (July 2023), 13 pages.
[23]
Merlin Nimier-David, Thomas M?ller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased Inverse Volume Rendering with Differential Trackers. ACM Trans. Graph. 41, 4, Article 44 (July 2022), 20 pages.
[24]
Merlin Nimier-David, S?bastien Speierer, Beno?t Ruiz, and Wenzel Jakob. 2020. Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering. ACM Trans. Graph. 39, 4, Article 146 (Aug. 2020), 15 pages.
[25]
Fabrice Rousselle, Wojciech Jarosz, and Jan Nov?k. 2016. Image-space control variates for rendering. ACM Trans. Graph. 35, 6, Article 169 (Dec. 2016), 12 pages.
[26]
Pradeep Sen and Soheil Darabi. 2012. On filtering the noise from the random parameters in Monte Carlo rendering. ACM Trans. Graph. 31, 3, Article 18 (May 2012), 15 pages.
[27]
Delio Vicini, S?bastien Speierer, and Wenzel Jakob. 2021. Path Replay Backpropagation: Differentiating Light Paths Using Constant Memory and Linear Time. ACM Trans. Graph. 40, 4, Article 108 (July 2021), 14 pages.
[28]
Yu-Chen Wang, Chris Wyman, Lifan Wu, and Shuang Zhao. 2023. Amortizing Samples in Physics-Based Inverse Rendering Using ReSTIR. ACM Trans. Graph. 42, 6, Article 214 (Dec. 2023), 17 pages.
[29]
Peiyu Xu, Sai Bangaru, Tzu-Mao Li, and Shuang Zhao. 2023. Warped-Area Reparameterization of Differential Path Integrals. ACM Trans. Graph. 42, 6, Article 213 (Dec. 2023), 18 pages.
[30]
Zihan Yu, Cheng Zhang, Derek Nowrouzezahrai, Zhao Dong, and Shuang Zhao. 2022. Efficient Differentiation of Pixel Reconstruction Filters for Path-Space Differentiable Rendering. ACM Trans. Graph. 41, 6, Article 191 (Nov. 2022), 16 pages.
[31]
Tizian Zeltner, S?bastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. ACM Trans. Graph. 40, 4, Article 78 (July 2021), 16 pages.
[32]
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-Space Differentiable Rendering. ACM Trans. Graph. 39, 4, Article 143 (Aug. 2020), 19 pages.
[33]
Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A Differential Theory of Radiative Transfer. ACM Trans. Graph. 38, 6, Article 227 (Nov. 2019), 16 pages.
[34]
Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021. Path-Space Differentiable Rendering of Participating Media. ACM Trans. Graph. 40, 4, Article 76 (July 2021), 15 pages.
[35]
M. Zwicker, W. Jarosz, J. Lehtinen, B. Moon, R. Ramamoorthi, F. Rousselle, P. Sen, C. Soler, and S.-E. Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum 34, 2 (2015), 667–681.
