“Input-Dependent Uncorrelated Weighting for Monte Carlo Denoising” by Back, Hua, Hachisuka and Moon
Conference:
Type(s):
Title:
- Input-Dependent Uncorrelated Weighting for Monte Carlo Denoising
Session/Category Title: Rendering
Presenter(s)/Author(s):
Abstract:
Image-space denoising techniques have been widely employed in Monte Carlo rendering, typically blending neighboring pixel estimates using a denoising kernel. It is widely recognized that a kernel should be adapted to characteristics of the input pixel estimates in order to ensure robustness to diverse image features and amount of noise. Denoising with such an input-dependent kernel, however, can introduce a bias that makes the denoised estimate even less accurate than the noisy input estimate. Consequently, it has been considered essential to balance the bias introduced by denoising and the reduction of noise. We propose a new framework to define an input-dependent kernel that departs from the existing approaches based on error estimation or supervised learning. Rather than seeking an optimal bias-noise balance as in those existing approaches, we propose to constrain the amount of bias introduced by denoising. Such a constraint is made possible by the concept of uncorrelated statistics, which has never been applied for denoising. By designing an input-dependent kernel with uncorrelated weights against the input pixel estimates, our denoising kernel can reduce data-dependent noise with a negligible amount of bias in most cases. We demonstrate the effectiveness of our method for various scenes.
References:
[1]
Jonghee Back, Binh-Son Hua, Toshiya Hachisuka, and Bochang Moon. 2020. Deep Combiner for Independent and Correlated Pixel Estimates. ACM Trans. Graph. 39, 6, Article 242 (Nov. 2020), 12 pages.
[2]
Jonghee Back, Binh-Son Hua, Toshiya Hachisuka, and Bochang Moon. 2022. Self-Supervised Post-Correction for Monte Carlo Denoising. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH ’22). Article 18, 8 pages.
[3]
Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan NováK, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Trans. Graph. 36, 4, Article 97 (July 2017), 14 pages.
[4]
Pablo Bauszat, Victor Petitjean, and Elmar Eisemann. 2017. Gradient-Domain Path Reusing. ACM Trans. Graph. 36, 6, Article 229 (Nov. 2017), 9 pages.
[5]
Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, José A. Iglesias-Guitián, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Novák. 2016. Nonlinearly Weighted First-order Regression for Denoising Monte Carlo Renderings. Computer Graphics Forum 35, 4 (2016), 107–117.
[6]
Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting. ACM Trans. Graph. 39, 4, Article 148 (Aug. 2020), 17 pages.
[7]
Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The Design and Evolution of Disney’s Hyperion Renderer. ACM Trans. Graph. 37, 3, Article 33 (July 2018), 22 pages.
[8]
Per Christensen, Julian Fong, Jonathan Shade, Wayne Wooten, Brenden Schubert, Andrew Kensler, Stephen Friedman, Charlie Kilpatrick, Cliff Ramshaw, Marc Bannister, Brenton Rayner, Jonathan Brouillat, and Max Liani. 2018. RenderMan: An Advanced Path-Tracing Architecture for Movie Rendering. ACM Trans. Graph. 37, 3, Article 30 (Aug. 2018), 21 pages.
[9]
Oskar Elek, Manu M. Thomas, and Angus Forbes. 2019. Learning Patterns in Sample Distributions for Monte Carlo Variance Reduction. arxiv:1906.00124 [cs.GR]
[10]
Arthur Firmino, Jeppe Revall Frisvad, and Henrik Wann Jensen. 2022. Progressive Denoising of Monte Carlo Rendered Images. Computer Graphics Forum 41, 2 (2022), 1–11.
[11]
Michaël Gharbi, Tzu-Mao Li, Miika Aittala, Jaakko Lehtinen, and Frédo Durand. 2019. Sample-Based Monte Carlo Denoising Using a Kernel-Splatting Network. ACM Trans. Graph. 38, 4, Article 125 (July 2019), 12 pages.
[12]
Jeongmin Gu, Jose A. Iglesias-Guitian, and Bochang Moon. 2022. Neural James-Stein Combiner for Unbiased and Biased Renderings. ACM Trans. Graph. 41, 6, Article 262 (Nov. 2022), 14 pages.
[13]
Jie Guo, Mengtian Li, Quewei Li, Yuting Qiang, Bingyang Hu, Yanwen Guo, and Ling-Qi Yan. 2019. GradNet: Unsupervised Deep Screened Poisson Reconstruction for Gradient-Domain Rendering. ACM Trans. Graph. 38, 6, Article 223 (Nov. 2019), 13 pages.
[14]
Toshiya Hachisuka, Shinji Ogaki, and Henrik Wann Jensen. 2008. Progressive Photon Mapping. ACM Trans. Graph. 27, 5, Article 130 (Dec. 2008), 8 pages.
[15]
Robert V. Hogg. 1960. Certain Uncorrelated Statistics. J. Amer. Statist. Assoc. 55, 290 (1960), 265–267.
[16]
Binh-Son Hua, Adrien Gruson, Victor Petitjean, Matthias Zwicker, Derek Nowrouzezahrai, Elmar Eisemann, and Toshiya Hachisuka. 2019. A Survey on Gradient-Domain Rendering. Computer Graphics Forum 38, 2 (2019), 455–472.
[17]
Yuchi Huo and Sung-eui Yoon. 2021. A survey on deep learning-based Monte Carlo denoising. Computational Visual Media 7, 2 (2021), 169–185.
[18]
Mustafa Işık, Krishna Mullia, Matthew Fisher, Jonathan Eisenmann, and Michaël Gharbi. 2021. Interactive Monte Carlo Denoising Using Affinity of Neural Features. ACM Trans. Graph. 40, 4, Article 37 (July 2021), 13 pages.
[19]
Wenzel Jakob. 2010. Mitsuba renderer. http://www.mitsuba-renderer.org.
[20]
Henrik Wann Jensen. 1996. Global Illumination using Photon Maps. In Rendering Techniques ’96. Springer Vienna, Vienna, 21–30.
[21]
James T. Kajiya. 1986. The rendering equation. In ACM SIGGRAPH ’86. 143–150.
[22]
Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Trans. Graph. 34, 4, Article 122 (July 2015), 12 pages.
[23]
Markus Kettunen, Erik Härkönen, and Jaakko Lehtinen. 2019. Deep Convolutional Reconstruction for Gradient-Domain Rendering. ACM Trans. Graph. 38, 4, Article 126 (July 2019), 12 pages.
[24]
Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Frédo Durand, and Matthias Zwicker. 2015. Gradient-Domain Path Tracing. ACM Trans. Graph. 34, 4, Article 123 (July 2015), 13 pages.
[25]
Christopher Kulla, Alejandro Conty, Clifford Stein, and Larry Gritz. 2018. Sony Pictures Imageworks Arnold. ACM Trans. Graph. 37, 3, Article 29 (Aug. 2018), 18 pages.
[26]
H. O. Lancaster. 1959. Zero Correlation and Independence. Australian Journal of Statistics 1, 2 (1959), 53–56.
[27]
Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-Domain Metropolis Light Transport. ACM Trans. Graph. 32, 4, Article 95 (July 2013), 12 pages.
[28]
Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012. SURE-Based Optimization for Adaptive Sampling and Reconstruction. ACM Trans. Graph. 31, 6, Article 194 (Nov. 2012), 9 pages.
[29]
Daqi Lin, Markus Kettunen, Benedikt Bitterli, Jacopo Pantaleoni, Cem Yuksel, and Chris Wyman. 2022. Generalized Resampled Importance Sampling: Foundations of ReSTIR. ACM Trans. Graph. 41, 4, Article 75 (July 2022), 23 pages.
[30]
M. Manzi, D. Vicini, and M. Zwicker. 2016. Regularizing Image Reconstruction for Gradient-Domain Rendering with Feature Patches. Computer Graphics Forum 35, 2 (2016), 263–273.
[31]
Xiaoxu Meng, Quan Zheng, Amitabh Varshney, Gurprit Singh, and Matthias Zwicker. 2020. Real-time Monte Carlo Denoising with the Neural Bilateral Grid. In Eurographics Symposium on Rendering – DL-only Track, Carsten Dachsbacher and Matt Pharr (Eds.). The Eurographics Association.
[32]
Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive Rendering Based on Weighted Local Regression. ACM Trans. Graph. 33, 5, Article 170 (Sept. 2014), 14 pages.
[33]
Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive Polynomial Rendering. ACM Trans. Graph. 35, 4, Article 40 (July 2016), 10 pages.
[34]
Bernard Ostle and George P. Steck. 1959. Correlation Between Sample Means and Sample Ranges. J. Amer. Statist. Assoc. 54, 286 (1959), 465–471.
[35]
Y. Ouyang, S. Liu, M. Kettunen, M. Pharr, and J. Pantaleoni. 2021. ReSTIR GI: Path Resampling for Real-Time Path Tracing. Computer Graphics Forum 40, 8 (2021), 17–29.
[36]
Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson Image Editing. ACM Trans. Graph. 22, 3 (July 2003), 313–318.
[37]
Fabrice Rousselle, Wojciech Jarosz, and Jan Novák. 2016. Image-Space Control Variates for Rendering. ACM Trans. Graph. 35, 6, Article 169 (Dec. 2016), 12 pages.
[38]
Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2011. Adaptive Sampling and Reconstruction Using Greedy Error Minimization. ACM Trans. Graph. 30, 6 (Dec. 2011), 1–12.
[39]
Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2012. Adaptive Rendering with Non-Local Means Filtering. ACM Trans. Graph. 31, 6, Article 195 (Nov. 2012), 11 pages.
[40]
Fabrice Rousselle, Marco Manzi, and Matthias Zwicker. 2013. Robust Denoising using Feature and Color Information. Computer Graphics Forum 32, 7 (2013), 121–130.
[41]
Farnood Salehi, Marco Manzi, Gerhard Roethlin, Romann Weber, Christopher Schroers, and Marios Papas. 2022. Deep Adaptive Sampling and Reconstruction Using Analytic Distributions. ACM Trans. Graph. 41, 6, Article 259 (Nov. 2022), 16 pages.
[42]
Pradeep Sen and Soheil Darabi. 2012. On Filtering the Noise from the Random Parameters in Monte Carlo Rendering. ACM Trans. Graph. 31, 3, Article 18 (May 2012), 15 pages.
[43]
Thijs Vogels, Fabrice Rousselle, Brian Mcwilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Trans. Graph. 37, 4, Article 124 (July 2018), 15 pages.
[44]
Douglas A. Wolfe. 1973. Some General Results about Uncorrelated Statistics. J. Amer. Statist. Assoc. 68, 344 (1973), 1013–1018.
[45]
Jiaqi Yu, Yongwei Nie, Chengjiang Long, Wenju Xu, Qing Zhang, and Guiqing Li. 2021. Monte Carlo Denoising via Auxiliary Feature Guided Self-Attention. ACM Trans. Graph. 40, 6, Article 273 (Dec. 2021), 13 pages.
[46]
Shaokun Zheng, Fengshi Zheng, Kun Xu, and Ling-Qi Yan. 2021. Ensemble Denoising for Monte Carlo Renderings. ACM Trans. Graph. 40, 6, Article 274 (Dec. 2021), 17 pages.
[47]
M. Zwicker, W. Jarosz, J. Lehtinen, B. Moon, R. Ramamoorthi, F. Rousselle, P. Sen, C. Soler, and S.-E. Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum 34, 2 (2015), 667–681.
