“Deep combiner for independent and correlated pixel estimates” by Back, Hua, Hachisuka and Moon
Conference:
Type(s):
Title:
- Deep combiner for independent and correlated pixel estimates
Session/Category Title: Light transport: Sampling
Presenter(s)/Author(s):
Abstract:
Monte Carlo integration is an efficient method to solve a high-dimensional integral in light transport simulation, but it typically produces noisy images due to its stochastic nature. Many existing methods, such as image denoising and gradient-domain reconstruction, aim to mitigate this noise by introducing some form of correlation among pixels. While those existing methods reduce noise, they are known to still suffer from method-specific residual noise or systematic errors. We propose a unified framework that reduces such remaining errors. Our framework takes a pair of images, one with independent estimates, and the other with the corresponding correlated estimates. Correlated pixel estimates are generated by various existing methods such as denoising and gradient-domain rendering. Our framework then combines the two images via a novel combination kernel. We model our combination kernel as a weighting function with a deep neural network that exploits the correlation among pixel estimates. To improve the robustness of our framework for outliers, we additionally propose an extension to handle multiple image buffers. The results demonstrate that our unified framework can successfully reduce the error of existing methods while treating them as black-boxes.
References:
1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.Google Scholar
2. Jonghee Back, Sung-Eui Yoon, and Bochang Moon. 2018. Feature Generation for Adaptive Gradient-Domain Path Tracing. Computer Graphics Forum 37, 7 (2018), 65–74.Google ScholarCross Ref
3. Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Trans. Graph. 36, 4, Article 97 (2017), 14 pages.Google ScholarDigital Library
4. Pablo Bauszat, Victor Petitjean, and Elmar Eisemann. 2017. Gradient-Domain Path Reusing. ACM Trans. Graph. 36, 6, Article 229 (2017), 9 pages.Google ScholarDigital Library
5. Philippe Bekaert, Mateu Sbert, and John Halton. 2002. Accelerating Path Tracing by Re-Using Paths. In Proceedings of the 13th Eurographics Workshop on Rendering (EGRW ’02). 125–134.Google ScholarDigital Library
6. Pravin Bhat, Brian Curless, Michael Cohen, and C. Lawrence Zitnick. 2008. Fourier Analysis of the 2D Screened Poisson Equation for Gradient Domain Problems. In Proceedings of the 10th European Conference on Computer Vision: Part II (ECCV ’08). 114–128.Google Scholar
7. Benedikt Bitterli. 2016. Rendering resources. https://benedikt-bitterli.me/resources/.Google Scholar
8. Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, José A. Iglesias-Guitián, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Novák. 2016. Nonlinearly Weighted First-order Regression for Denoising Monte Carlo Renderings. Computer Graphics Forum 35, 4 (2016), 107–117.Google ScholarDigital Library
9. Malik Boughida and Tamy Boubekeur. 2017. Bayesian Collaborative Denoising for Monte Carlo Rendering. Computer Graphics Forum 36, 4 (2017), 137–153.Google ScholarDigital Library
10. Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Trans. Graph. 36, 4, Article 98 (2017), 12 pages.Google ScholarDigital Library
11. Michaël Gharbi, Tzu-Mao Li, Miika Aittala, Jaakko Lehtinen, and Frédo Durand. 2019. Sample-Based Monte Carlo Denoising Using a Kernel-Splatting Network. ACM Trans. Graph. 38, 4, Article 125 (2019), 12 pages.Google ScholarDigital Library
12. Jie Guo, Mengtian Li, Quewei Li, Yuting Qiang, Bingyang Hu, Yanwen Guo, and Ling-Qi Yan. 2019. GradNet: Unsupervised Deep Screened Poisson Reconstruction for Gradient-Domain Rendering. ACM Trans. Graph. 38, 6, Article 223 (2019), 13 pages.Google ScholarDigital Library
13. Toshiya Hachisuka, Shinji Ogaki, and Henrik Wann Jensen. 2008. Progressive Photon Mapping. ACM Trans. Graph. 27, 5, Article 130 (2008), 8 pages.Google ScholarDigital Library
14. Binh-Son Hua, Adrien Gruson, Victor Petitjean, Matthias Zwicker, Derek Nowrouzezahrai, Elmar Eisemann, and Toshiya Hachisuka. 2019. A Survey on Gradient-Domain Rendering. Computer Graphics Forum 38, 2 (2019), 455–472.Google ScholarCross Ref
15. Wenzel Jakob. 2010. Mitsuba renderer.Google Scholar
16. Henrik Wann Jensen. 1996. Global Illumination using Photon Maps. In Rendering Techniques ’96. Springer Vienna, Vienna, 21–30.Google Scholar
17. James T. Kajiya. 1986. The rendering equation. In ACM SIGGRAPH ’86. 143–150.Google ScholarDigital Library
18. Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Trans. Graph. 34, 4, Article 122 (2015), 12 pages.Google ScholarDigital Library
19. Alexander Keller and Wolfgang Heidrich. 2001. Interleaved Sampling. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques. 269–276.Google Scholar
20. Markus Kettunen, Erik Härkönen, and Jaakko Lehtinen. 2019. Deep Convolutional Reconstruction for Gradient-Domain Rendering. ACM Trans. Graph. 38, 4, Article 126 (2019), 12 pages.Google ScholarDigital Library
21. Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Frédo Durand, and Matthias Zwicker. 2015. Gradient-domain Path Tracing. ACM Trans. Graph. 34, 4, Article 123 (2015), 13 pages.Google ScholarDigital Library
22. Diederik Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (2014).Google Scholar
23. Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-domain Metropolis Light Transport. ACM Trans. Graph. 32, 4, Article 95 (2013), 12 pages.Google ScholarDigital Library
24. Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012. SURE-based optimization for adaptive sampling and reconstruction. ACM Trans. Graph. 31, 6, Article 194 (2012), 9 pages.Google ScholarDigital Library
25. Michael D. McCool. 1999. Anisotropic Diffusion for Monte Carlo Noise Reduction. ACM Trans. Graph. 18, 2 (1999), 171–194.Google ScholarDigital Library
26. Peyman Milanfar. 2013. A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical. IEEE Signal Processing Magazine 30, 1 (2013), 106–128.Google ScholarCross Ref
27. Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive Rendering Based on Weighted Local Regression. ACM Trans. Graph. 33, 5, Article 170 (2014), 14 pages.Google ScholarDigital Library
28. Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive Polynomial Rendering. ACM Trans. Graph. 35, 4, Article 40 (2016), 10 pages.Google ScholarDigital Library
29. Ryan S. Overbeck, Craig Donner, and Ravi Ramamoorthi. 2009. Adaptive Wavelet Rendering. ACM Trans. Graph. 28, 5, Article 140 (2009), 12 pages.Google ScholarDigital Library
30. Matt Pharr. 2018. Guest Editor’s Introduction: Special Issue on Production Rendering. ACM Trans. Graph. 37, 3, Article 28 (2018), 4 pages.Google ScholarDigital Library
31. O. Ronneberger, P. Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS), Vol. 9351. Springer, 234–241.Google Scholar
32. Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2011. Adaptive Sampling and Reconstruction Using Greedy Error Minimization. ACM Trans. Graph. 30, 6, Article 159 (2011), 12 pages.Google ScholarDigital Library
33. Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2012. Adaptive Rendering with Non-local Means Filtering. ACM Trans. Graph. 31, 6, Article 195 (2012), 11 pages.Google ScholarDigital Library
34. Fabrice Rousselle, Marco Manzi, and Matthias Zwicker. 2013. Robust Denoising using Feature and Color Information. Computer Graphics Forum 32, 7 (2013), 121–130.Google ScholarCross Ref
35. Iman Sadeghi, Bin Chen, and Henrik Wann Jensen. 2009. Coherent path tracing. Journal of Graphics, GPU, and Game Tools 14, 2 (2009), 33–43.Google ScholarCross Ref
36. Pradeep Sen and Soheil Darabi. 2012. On Filtering the Noise from the Random Parameters in Monte Carlo Rendering. ACM Trans. Graph. 31, 3, Article 18 (2012), 15 pages.Google ScholarDigital Library
37. Hossein Talebi, Xiang Zhu, and Peyman Milanfar. 2012. How to SAIF-ly boost denoising performance. IEEE transactions on image processing 22 (12 2012), 16.Google Scholar
38. Bing Xu, Junfei Zhang, Rui Wang, Kun Xu, Yong-Liang Yang, Chuan Li, and Rui Tang. 2019. Adversarial Monte Carlo Denoising with Conditioned Auxiliary Feature Modulation. ACM Trans. Graph. 38, 6, Article 224 (2019), 12 pages.Google ScholarDigital Library
39. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum 34, 2 (2015), 667–681.Google ScholarDigital Library
