“Deep Adaptive Sampling and Reconstruction Using Analytic Distributions” by Salehi, Manzi, Roethlin, Weber, Schroers, et al. … – ACM SIGGRAPH HISTORY ARCHIVES

“Deep Adaptive Sampling and Reconstruction Using Analytic Distributions” by Salehi, Manzi, Roethlin, Weber, Schroers, et al. …

  • 2022 SA Technical Papers_Salehi_Deep Adaptive Sampling and Reconstruction using Analytic Distributions

Conference:


Type(s):


Title:

    Deep Adaptive Sampling and Reconstruction Using Analytic Distributions

Session/Category Title:   Sampling and Reconstruction

Presenter(s)/Author(s):


Abstract:


    We propose an adaptive sampling and reconstruction method for offline Monte Carlo rendering. Our method produces sampling maps constrained by a user-defined budget that minimize the expected future denoising error. Compared to other state-of-the-art methods, which produce the necessary training data on the fly by composing pre-rendered images, our method samples from analytic noise distributions instead. These distributions are compact and closely approximate the pixel value distributions stemming from Monte Carlo rendering. Our method can efficiently sample training data by leveraging only a few per-pixel statistics of the target distribution, which provides several benefits over the current state of the art. Most notably, our analytic distributions’ modeling accuracy and sampling efficiency increase with sample count, essential for high-quality offline rendering. Although our distributions are approximate, our method supports joint end-to-end training of the sampling and denoising networks. Finally, we propose the addition of a global summary module to our architecture that accumulates valuable information from image regions outside of the network’s receptive field. This information discourages sub-optimal decisions based on local information. Our evaluation against other state-of-the-art neural sampling methods demonstrates denoising quality and data efficiency improvements.

References:


    1. Martín Abadi, Ashish Agarwal, Paul Barham, et al. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. http://tensorflow.org/
    2. Pontus Andersson, Jim Nilsson, Tomas Akenine-Möller, Magnus Oskarsson, Kalle Åström, and Mark D. Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. Proc. ACM Comput. Graph. Interact. Tech. 3, 2, Article 15 (aug 2020), 23 pages.
    3. Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graphics (Proc. SIGGRAPH) 36, 4, Article 97 (July 2017), 14 pages.
    4. Kavita Bala, Bruce Walter, and Donald P Greenberg. 2003. Combining edges and points for interactive high-quality rendering. ACM Transactions on Graphics (TOG) 22, 3 (2003), 631–640.
    5. Pablo Bauszat, Martin Eisemann, Elmar Eisemann, and Marcus Magnor. 2015. General and Robust Error Estimation and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (Proc. of Eurographics EG) 34, 2 (May 2015), 597–608. Eurographics 2015 paper.
    6. Pablo Bauszat, Martin Eisemann, and Marcus Magnor. 2011. Adaptive Sampling for Geometry-aware Reconstruction Filters. In Proc. Vision, Modeling and Visualization (VMV). Eurographics, 183–190.
    7. Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, José A. Iglesias-Guitián, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Novák. 2016. Nonlinearly Weighted First-order Regression for Denoising Monte Carlo Renderings. Computer Graphics Forum 35, 4 (2016), 107–117. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12954
    8. Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Trans. Graph. 36, 4, Article 98 (July 2017), 12 pages.
    9. Frédo Durand, Nicolas Holzschuch, Cyril Soler, Eric Chan, and François X. Sillion. 2005. A Frequency Analysis of Light Transport. ACM Trans. Graph. 24, 3 (July 2005), 1115–1126.
    10. Oskar Elek, Manu M. Thomas, and Angus Forbes. 2019. Learning Patterns in Sample Distributions for Monte Carlo Variance Reduction. arXiv:1906.00124 [cs.GR]
    11. Mikhail Figurnov, Shakir Mohamed, and Andriy Mnih. 2018. Implicit Reparameterization Gradients. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2018/file/92c8c96e4c37100777c7190b76d28233-Paper.pdf
    12. Baining Guo. 1998. Progressive radiance evaluation using directional coherence maps. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 255–266.
    13. Toshiya Hachisuka, Wojciech Jarosz, Richard Peter Weistroffer, Kevin Dale, Greg Humphreys, Matthias Zwicker, and Henrik Wann Jensen. 2008. Multidimensional Adaptive Sampling and Reconstruction for Ray Tracing. ACM Trans. Graph. 27, 3 (Aug. 2008), 1–10.
    14. Jon Hasselgren, Jacob Munkberg, Marco Salvi, Anjul Patney, and Aaron Lefohn. 2020. Neural Temporal Adaptive Sampling and Denoising. Computer Graphics Forum (2020).
    15. Mustafa Işık, Krishna Mullia, Matthew Fisher, Jonathan Eisenmann, and Michaël Gharbi. 2021. Interactive Monte Carlo Denoising using Affinity of Neural Features. ACM Transactions on Graphics (TOG) 40, 4, Article 37 (2021), 13 pages.
    16. Wenzel Jakob. 2010. Mitsuba renderer. http://www.mitsuba-renderer.org.
    17. Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Trans. Graph. 34, 4, Article 122 (July 2015), 12 pages.
    18. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
    19. David Kirk and James Arvo. 1991. Unbiased Sampling Techniques for Image Synthesis. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’91). Association for Computing Machinery, New York, NY, USA, 153–156.
    20. Alexandr Kuznetsov, Nima Khademi Kalantari, and Ravi Ramamoorthi. 2018. Deep Adaptive Sampling for Low Sample Count Rendering. Computer Graphics Forum 37 (07 2018), 35–44.
    21. Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012a. SURE-Based Optimization for Adaptive Sampling and Reconstruction. ACM Trans. Graph. 31, 6, Article 194 (Nov. 2012), 9 pages.
    22. Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012b. SURE-based optimization for adaptive sampling and reconstruction. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 31, 6, Article 194 (Nov. 2012), 9 pages.
    23. Don P. Mitchell. 1987. Generating Antialiased Images at Low Sampling Densities. SIGGRAPH Comput. Graph. 21, 4 (Aug. 1987), 65–72.
    24. Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive Rendering Based on Weighted Local Regression. ACM Trans. Graph. 33, 5, Article 170 (Sept. 2014), 14 pages.
    25. Bochang Moon, Jose A. Iglesias-Guitian, Sung-Eui Yoon, and Kenny Mitchell. 2015. Adaptive Rendering with Linear Predictions. ACM Trans. Graph. 34, 4, Article 121 (July 2015), 11 pages.
    26. Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive Polynomial Rendering. ACM Trans. Graph. 35, 4, Article 40 (July 2016), 10 pages.
    27. Ryan S. Overbeck, Craig Donner, and Ravi Ramamoorthi. 2009. Adaptive Wavelet Rendering. In ACM SIGGRAPH Asia 2009 Papers (Yokohama, Japan) (SIGGRAPH Asia ’09). Association for Computing Machinery, New York, NY, USA, Article 140, 12 pages.
    28. Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2011. Adaptive Sampling and Reconstruction Using Greedy Error Minimization. ACM Trans. Graph. 30, 6 (Dec. 2011), 1–12.
    29. Fabrice Rousselle, Claude Knaus, and Matthias Zwicker. 2012. Adaptive Rendering with Non-Local Means Filtering. ACM Trans. Graph. 31, 6, Article 195 (Nov. 2012), 11 pages.
    30. Fabrice Rousselle, Marco Manzi, and Matthias Zwicker. 2013. Robust Denoising using Feature and Color Information. Computer Graphics Forum 32 (10 2013).
    31. Rasmus Tamstorf and Henrik Jensen. 1997. Adaptive Sampling and Bias Estimation in Path Tracing. Eurographics Workshop on Rendering Techniques 97 (07 1997).
    32. Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2018) 37, 4, Article 124 (2018), 124:1–124:15 pages.
    33. Tiange Xiang, Hongliang Yuan, Haozhi Huang, and Yujin Shi. 2021. Two-Stage Monte Carlo Denoising with Adaptive Sampling and Kernel Pool. arXiv:2103.16115 [cs.CV]
    34. Xianyao Zhang, Marco Manzi, Thijs Vogels, Henrik Dahlberg, Markus Gross, and Marios Papas. 2021. Deep Compositional Denoising for High-quality Monte Carlo Rendering. Computer Graphics Forum 40, 4 (2021), 1–13. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14337
    35. Xianyao Zhang, Melvin Ott, Marco Manzi, Markus Gross, and Marios Papas. 2022. Automatic Feature Selection for Denoising Volumetric Renderings. Computer Graphics Forum 41, 4 (2022), 63–77. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14587

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org