“Adaptive rendering with non-local means filtering”
Conference:
Type(s):
Title:
- Adaptive rendering with non-local means filtering
Session/Category Title: Global Illumination
Presenter(s)/Author(s):
Abstract:
We propose a novel approach for image space adaptive sampling and filtering in Monte Carlo rendering. We use an iterative scheme composed of three steps. First, we adaptively distribute samples in the image plane. Second, we denoise the image using a non-linear filter. Third, we estimate the residual per-pixel error of the filtered rendering, and the error estimate guides the sample distribution in the next iteration. The effectiveness of our approach hinges on the use of a state of the art image denoising technique, which we extend to an adaptive rendering framework. A key idea is to split the Monte Carlo samples into two buffers. This improves denoising performance and facilitates variance and error estimation. Our method relies only on the Monte Carlo samples, allowing us to handle arbitrary light transport and lens effects. In addition, it is robust to high noise levels and complex image content. We compare our approach to a state of the art adaptive rendering technique based on adaptive bandwidth selection and demonstrate substantial improvements in terms of both numerical error and visual quality. Our framework is easy to implement on top of standard Monte Carlo renderers and it incurs little computational overhead.
References:
1. Bala, K., Walter, B., and Greenberg, D. P. 2003. Combining edges and points for interactive high-quality rendering. ACM Trans. Graph. 22 (July), 631–640.
2. Bauszat, P., Eisemann, M., and Magnor, M. 2011. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering (EGSR)) 30, 4 (June), 1361–1368.
3. Bolin, M. R., and Meyer, G. W. 1998. A perceptually based adaptive sampling algorithm. In SIGGRAPH ’98, 299–309.
4. Buades, A., Coll, B., Morel, J., et al. 2005. A review of image denoising algorithms, with a new one. SIAM Journal on Multiscale Modeling and Simulation 4, 2, 490–530.
5. Buades, A., Coll, B., and Morel, J. 2008. Nonlocal image and movie denoising. International Journal of Computer Vision 76, 2, 123–139.
6. Chen, J., Wang, B., Wang, Y., Overbeck, R. S., Yong, J.-H., and Wang, W. 2011. Efficient depth-of-field rendering with adaptive sampling and multiscale reconstruction. Computer Graphics Forum.
7. Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. Image Processing, IEEE Transactions on 16, 8 (aug.), 2080–2095.
8. Dammertz, H., Sewtz, D., Hanika, J., and Lensch, H. P. A. 2010. Edge-avoiding à-trous wavelet transform for fast global illumination filtering. In Proceedings of the Conference on High Performance Graphics, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, HPG ’10, 67–75.
9. Deussen, O., Hanrahan, P., Lintermann, B., Měch, R., Pharr, M., and Prusinkiewicz, P. 1998. Realistic modeling and rendering of plant ecosystems. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, SIGGRAPH ’98, 275–286.
10. Donoho, D. L., and Johnstone, J. M. 1994. Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 3, 425–455.
11. Egan, K., Tseng, Y.-T., Holzschuch, N., Durand, F., and Ramamoorthi, R. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. 28 (July), 93:1–93:13.
12. Egan, K., Hecht, F., Durand, F., and Ramamoorthi, R. 2011. Frequency analysis and sheared filtering for shadow light fields of complex occluders. ACM Transactions on Graphics 30, 2 (Apr.), 9:1–9:13.
13. Gastal, E. S. L., and Oliveira, M. M. 2012. Adaptive manifolds for real-time high-dimensional filtering. ACM Trans. Graph. 31, 4 (July), 33:1–33:13.
14. Hachisuka, T., Jarosz, W., Weistroffer, R. P., Dale, K., Humphreys, G., Zwicker, M., and Jensen, H. W. 2008. Multidimensional adaptive sampling and reconstruction for ray tracing. ACM Trans. Graph. 27 (August), 33:1–33:10.
15. Ji, Z., Chen, Q., Sun, Q., and Xia, D. 2009. A moment-based nonlocal-means algorithm for image denoising. Information Processing Letters 109, 23, 1238–1244.
16. Kervrann, C., and Boulanger, J. 2006. Optimal spatial adaptation for patch-based image denoising. Image Processing, IEEE Transactions on 15, 10, 2866–2878.
17. Kollig, T., and Keller, A. 2002. Efficient multidimensional sampling. Computer Graphics Forum 21, 3, 557–563.
18. Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal light field reconstruction for rendering distribution effects. ACM Trans. Graph. 30 (August), 55:1–55:12.
19. Liu, Y., Wang, J., Chen, X., Guo, Y., and Peng, Q. 2008. A robust and fast non-local means algorithm for image denoising. Journal of Computer Science and Technology 23, 2, 270–279.
20. Mairal, J., Elad, M., and Sapiro, G. 2008. Sparse representation for color image restoration. Image Processing, IEEE Transactions on 17, 1 (jan.), 53–69.
21. Mitchell, D. P. 1987. Generating antialiased images at low sampling densities. SIGGRAPH Comput. Graph. 21 (August), 65–72.
22. Overbeck, R. S., Donner, C., and Ramamoorthi, R. 2009. Adaptive wavelet rendering. ACM Trans. Graph. 28 (December), 140:1–140:12.
23. Parker, S. G., Bigler, J., Dietrich, A., Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M., Morley, K., Robison, A., and Stich, M. 2010. Optix: a general purpose ray tracing engine. In ACM SIGGRAPH 2010 papers, ACM, New York, NY, USA, SIGGRAPH ’10, 66:1–66:13.
24. Pharr, M., and Humphreys, G. 2010. Physically Based Rendering, Second Edition: From Theory To Implementation, 2nd ed. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
25. Portilla, J., Strela, V., Wainwright, M., and Simoncelli, E. 2003. Image denoising using scale mixtures of gaussians in the wavelet domain. Image Processing, IEEE Transactions on 12, 11, 1338–1351.
26. Ritschel, T., Engelhardt, T., Grosch, T., Seidel, H.-P., Kautz, J., and Dachsbacher, C. 2009. Micro-rendering for scalable, parallel final gathering. ACM Trans. Graph. 28 (December), 132:1–132:8.
27. Rousselle, F., Knaus, C., and Zwicker, M. 2011. Adaptive sampling and reconstruction using greedy error minimization. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA ’11, 159:1–159:12.
28. Sen, P., and Darabi, S. 2012. On filtering the noise from the random parameters in monte carlo rendering. ACM Trans. Graph. 31, 3 (June), 18:1–18:15.
29. Shirley, P., Aila, T., Cohen, J., Enderton, E., Laine, S., Luebke, D., and McGuire, M. 2011. A local image reconstruction algorithm for stochastic rendering. In Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’11, 9–14 [email protected]
30. Sijbers, J., Den Dekker, A., Van Audekerke, J., Verhoye, M., and Van Dyck, D. 1998. Estimation of the noise in magnitude mr images. Magnetic Resonance Imaging 16, 1, 87–90.
31. Soler, C., Subr, K., Durand, F., Holzschuch, N., and Sillion, F. 2009. Fourier depth of field. ACM Trans. Graph. 28 (May), 18:1–18:12.
32. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Computer Vision, 1998. Sixth International Conference on, IEEE, 839–846.
33. Veach, E., and Guibas, L. J. 1997. Metropolis light transport. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH ’97, 65–76.
34. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Transactions on 13, 4 (april), 600–612.
35. Xu, R., and Pattanaik, S. 2005. Non-iterative, robust monte carlo noise reduction. IEEE Computer Graphics and Applications 25, 2, 31–35.
