“Real-time noise-aware tone mapping”
Conference:
Type(s):
Title:
- Real-time noise-aware tone mapping
Session/Category Title: Video Processing
Presenter(s)/Author(s):
Abstract:
Real-time high quality video tone mapping is needed for many applications, such as digital viewfinders in cameras, display algorithms which adapt to ambient light, in-camera processing, rendering engines for video games and video post-processing. We propose a viable solution for these applications by designing a video tone-mapping operator that controls the visibility of the noise, adapts to display and viewing environment, minimizes contrast distortions, preserves or enhances image details, and can be run in real-time on an incoming sequence without any preprocessing. To our knowledge, no existing solution offers all these features. Our novel contributions are: a fast procedure for computing local display-adaptive tone-curves which minimize contrast distortions, a fast method for detail enhancement free from ringing artifacts, and an integrated video tone-mapping solution combining all the above features.
References:
1. Adams, A., Gelfand, N., Dolson, J., and Levoy, M. 2009. Gaussian kd-trees for fast high-dimensional filtering. ACM Trans. Graphics 28, 3, 21:1–21:12.
2. Adams, A., Baek, J., and Davis, M. 2010. Fast High-Dimensional Filtering Using the Permutohedral Lattice. Computer Graphics Forum 29, 2, 753–762.
3. Allred, S., Radonjić, A., and Gilchrist, A. 2012. Lightness perception in high dynamic range images: Local and remote luminance effects. Journal of Vision 12, 2, 1–16.
4. Aubry, M., Paris, S., Hasinoff, S. W., Kautz, J., and Durand, F. 2014. Fast local laplacian filters: Theory and applications. ACM Trans. Graphics 33, 5, 167:1–167:14.
5. Aurich, V., and Weule, J. 1995. Non-linear gaussian filters performing edge preserving diffusion. In DAGM-Symposium, Springer, G. Sagerer, S. Posch, and F. Kummert, Eds., Informatik Aktuell, 538–545.
6. Aydin, T. O., Stefanoski, N., Croci, S., Gross, M., and Smolic, A. 2014. Temporally coherent local tone mapping of HDR video. ACM Trans. Graphics 33, 6, 1–13.
7. Baek, J., and Jacobs, D. E. 2010. Accelerating spatially varying gaussian filters. ACM Trans. Graphics 29, 6, 169:1–169:10.
8. Banterle, F., Corsini, M., Cignoni, P., and SCOPIGNO, R. 2012. A low-memory, straightforward and fast bilateral filter through subsampling in spatial domain. Computer Graphics Forum 31, 1, 19–32.
9. Barash, D. 2002. Fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE Trans. Pattern Analysis and Machine Intelligence 24, 6, 844–847.
10. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. Graphics 24, 3, 845–852.
11. Berns, R. S. 1996. Methods for characterizing CRT displays. Displays 16, 4, 173–182.
12. Black, M. J., Sapiro, G., Marimont, D. H., and Heeger, D. 1998. Robust anisotropic diffusion. Trans. Image Processing 7, 3, 421–432.
13. Boitard, R., Bouatouch, K., Cozot, R., Thoreau, D., and Gruson, A. 2012. Temporal coherency for video tone mapping. In Proc. SPIE 8499, Applications of Digital Image Processing XXXV, 84990D–84990D–10.
14. Boitard, R., Cozot, R., Thoreau, D., and Bouatouch, K. 2014. Zonal brightness coherency for video tone mapping. Signal Processing: Image Communication 29, 2, 229–246.
15. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graphics 26, 3, 103:1–103:9.
16. Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. Eurographics workshop on Rendering 14, 186–196.
17. Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 419–426.
18. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graphics 21, 3, 257–266.
19. Eilertsen, G., Wanat, R., Mantiuk, R. K., and Unger, J. 2013. Evaluation of Tone Mapping Operators for HDR-Video. Computer Graphics Forum 32, 7, 275–284.
20. Foi, A., Trimeche, M., Katkovnik, V., and Egiazarian, K. 2008. Practical poissonian-gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Processing 17, 10, 1737–1754.
21. Froehlich, J., Grandinetti, S., Eberhardt, B., Walter, S., Schilling, A., and Brendel, H. 2014. Creating Cinematic Wide Gamut HDR-Video for the Evaluation of Tone Mapping Operators and HDR-Displays. In Proc. SPIE 9023, Digital Photography X, 90230X–90230X–10.
22. Irawan, P., Ferwerda, J. A., and Marschner, S. R. 2005. Perceptually based tone mapping of high dynamic range image streams. In Proc. Eurographics Conference on Rendering Techniques 16, 231–242.
23. Katkovnik, V., Katkovnik, V., Egiazarian, K., and Astola, J. 2006. Local Approximation Techniques in Signal and Image Processing. Press Monographs. SPIE Press.
24. Kiser, C., Reinhard, E., Tocci, M., and Tocci, N. 2012. Real time automated tone mapping system for HDR video. In Proc. IEEE International Conference on Image Processing 19, 2749–2752.
25. Kronander, J., Gustavson, S., Bonnet, G., and Unger, J. 2013. Unified HDR reconstruction from raw CFA data. In Proc. IEEE International Conference on Computational Photography 5, 1–9.
26. Ledda, P., Santos, L. P., and Chalmers, A. 2004. A local model of eye adaptation for high dynamic range images. In Proc. International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa 3, 151–160.
27. Maggioni, M., Boracchi, G., Foi, A., and Egiazarian, K. 2012. Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms. IEEE Trans. Image Processing 21, 9, 3952–3966.
28. Mai, Z., Mansour, H., Mantiuk, R., Nasiopoulos, P., Ward, R., and Heidrich, W. 2011. Optimizing a tone curve for backward-compatible high dynamic range image and video compression. IEEE Trans. Image Processing 20, 6, 1558–1571.
29. Mantiuk, R., Daly, S., and Kerofsky, L. 2008. Display adaptive tone mapping. ACM Trans. Graphics 27, 3, 68:1–68:10.
30. Mantiuk, R., Mantiuk, R. K., Tomaszewska, A., and Heidrich, W. 2009. Color correction for tone mapping. Computer Graphics Forum 28, 2, 193–202.
31. Mantiuk, R., Kim, K. J., Rempel, A. G., and Heidrich, W. 2011. Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graphics 30, 4, 40:1–40:14.
32. Milanfar, P. 2013. A tour of modern image filtering: New insights and methods, both practical and theoretical. IEEE Signal Processing Magazine 30, 1, 106–128.
33. Pattanaik, S. N., Tumblin, J., Yee, H., and Greenberg, D. P. 2000. Time-dependent visual adaptation for fast realistic image display. In Proc. SIGGRAPH 00, Annual Conference Series, 47–54.
34. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Analysis Machine Intelligence 12, 7, 629–639.
35. Reinhard, E., and Devlin, K. 2005. Dynamic range reduction inspired by photoreceptor physiology. IEEE Trans. Visualization and Computer Graphics 11, 1, 13–24.
36. Takeda, H., Farsiu, S., and Milanfar, P. 2007. Kernel regression for image processing and reconstruction. IEEE Trans. Image Processing 16, 2, 349–366.
37. Takeda, H., Farsiu, S., and Milanfar, P. 2007. Higher order bilateral filters and their properties. Proc. SPIE 6498, Computational Imaging V, 64980S–64980S–9.
38. Tocci, M. D., Kiser, C., Tocci, N., and Sen, P. 2011. A versatile hdr video production system. ACM Trans. Graphics 30, 4, 41:1–41:10.
39. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. International Conference on Computer Vision 6, 839–846.
40. Van Hateren, J. H. 2006. Encoding of high dynamic range video with a model of human cones. ACM Trans. Graphics 25, 1380–1399.
41. Ward Larson, G., Rushmeier, H., and Piatko, C. 1997. A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Trans. Visualization and Computer Graphics 3, 4, 291–306.
42. Yang, Q. 2012. Recursive bilateral filtering. In Proc. European Conference on Computer Vision 12, 399–413.
43. Yoshizawa, S., Belyaev, A. G., and Yokota, H. 2010. Fast gauss bilateral filtering. Computer Graphics Forum 29, 1, 60–74.
