“Temporally coherent local tone mapping of HDR video” by Aydin, Stefanoski, Croci, Gross and Smolic
Conference:
Type(s):
Title:
- Temporally coherent local tone mapping of HDR video
Session/Category Title: Moving Pictures
Presenter(s)/Author(s):
Abstract:
Recent subjective studies showed that current tone mapping operators either produce disturbing temporal artifacts, or are limited in their local contrast reproduction capability. We address both of these issues and present an HDR video tone mapping operator that can greatly reduce the input dynamic range, while at the same time preserving scene details without causing significant visual artifacts. To achieve this, we revisit the commonly used spatial base-detail layer decomposition and extend it to the temporal domain. We achieve high quality spatiotemporal edge-aware filtering efficiently by using a mathematically justified iterative approach that approximates a global solution. Comparison with the state-of-the-art, both qualitatively, and quantitatively through a controlled subjective experiment, clearly shows our method’s advantages over previous work. We present local tone mapping results on challenging high resolution scenes with complex motion and varying illumination. We also demonstrate our method’s capability of preserving scene details at user adjustable scales, and its advantages for low light video sequences with significant camera noise.
References:
1. adams, A. 1981. The Print, The Ansel Adams Photography Series 3. New York Graphic Society.
2. Aubry, M., Paris, S., Hasinoff, S. Kautz, J., and Durand, F. 2014. Fast local laplacian filters: Theory and applications. ACM Trans. Graph. (To Appear) 33, 4.
3. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. Graph. 24, 3.
4. Benoit, A., Alleysson, D., Herault, J., and Callet, P. L. 2009. Spatiotemporal tone mapping operator based on a retina model. In CCIW, Springer, vol. 5646 of Lecture Notes in Computer Science, 12–22.
5. Boitard, R., Bouatouch, K., Cozot, R., Thoreau, D., and Gruson, A. 2012. Temporal coherency for video tone mapping. Proc. SPIE 8499.
6. Boitard, R., Cozot, R., Thoreau, D., and Bouatouch, K. 2014. Survey of Temporal Brightness Artifacts in Video Tone Mapping. In HDRi2014.
7. Boitard, R., Cozot, R., Thoreau, D., and Bouatouch, K. 2014. Zonal brightness coherency for video tone mapping. Signal Processing: Image Communication 29, 2, 229–246.
8. Cigla, C., and Alatan, A. A. 2013. Information permeability for stereo matching. Signal Processing: Image Communication 28, 9, 1072–1088.
9. Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum 22, 3, 419–426.Cross Ref
10. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21, 3, 257–266.
11. Eilertsen, G., Wanat, R., Mantiuk, R. K., and Unger, J. 2013. Evaluation of tone mapping operators for hdr-video. Computer Graphics Forum 32, 7, 275–284.Cross Ref
12. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. 27, 3, 67:1–67:10.
13. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Trans. Graph. 21, 3, 249–256.
14. Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM Trans. Graph. 28, 3, 22:1–22:10.
15. Ferwerda, J. A., Pattanaik, S. N., Shirley, P., and Greenberg, D. P. 1996. A model of visual adaptation for realistic image synthesis. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, 249–258.
16. Gastal, E. S. L., and Oliveira, M. M. 2011. Domain transform for edge-aware image and video processing. ACM Trans. Graph. 30, 4, 69:1–69:12.
17. Guthier, B., S., K., M., E., and W., E. 2011. Flicker reduction in tone mapped high dynamic range video. Proc. SPIE 7866.Cross Ref
18. He, K., Sun, J., and Tang, X. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 6, 1397–1409.
19. Irawan, P., Ferwerda, J. A., and Marschner, S. R. 2005. Perceptually based tone mapping of high dynamic range image streams. In Proceedings of Eurographics Conf. on Rendering Techniques, EGSR’05, 231–242.
20. Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. ACM Trans. Graph. 22, 3, 319–325.
21. Kiser, C., Reinhard, E., Tocci, M., and Tocci, N. 2012. Real time automated tone mapping system for HDR video. In Proceedings of the IEEE International Conference on Image Processing, 2749–2752.
22. Kronander, J., Gustavson, S., Bonnet, G., and Unger, J. 2013. Unified HDR reconstruction from raw cfa data. In Computational Photography (ICCP), 2013 IEEE International Conference on, 1–9.
23. Kronander, J., Gustavson, S., Bonnet, G., Ynmerman, J., and Unger, J. 2013. A unified framework for multi-sensor HDR video reconstruction. In Signal Processing: Image Communications.
24. Lang, M., Wang, O., Aydin, T., Smolic, A., and Gross, M. 2012. Practical temporal consistency for image-based graphics applications. ACM Trans. Graph. 31, 4 (July), 34:1–34:8.
25. Ledda, P., Santos, L. P., and Chalmers, A. 2004. A local model of eye adaptation for high dynamic range images. AFRIGRAPH, 151–160.
26. Lee, C., and Kim, C.-S. 2007. Gradient domain tone mapping of high dynamic range videos. In Image Processing, 2007. ICIP 2007. IEEE International Conference on, vol. 3, III — 461–III — 464.
27. Lischinski, D., Farbman, Z., Uyttendaele, and M., Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph., 646–653.
28. Mac Aodha, O., Humayun, A., Pollefeys, M., and Brostow, G. J. 2013. Learning a confidence measure for optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 35, 5, 1107–1120.
29. Mantiuk, R., Myszkowski, K., and Seidel, H.-P. 2006. A perceptual framework for contrast processing of high dynamic range images. ACM Trans. Appl. Percept. 3, 3, 286–308.
30. Mantiuk, R., Daly, S., and Kerofsky, L. 2008. Display adaptive tone mapping. ACM Trans. Graph. 27, 3, 68:1–68:10.
31. Meyer, C. D. 2001. Matrix Analysis and Applied Linear Algebra. SIAM: Society for Industrial and Applied Mathematics.
32. Milanfar, P. 2013. A tour of modern image filtering. IEEE Signal Processing Magazine 30, 1, 106–128.Cross Ref
33. Mrázek, P., Weickert, J., and Bruhn, A. 2004. On robust estimation and smoothing with spatial and tonal kernels. In Proc. Dagstuhl Seminar: Geometric Properties from Incomplete Data, Springer, 3–88722.
34. Nordstrom, K. N. 1989. Biased anisotropic diffusion–a unified regularization and diffusion approach to edge detection. Tech. rep., EECS Dept., UC Berkeley.
35. Paris, S., Hasinoff, S. W., and Kautz, J. 2011. Local laplacian filters: Edge-aware image processing with a laplacian pyramid. ACM Trans. Graph. 30, 4, 68:1–68:12.
36. Pattanaik, S. N., Tumblin, J., Yee, H., and Greenberg, D. P. 2000. Time-dependent visual adaptation for fast realistic image display. In Proc. of Conf. on Computer Graphics and Interactive Techniques, SIGGRAPH ’00, 47–54.
37. Petit, J., and Mantiuk, R. K. 2013. Assessment of video tone-mapping: Are cameras s-shaped tone-curves good enough? Journal of Visual Communication and Image Representation 24, 7, 1020–1030.
38. Ramsey, S. D., Johnson III, J. T., and Hansen, C. 2004. Adaptive temporal tone mapping. In Proceedings of the 7th IASTED International Conference on Computer Graphics and Imaging, 124–128.
39. Reinhard, E., and Devlin, K. 2005. Dynamic range reduction inspired by photoreceptor physiology. IEEE Transactions on Visualization and Computer Graphics 11, 13–24.
40. Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 3, 267–276.
41. Reinhard, E., Ward, G., Pattanaik, S., Debevec, P., Heidrich, W., and Myszkowski, K. 2010. HDR Imaging – Acquisition, Display, and Image-Based Lighting, Second Edition. Morgan Kaufmann.
42. Reinhard, E., Pouli, T., Kunkel, T., Long, B., Ballestad, A., and Damberg, G. 2012. Calibrated image appearance reproduction. ACM Trans. Graph. 31, 6 (Nov.), 201:1–201:11.
43. Tocci, M. D., Kiser, C., Tocci, N., and Sen, P. 2011. A versatile hdr video production system. ACM Trans. Graph. 30, 4, 41:1–41:10.
44. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In ICCV, 839–846.
45. Van Hateren, J. H. 2006. Encoding of high dynamic range video with a model of human cones. ACM Trans. Graph. 25, 4, 1380–1399.
46. Ye, G., Garces, E., Liu, Y., Dai, Q., and Gutierrez, D. 2014. Intrinsic Video and Applications. ACM Trans. Graph. (To Appear) 33, 4.
47. Zimmer, H., Bruhn, A., and Weickert, J. 2011. Optic Flow in Harmony. International Journal of Computer Vision 93, 3, 368–388.
