“Perception-motivated high dynamic range video encoding” by Mantiuk, Krawczyk, Myszkowski and Seidel

Due to rapid technological progress in high dynamic range (HDR) video capture and display, the efficient storage and transmission of such data is crucial for the completeness of any HDR imaging pipeline. We propose a new approach for inter-frame encoding of HDR video, which is embedded in the well-established MPEG-4 video compression standard. The key component of our technique is luminance quantization that is optimized for the contrast threshold perception in the human visual system. The quantization scheme requires only 10–11 bits to encode 12 orders of magnitude of visible luminance range and does not lead to perceivable contouring artifacts. Besides video encoding, the proposed quantization provides perceptually-optimized luminance sampling for fast implementation of any global tone mapping operator using a lookup table. To improve the quality of synthetic video sequences, we introduce a coding scheme for discrete cosine transform (DCT) blocks with high contrast. We demonstrate the capabilities of HDR video in a player, which enables decoding, tone mapping, and applying post-processing effects in real-time. The tone mapping algorithm as well as its parameters can be changed interactively while the video is playing. We can simulate post-processing effects such as glare, night vision, and motion blur, which appear very realistic due to the usage of HDR data.

1. BOGART, R., KAINZ, F., AND HESS, D. 2003. OpenEXR image file format. In ACM SIGGRAPH 2003, Sketches & Applications.Google Scholar
2. BORDER, P., AND GUILLOTEL, P. 2000. Perceptually adapted MPEG video encoding. In IS&T/SPIE Conf. on Hum. Vis. and Electronic Imaging V, Proc. of SPIE, volume 3959, 168–175.Google Scholar
3. BURT, P., AND KOLCZYNSKI, R. 1993. Enhanced image capture through fusion. In Proc. of International Conference on Computer Vision (ICCV), 173–182.Google ScholarCross Ref
4. CIE. 1981. An Analytical Model for Describing the Influence of Lighting Parameters Upon Visual Performance, vol. 1. Technical Foundations, CIE 19/2.1. International Organization for Standardization.Google Scholar
5. DALY, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Image and Human Vision, Cambridge, MA: MIT Press, A. Watson, Ed., 179–206. Google ScholarDigital Library
6. DEBEVEC, P., AND MALIK, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 369–378. Google ScholarDigital Library
7. DEVLIN, K., CHALMERS, A., WILKIE, A., AND PURGATHOFER, W. 2002. Tone Reproduction and Physically Based Spectral Rendering. In Eurographics 2002: State of the Art Reports, Eurographics, 101–123.Google Scholar
8. DRAGO, F., MYSZKOWSKI, K., ANNEN, T., AND CHIBA, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum, proceedings of Eurographics 2003 22, 3, 419–426.Google Scholar
9. DURAND, F., AND DORSEY, J. 2000. Interactive tone mapping. In Rendering Techniques 2000: 11th Eurographics Workshop on Rendering, 219–230. Google ScholarDigital Library
10. FERWERDA, J., PATTANAIK, S., SHIRLEY, P., AND GREENBERG, D. 1996. A model of visual adaptation for realistic image synthesis. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 249–258. Google ScholarDigital Library
11. GOODNIGHT, N., WANG, R., WOOLLEY, C., AND HUMPHREYS, G. 2003. Interactive time-dependent tone mapping using programmable graphics hardware. In Rendering Techniques 2003: 14th Eurographics Symposium on Rendering, 26–37. Google ScholarDigital Library
12. HOOD, D., AND FINKELSTEIN, M. 1986. Sensitivity to light. In Handbook of Perception and Human Performance: 1. Sensory Processes and Perception, Wiley, New York, K. Boff, L. Kaufman, and J. Thomas, Eds., vol. 1.Google Scholar
13. HUNT, R. 1995. The Reproduction of Colour in Photography, Printing and Television: 5th Edition. Fountain Press.Google Scholar
14. ISO-IEC-14496-2. 1999. Information technology: Coding of audio-visual objects, Part 2: Visual. International Organization for Standardization, Geneva, Switzerland.Google Scholar
15. KANG, S., UYTTENDAELE, M., WINDER, S., AND SZELISKI, R. 2003. High dynamic range video. ACM Transactions on Graphics 22, 3, 319–325. Google ScholarDigital Library
16. LUBIN, J., AND PICA, A. 1991. A non-uniform quantizer matched to the human visual performance. Society of Information Display Int. Symposium Technical Digest of Papers, 22, 619–622.Google Scholar
17. NADENAU, M. 2000. Integration of Human color vision Models into High Quality Image Compression. PhD thesis, École Polytechnique Fédéral Lausane.Google Scholar
18. NAYAR, S., AND BRANZOI, V. 2003. Adaptive dynamic range imaging: Optical control of pixel exposures over space and time. In Proc. of IEEE International Conference on Computer Vision (ICCV 2003), 1168–1175. Google ScholarDigital Library
19. NAYAR, S., AND MITSUNAGA, T. 2000. High dynamic range imaging: Spatially varying pixel exposures. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, 472–479.Google ScholarCross Ref
20. PATTANAIK, S., TUMBLIN, J., YEE, H., AND GREENBERG, D. 2000. Time-dependent visual adaption for realistic image display. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 47–54. Google ScholarDigital Library
21. PRESS, W., TEUKOLSKY, S., VETTERLING, W., AND FLANNERY, B. 1993. Numerical Recipes in C. Cambridge Univ. Press.Google Scholar
22. REINHARD, E., STARK, M., SHIRLEY, P., AND FERWERDA, J. 2002. Photographic tone reproduction for digital images. ACM Transactions on Graphics 21, 3, 267–276. Google ScholarDigital Library
23. SAITO, K. 1995. Electronic image pickup device. Japanese Patent 07-254965.Google Scholar
24. SEETZEN, H., HEIDRICH, W., STUERZLINGER, W., WARD, G., WHITE-HEAD, L., TRENTACOSTE, M., GHOSH, A., AND VOROZCOVS, A. 2004. High dynamic range display systems. ACM Transactions on Graphics 23, 3. Google ScholarDigital Library
25. SEZAN, M., YIP, K., AND DALY, S. 1987. Uniform perceptual quantization: Applications to digital radiography. IEEE Transactions on Systems, Man, and Cybernetics 17, 4, 622–634.Google ScholarCross Ref
26. SHAPLEY, R., AND ENROTH-CUGELL, C. 1984. Visual adaption and retinal gain controls. In Progress in Retinal Research, Oxford: Pergamon Press, vol. 3, 263–346.Google Scholar
27. SHEN, K., AND DELP, E. 1999. Wavelet based rate scalable video compression. IEEE Transactions on Circuits and Systems for Video Technology 9, 1, 109–122. Google ScholarDigital Library
28. SPENCER, G., SHIRLEY, P., ZIMMERMAN, K., AND GREENBERG, D. 1995. Physically-based glare effects for digital images. In Proceedings of ACM SIGGRAPH 95, 325–334. Google ScholarDigital Library
29. THOMSPON, W. B., SHIRLEY, P., AND FERWERDA, J. A. 2002. A spatial post-processing algorithm for images of night scenes. Journal of Graphics Tools 7, 1, 1–12. Google ScholarDigital Library
30. VAN NES, F., AND BOUMAN, M. 1967. Spatial modulation transfer in the human eye. Journal of the Optical Society of America 57, 401–406.Google ScholarCross Ref
31. WANG, Z., AND BOVIK, A. 2002. A universal image quality index. IEEE Signal Processing Letters 9, 3, 81–84.Google ScholarCross Ref
32. WARD LARSON, G., RUSHMEIER, H., AND PIATKO, C. 1997. A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Transactions on Visualization and Computer Graphics 3, 4, 291–306. Google ScholarDigital Library
33. WARD LARSON, G. 1998. Logluv encoding for full-gamut, high-dynamic range images. Journal of Graphics Tools 3, 1, 815–30. Google ScholarDigital Library
34. WARD, G. 1991. Real pixels. In Graphics Gems II, J. Arvo, Ed. Academic Press, 80–83.Google Scholar