“Backward compatible high dynamic range MPEG video compression” by Mantiuk, Efremov, Myszkowski and Seidel

To embrace the imminent transition from traditional low-contrast video (LDR) content to superior high dynamic range (HDR) content, we propose a novel backward compatible HDR video compression (HDR MPEG) method. We introduce a compact reconstruction function that is used to decompose an HDR video stream into a residual stream and a standard LDR stream, which can be played on existing MPEG decoders, such as DVD players. The reconstruction function is finely tuned to the content of each HDR frame to achieve strong decorrelation between the LDR and residual streams, which minimizes the amount of redundant information. The size of the residual stream is further reduced by removing invisible details prior to compression using our HDR-enabled filter, which models luminance adaptation, contrast sensitivity, and visual masking based on the HDR content. Designed especially for DVD movie distribution, our HDR MPEG compression method features low storage requirements for HDR content resulting in a 30% size increase to an LDR video sequence. The proposed compression method does not impose restrictions or modify the appearance of the LDR or HDR video. This is important for backward compatibility of the LDR stream with current DVD appearance, and also enables independent fine tuning, tone mapping, and color grading of both streams.

1. Ahumada, A., and Peterson, H. 1993. Luminance-model-based DCT quantization for color image compression. In Human Vision, Visual Processing and Digital Display, SPIE, volume 3299, 191–201.Google Scholar
2. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. on Graph. 24, 3, 845–852. Google ScholarDigital Library
3. Bogart, R., Kainz, F., and Hess, D. 2003. OpenEXR image file format. In ACM SIGGRAPH 2003, Sketches & Applications.Google Scholar
4. Bolin, M. R., and Meyer, G. W. 1998. A perceptually based adaptive sampling Algorithm. In Proc. of ACM SIGGRAPH 1998, 299–309. Google ScholarDigital Library
5. Border, P., and Guillotel, P. 2000. Perceptually adapted MPEG video encoding. In Proc. of Human Vision and Electronic Imaging V, SPIE, volume 3959, 168–175.Google ScholarCross Ref
6. Bradley, A. P. 1999. A wavelet visible difference predictor. IEEE Transactions on Image Processing 8, 5, 717–730. Google ScholarDigital Library
7. Daly, S. J., and Feng, X. 2003. Bit-depth extension using spatiotemporal microdither based on models of the equivalent input noise of the visual system. In Color Imaging VIII: Processing, Hardcopy, and Applications, SPIE, volume 5008, 455–466.Google Scholar
8. Daly, S. J., and Feng, X. 2004. Decontouring: Prevention and removal of false contour artifacts. In Proc. of Human Vision and Electronic Imaging IX, SPIE, vol. 5292, 130–149.Google Scholar
9. Daly, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Image and Human Vision, Cambridge, MA: MIT Press, A. Watson, Ed., 179–206. Google ScholarDigital Library
10. Demos, G. 2004. High quality, wide dynamic range, compression system. In SMPTE Technical Conference Proceedings.Google Scholar
11. Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum, Proceedings of Eurographics 2003 22, 3, 419–426.Google Scholar
12. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. on Graph. 21, 3, 257–266. Google ScholarDigital Library
13. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Trans. on Graph. 21, 3, 249–256. Google ScholarDigital Library
14. Ferwerda, J. A., Shirley, P., Pattanaik, S. N., and Greenberg, D. P. 1997. A model of Visual masking for computer graphics. In Proc. of ACM SIGGRAPH 1997, 143–152. Google ScholarDigital Library
15. Krawczyk, G., Myszkowski, K., and Seidel, H.-P. 2005. Perceptual effects in real-time tone mapping. In SCCG ’05: Proc. of the 21st Spring Conference on Computer Graphics, 195–202. Google ScholarDigital Library
16. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. on Graph. 24, 3, 836–844. Google ScholarDigital Library
17. Lucian, I., Felicia, S., Charles, S., and Siefkien, H. 2005. Digital encode and method of encoding high dynamic range video images. In US Patent 6,867,717.Google Scholar
18. Mantiuk, R., Krawczyk, G., Myszkowski, K., and Seidel, H.-P. 2004. Perception-motivated high dynamic range video encoding. ACM Trans. on Graph. 23, 3, 730–738. Google ScholarDigital Library
19. Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Predicting visible differences in high dynamic range images – model and its calibration. In Proc. of Human Vision and Electronic Imaging X, SPIE, volume 5666, 204–214.Google Scholar
20. Mantiuk, R., Efremov, A., Myszkowski, K., and Seidel, H.-P. 2006. Design and evaluation of backward compatible high dynamic range video compression. MPI Technical Report MPI-I-2006-4-001.Google Scholar
21. Mantiuk, R., Myszkowski, K., and Seidel, H.-P. 2006. Lossy compression of high dynamic range images and video. In Proc. of Human Vision and Electronic Imaging XI, SPIE, San Jose, USA, vol. 6057 of Proceedings of SPIE, 60570V.Google Scholar
22. Matusik, W., and Pfister, H. 2004. 3D TV: a scalable system for realtime acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. on Graph. 23, 3, 814–824. Google ScholarDigital Library
23. Pattanaik, S., Tumblin, J., Yee, H., and Greenberg, D. 2000. Time-dependent visual adaptation for realistic image display. In Proceedings of ACM SIGGRAPH 2000, ACM Press, New York, NY, USA, Computer Graphics Proceedings, Annual Conference Series, 47–54. Google ScholarDigital Library
24. Ramasubramanian, M., Pattanaik, S. N., and Greenberg, D. P. 1999. A perceptually based physical error metric for realistic image synthesis. In Proc. of ACM SIGGRAPH 1999, 73–82. Google ScholarDigital Library
25. Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. ACM Trans. on Graph. 21, 3, 267–276. Google ScholarDigital Library
26. Reinhard, E., Ward, G., Pattanaik, S., and Debevec, P. 2005. High Dynamic Range Imaging. Data Acquisition, Manipulation, and Display. Morgan Kaufmann.Google Scholar
27. Safranek, R. J. 1993. JPEG compliant encoder using perceptually based quantization. In Human Vision, Visual Processing, and Digital Display IV, SPIE, volume 1913, 117–126.Google Scholar
28. Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., White-Head, L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High dynamic range display systems. ACM Trans. on Graph. 23, 3, 757–765. Google ScholarDigital Library
29. Spaulding, K. E., Woolfe, G. J., and Joshi, R. L. 2003. Using a residual image to extend the color gamut and dynamic range of an sRGB image. In Proc. of IS&T PICS Conference, 307–314.Google Scholar
30. Wang, Z., and Bovik, A. 2002. A universal image quality index. IEEE Signal Processing Letters 9, 3, 81–84.Google ScholarCross Ref
31. Ward, G., and Simmons, M. 2004. Subband encoding of high dynamic range imagery. In APGV ’04: 1st Symposium on Applied Perception in Graphics and Visualization, 83–90. Google ScholarDigital Library
32. Ward, G., and Simmons, M. 2005. JPEG-HDR: A backwards-compatible, high dynamic range extension to JPEG. In Proceedings of the 13th Color Imaging Conference, 283–290. Google ScholarDigital Library
33. Ward Larson, G. 1998. LogLuv encoding for full-gamut, high-dynamic range images. Journal of Graphics Tools 3, 1, 815–30. Google ScholarDigital Library
34. Watson, A. B., Solomon, J. A., Ahumada, A., and Gale, A. 1994. DCT basis function visibility: Effects of viewing distances and contrast masking. In Human Vision, Visual Processing, and Digital Display V, SPIE, volume 2179, 99–108.Google Scholar
35. Watson, A. 1987. The cortex transform: Rapid computation of simulated neural images. Comp. Vis. Graph. and Image Proc. 39, 311–327. Google ScholarDigital Library
36. Wenjun, Z., Daly, S., and Shawmin, L. 2000. Visual optimization tools in JPEG 2000. In IEEE Intern. Conf. on Image Processing, 37–40.Google Scholar
37. Xu, R., Pattanaik, S., and Hughes, C. 2005. High-dynamic range still-image encoding in JPEG 2000. IEEE Comp. Graph. and Appl. 26, 6, 57–64. Google ScholarDigital Library