“Perceptually based downscaling of images” by Fattal
Conference:
Type:
Title:
- Perceptually based downscaling of images
Session/Category Title: Image Processing
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We propose a perceptually based method for downscaling images that provides a better apparent depiction of the input image. We formulate image downscaling as an optimization problem where the difference between the input and output images is measured using a widely adopted perceptual image quality metric. The downscaled images retain perceptually important features and details, resulting in an accurate and spatio-temporally consistent representation of the high resolution input. We derive the solution of the optimization problem in closed-form, which leads to a simple, efficient and parallelizable implementation with sums and convolutions. The algorithm has running times similar to linear filtering and is orders of magnitude faster than the state-of-the-art for image downscaling. We validate the effectiveness of the technique with extensive tests on many images, video, and by performing a user study, which indicates a clear preference for the results of the new algorithm.
References:
1. Banterle, F., Artusi, A., Aydin, T., Didyk, P., Eisemann, E., Gutierrez, D., Mantiuk, R., and Myszkowski, K. 2011. Multidimensional image retargeting. In ACM Siggraph Asia 2011 Courses, ACM, ACM Siggraph Asia. Google ScholarDigital Library
2. Bonnier, N., Schmitt, F., Brettel, H., and Berche, S. 2006. Evaluation of spatial gamut mapping algorithms. In Proc. 14th Color Imag. Conf., 56–61.Google Scholar
3. Brunet, D., Vrscay, E., and Wang, Z. 2010. Structural similarity-based approximation of signals and images using orthogonal bases. In Image Analysis and Recognition, A. Campilho and M. Kamel, Eds., vol. 6111 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 11–22. Google ScholarDigital Library
4. Brunet, D., Vrscay, E., and Wang, Z. 2012. On the mathematical properties of the structural similarity index. Image Processing, IEEE Trans. on 21, 4 (April), 1488–1499. Google ScholarDigital Library
5. Brunet, D. 2012. A Study of the Structural Similarity Image Quality Measure with Applications to Image Processing. PhD thesis, University of Waterloo.Google Scholar
6. Chai, L., Sheng, Y., and Zhang, J. 2014. Ssim performance limitation of linear equalizers. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, 1220–1224.Google Scholar
7. Channappayya, S., Bovik, A., and Heath, R. 2006. A linear estimator optimized for the structural similarity index and its application to image denoising. In Image Processing, 2006 IEEE International Conference on, 2637–2640.Google Scholar
8. Channappayya, S., Bovik, A., Caramanis, C., and Heath, R. 2008. Ssim-optimal linear image restoration. In Acoustics, Speech and Signal Processing (ICASSP), 2008. IEEE International Conference on, 765–768.Google Scholar
9. Channappayya, S., Bovik, A., and Heath, R. 2008. Rate bounds on ssim index of quantized images. Image Processing, IEEE Trans. on 17, 9 (Sept), 1624–1639. Google ScholarDigital Library
10. Channappayya, S. S., Bovik, A. C., Caramanis, C., and Jr., R. W. H. 2008. Design of linear equalizers optimized for the structural similarity index. Image Processing, IEEE Trans. on 17, 6, 857–872. Google ScholarDigital Library
11. Chen, G.-H., Yang, C.-L., and Xie, S.-L. 2006. Gradient-based structural similarity for image quality assessment. In Image Processing, IEEE International Conference on, 2929–2932.Google Scholar
12. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image Melding: Combining inconsistent images using patch-based synthesis. ACM Trans. Graph. (Proc. of SIGGRAPH 2012) 31, 4, 82:1–82:10. Google ScholarDigital Library
13. Demirtas, A., Reibman, A., and Jafarkhani, H. 2014. Full-reference quality estimation for images with different spatial resolutions. Image Processing, IEEE Trans. on 23, 5 (May), 2069–2080.Google ScholarCross Ref
14. Didyk, P., Ritschel, T., Eisemann, E., and Myszkowski, K. 2012. Perceptual Digital Imaging: Methods and Applications. CRC Press, ch. Exceeding Physical Limitations: Apparent Display Qualities.Google Scholar
15. Dong, J., and Ye, Y. 2012. Adaptive downsampling for high-definition video coding. In ICIP 2012, 2925–2928.Google Scholar
16. Gerstner, T., DeCarlo, D., Alexa, M., Finkelstein, A., Gingold, Y., and Nealen, A. 2012. Pixelated image abstraction. In NPAR 2012, Proc. of the 10th International Symposium on Non-photorealistic Animation and Rendering. Google ScholarDigital Library
17. He, L., Gao, F., Hou, W., and Hao, L. 2014. Objective image quality assessment: A survey. Int. J. Comput. Math. 91, 11 (Nov.), 2374–2388. Google ScholarDigital Library
18. Kopf, J., Shamir, A., and Peers, P. 2013. Content-adaptive image downscaling. ACM Trans. Graph. 32, 6 (Nov.), 173:1–173:8. Google ScholarDigital Library
19. Krawczyk, G., Myszkowski, K., and Seidel, H.-P. 2007. Contrast restoration by adaptive countershading. In Proc. of Eurographics 2007, Blackwell, vol. 26 of Computer Graphics Forum.Google Scholar
20. Lissner, I., Preiss, J., Urban, P., Lichtenauer, M. S., and Zolliker, P. 2013. Image-difference prediction: From grayscale to color. Image Processing, IEEE Trans. on 22, 2, 435–446. Google ScholarDigital Library
21. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., and Shum, H.-Y. 2011. Learning to detect a salient object. Pattern Analysis and Machine Intelligence, IEEE Trans. on 33, 2 (Feb), 353–367. Google ScholarDigital Library
22. Mitchell, D. P., and Netravali, A. N. 1988. Reconstruction filters in computer-graphics. In Proc. of SIGGRAPH ’88, ACM, New York, NY, USA, 221–228. Google ScholarDigital Library
23. Nehab, D., and Hoppe, H. 2011. Generalized sampling in computer graphics. Tech. Rep. MSR-TR-2011-16, February.Google Scholar
24. Ogawa, T., and Haseyama, M. 2013. Image inpainting based on sparse representations with a perceptual metric. EURASIP Journal on Advances in Signal Processing 2013, 1.Google ScholarCross Ref
25. Pang, W.-M., Qu, Y., Wong, T.-T., Cohen-Or, D., and Heng, P.-A. 2008. Structure-aware halftoning. ACM Trans. Graph. 27, 3 (Aug.), 89:1–89:8. Google ScholarDigital Library
26. Polesel, A., Ramponi, G., and Mathews, V. J. 1997. Adaptive unsharp masking for contrast enhancement. In ICIP ’97 3- Volume Set-Volume 1 – Volume 1, IEEE Computer Society, Washington, DC, USA, 267–. Google ScholarDigital Library
27. Rehman, A., Wang, Z., Brunet, D., and Vrscay, E. 2011. Ssim-inspired image denoising using sparse representations. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, 1121–1124.Google Scholar
28. Ritschel, T., Smith, K., Ihrke, M., Grosch, T., Myszkowski, K., and Seidel, H.-P. 2008. 3D Unsharp Masking for Scene Coherent Enhancement. ACM Trans. Graph. (Proc. of SIGGRAPH 2008) 27, 3. Google ScholarDigital Library
29. Shannon, C. 1998. Communication in the presence of noise. Proc. of the IEEE 86, 2 (Feb), 447–457.Google ScholarCross Ref
30. Shao, Y., Sun, F., Li, H., and Liu, Y. 2014. Structural similarity-optimal total variation algorithm for image denoising. In Foundations and Practical Applications of Cognitive Systems and Information Processing, vol. 215. Springer Berlin Heidelberg, 833–843.Google Scholar
31. Silvestre-Blanes, J. 2011. Structural similarity image quality reliability: Determining parameters and window size. Signal Processing 91, 4, 1012–1020. Google ScholarDigital Library
32. Smith, K., Landes, P.-E., Thollot, J., and Myszkowski, K. 2008. Apparent greyscale: A simple and fast conversion to perceptually accurate images and video. Computer Graphics Forum (Proc. of Eurographics 2008) 27, 2 (apr).Google Scholar
33. Thévenaz, P., Blu, T., and Unser, M. 2000. Interpolation revisited. Medical Imaging, IEEE Trans. on 19, 7, 739–758.Google ScholarCross Ref
34. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Computer Vision, 1998. Sixth International Conference on, 839–846. Google ScholarDigital Library
35. Trentacoste, M., Mantiuk, R., and Heidrich, W. 2011. Blur-Aware Image Downsizing. In Proc. of Eurographics.Google Scholar
36. Wang, Z., and Bovik, A. 2009. Mean squared error: Love it or leave it? a new look at signal fidelity measures. Signal Processing Magazine, IEEE 26, 1 (Jan), 98–117.Google ScholarCross Ref
37. Wang, Z., and Li, Q. 2007. Video quality assessment using a statistical model of human visual speed perception. J. Opt. Soc. Am. A 24, 12, B61B69.Google ScholarCross Ref
38. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Trans. on 13, 4 (April), 600–612. Google ScholarDigital Library
39. Wang, S., Rehman, A., Wang, Z., Ma, S., and Gao, W. 2011. Rate-ssim optimization for video coding. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, 833–836.Google Scholar
40. Wu, X., Zhang, X., and Wang, X. 2009. Low bit-rate image compression via adaptive down-sampling and constrained least squares upconversion. Trans. Img. Proc. 18, 3 (Mar.), 552–561. Google ScholarDigital Library
41. Yeganeh, H. 2014. Cross Dynamic Range And Cross Resolution Objective Image Quality Assessment With Applications. PhD thesis, University of Waterloo.Google Scholar
42. Zhang, Y., Zhao, D., Zhang, J., Xiong, R., and Gao, W. 2011. Interpolation-dependent image downsampling. Image Processing, IEEE Trans. on 20, 11 (Nov), 3291–3296. Google ScholarDigital Library
43. Zhang, L., Zhang, L., Mou, X., and Zhang, D. 2012. A comprehensive evaluation of full reference image quality assessment algorithms. In ICIP 2012, 1477–1480.Google Scholar
44. Zhou, F., and Liao, Q. 2015. Single-frame image super-resolution inspired by perceptual criteria. Image Processing, IET 9, 1, 1–11.Google ScholarCross Ref
