“A comparative study of image retargeting”
Conference:
Type(s):
Title:
- A comparative study of image retargeting
Session/Category Title: Image & video applications
Presenter(s)/Author(s):
Moderator(s):
Abstract:
The numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. First, we create a benchmark of images and conduct a large scale user study to compare a representative number of state-of-the-art retargeting methods. Second, we present analysis of the users’ responses, where we find that humans in general agree on the evaluation of the results and show that some retargeting methods are consistently more favorable than others. Third, we examine whether computational image distance metrics can predict human retargeting perception. We show that current measures used in this context are not necessarily consistent with human rankings, and demonstrate that better results can be achieved using image features that were not previously considered for this task. We also reveal specific qualities in retargeted media that are more important for viewers. The importance of our work lies in promoting better measures to assess and guide retargeting algorithms in the future. The full benchmark we collected, including all images, retargeted results, and the collected user data, are available to the research community for further investigation at http://people.csail.mit.edu/mrub/retargetme.
References:
1. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., and Szeliski, R. 2007. A database and evaluation methodology for optical flow. In Proc. ICCV, vol. 5.Google Scholar
2. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM TOG 28, 3. Google ScholarDigital Library
3. Bose, R. C. 1955. Paired comparison designs for testing concordance between judges. Biometrika, 42, 113–121.Google Scholar
4. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. ACM TOG 28, 3. Google ScholarDigital Library
5. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM TOG 27, 3. Google ScholarDigital Library
6. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? ACM TOG 28, 3. Google ScholarDigital Library
7. David, H. 1963. The Method of Paired Comparison. Charles Griffin and Company.Google Scholar
8. Dong, W., Zhou, N., Paul, J.-C., and Zhang, X. 2009. Optimized image resizing using seam carving and scaling. ACM TOG 28, 5. Google ScholarDigital Library
9. van der Helm, P. 2000. Principles of Symmetry Perception. International Congress of Psychology.Google Scholar
10. Karni, Z., Freedman, D., and Gotsman, C. 2009. Energy-based image deformation. CGF 28, 5, 1257–1268. Google ScholarDigital Library
11. Kasutani, E., and Yamada, A. 2001. The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In International Conference on Image Processing, 674–677.Google Scholar
12. Kendall, M. G., and Babington-Smith, B. 1940. On the method of paired comparisons. Biometrica 31, 324–345.Google ScholarCross Ref
13. Kendall, M. G. 1938. A new measure of rank correlation. Biometrika, 1/2 (June), 81–93.Google Scholar
14. Kendall, M. G. 1955. Reviews. Biometrika 42.Google Scholar
15. Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM TOG 28, 5. Google ScholarDigital Library
16. Liu, F., and Gleicher, M. 2006. Video retargeting: automating pan and scan. In MULTIMEDIA, ACM, 241–250. Google ScholarDigital Library
17. Liu, C., Yuen, J., Torralba, A., Sivic, J., and Freeman, W. T. 2008. SIFT Flow: Dense correspondence across different scenes. In ECCV, 28–42. Google ScholarDigital Library
18. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91–110. Google ScholarDigital Library
19. Manjunath, B. S., Ohm, J. R., Vasudevan, V. V., and Yamada, A. 2001. Color and texture descriptors. IEEE Trans. Circuits and Systems for Video Technology 11, 703–715. Google ScholarDigital Library
20. Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings ICCV, vol. 2.Google Scholar
21. MPEG-7. 2002. ISO/IEC 15938: Multimedia Content Description Interface.Google Scholar
22. Pearson, E. S., and Hartley, H. O. 1966. Biometrika Tables for Statisticians, 3rd ed., vol. 1. Cambridge University Press.Google Scholar
23. Pele, O., and Werman, M. 2009. Fast and robust earth mover’s distances. In ICCV ’09.Google Scholar
24. Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In ICCV, 151–158.Google Scholar
25. Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM TOG 27, 3. Google ScholarDigital Library
26. Rubinstein, M., Shamir, A., and Avidan, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
27. Setyawan, I., and Lagendijk, R. L. 2004. Human perception of geometric distortions in images. In Proceedings of SPIE, Security, Steganography and Watermarking of Multimedia Contents VI, 256–267.Google Scholar
28. Shamir, A., and Sorkine, O. 2009. Visual media retargeting. In ACM SIGGRAPH Asia Courses. Google ScholarDigital Library
29. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR, 1–8.Google Scholar
30. Tyler, C. W., Ed. 1996. Human Symmetry Perception and its Computational Analysis. VSP International Science Publishers, Utrecht.Google Scholar
31. Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM TOG 27, 5. Google ScholarDigital Library
32. Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In ICCV.Google Scholar
