“Understanding and improving the realism of image composites” by Xue, Agarwala, Dorsey and Rushmeier

Compositing is one of the most commonly performed operations in computer graphics. A realistic composite requires adjusting the appearance of the foreground and background so that they appear compatible; unfortunately, this task is challenging and poorly understood. We use statistical and visual perception experiments to study the realism of image composites. First, we evaluate a number of standard 2D image statistical measures, and identify those that are most significant in determining the realism of a composite. Then, we perform a human subjects experiment to determine how the changes in these key statistics influence human judgements of composite realism. Finally, we describe a data-driven algorithm that automatically adjusts these statistical measures in a foreground to make it more compatible with its background in a composite. We show a number of compositing results, and evaluate the performance of both our algorithm and previous work with a human subjects study.

1. Alexander, T., 2011. Visual effects supervisor at Industry Light & Magic. Rules of thumb in image compositing. Personal communication, Oct.Google Scholar
2. Berens, P. 2009. Circstat: a matlab toolbox for circular statistics. Journal of Statistical Software 31, 10, 1–21.Google ScholarCross Ref
3. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proceedings of CVPR, 97–104. Google ScholarDigital Library
4. Chang, C.-C., and Lin, C.-J. 2011. LIBSVM: A library for support vector machines. ACM Trans. on Intelligent Systems and Technology 2, 27:1–27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm. Google ScholarDigital Library
5. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graph. 25 (July), 624–630. Google ScholarDigital Library
6. David, H. A. 1988. The Method of Paired Comparisons. Oxford University Press, 2nd edition.Google Scholar
7. Gijsenij, A., Gevers, T., and van de Weijer, J. 2011. Computational color constancy: Survey and experiments. IEEE Trans. on Image Processing 20, 9 (Sep), 2475–2489. Google ScholarDigital Library
8. Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. ACM Trans. on Graphics 25, 3 (July), 631–637. Google ScholarDigital Library
9. Johnson, M. K., and Farid, H. 2005. Exposing digital forgeries by detecting inconsistencies in lighting. In Proceedings of the 7th Workshop on Multimedia and Security, 1–10. Google ScholarDigital Library
10. Lalonde, J.-F., and Efros, A. 2007. Using color compatibility for assessing image realism. In IEEE 11th International Conference on Computer Vision, 1–8.Google Scholar
11. Liaw, A., and Wiener, M. 2002. Classification and regression by randomforest. R News 2, 3, 18–22.Google Scholar
12. Lopez-Moreno, J., Sundstedt, V., Sangorrin, F., and Gutierrez, D. 2010. Measuring the perception of light in-consistencies. In Proceedings of APGV, ACM, 25–32. Google ScholarDigital Library
13. Lotto, R., and Purves, D. 2002. The empirical basis of color perception. Consciousness and Cognition 11, 4 (Dec.), 609–629.Google Scholar
14. Ogden, J. M., Adelson, E. H., Bergen, J., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4–15.Google Scholar
15. Ohta, N., and Robertson, A. R. 2005. Colorimetry: Fundamentals and Applications. Wiley, Chichester.Google ScholarCross Ref
16. Ostrovsky, Y., Cavanagh, P., and Sinha, P. 2005. Perceiving illumination inconsistencies in scenes. Perception 34, 11, 1301–1314.Google ScholarCross Ref
17. Paris, S., Hasinoff, S. W., and Kautz, J. 2011. Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans. on Graphics 30 (Aug), 68:1–68:12. Google ScholarDigital Library
18. Peli, E. 1990. Contrast in complex images. Journal of Optical Society of America 7, 10 (Oct), 2032–2040.Google Scholar
19. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. on Graphics 22 (Jul), 313–318. Google ScholarDigital Library
20. Pouli, T., and Reinhard, E. 2010. Progressive histogram reshaping for creative color transfer and tone reproduction. In Proceedings of NPAR, 81–90. Google ScholarDigital Library
21. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. S. 2001. Color transfer between images. IEEE Computer Graphics & Applications 21, 5 (Sept./Oct.), 34–41. Google ScholarDigital Library
22. Reinhard, E., Aküyz, A., Colbert, M., Hughes, C. E., and OConnor, M. 2004. Real-time color blending of rendered and captured video. In Interservice/Industry Training, Simulation and Education Conference, 1–9.Google Scholar
23. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., and Rott, P. 2009. A perceptually motivated online benchmark for image matting. In Proceedings of CVPR, 1826–1833.Google Scholar
24. Russell, B., Torralba, A., Murphy, K., and Freeman, W. 2008. LabelMe: A Database and Web-Based Tool for Image Annotation. International Journal of Computer Vision 77, 1 (May), 157–173. Google ScholarDigital Library
25. Smith, A. R., and Blinn, J. F. 1996. Blue screen matting. In Proceedings of SIGGRAPH 96, 259–268. Google ScholarDigital Library
26. Stokes, M., Anderson, M., Chandrasekar, S., and Motta, R. 1996. A standard default color space for the internet-srgb. Microsoft and Hewlett-Packard Joint Report.Google Scholar
27. Sunkavalli, K., Johnson, M. K., Matusik, W., and Pfister, H. 2010. Multi-scale image harmonization. ACM Trans. on Graphics 29, 4 (July), 125:1–125:10. Google ScholarDigital Library
28. Tao, M. W., Johnson, M. K., and Paris, S. 2010. Error-tolerant image compositing. In European Conference on Computer Vision, 31–44. Google ScholarDigital Library
29. Tsoumakas, G., and Katakis, I. 2007. Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3 (July/Sept.), 1–13.Google ScholarCross Ref
30. Wagberg, J., 2007. Optpop – a color properties toolbox, Mar. Software available at http://www.mathworks.com/matlabcentral/fileexchange/13788.Google Scholar
31. Wang, J., and Cohen, M. F. 2007. Image and video matting: a survey. Found. Trends. Comput. Graph. Vis. 3 (January), 97–175. Google ScholarDigital Library