“Inverse image editing: recovering a semantic editing history from a before-and-after image pair” – ACM SIGGRAPH HISTORY ARCHIVES

“Inverse image editing: recovering a semantic editing history from a before-and-after image pair”

  • 2013 SA Technical Papers_Hu_Inverse Image Editing-Recovering a Semantic Editing History from a.jpg

Conference:


Type(s):


Title:

    Inverse image editing: recovering a semantic editing history from a before-and-after image pair

Session/Category Title:   Image & Video Editing

Presenter(s)/Author(s):


Abstract:


    We study the problem of inverse image editing, which recovers a semantically-meaningful editing history from a source image and an edited copy. Our approach supports a wide range of commonly-used editing operations such as cropping, object insertion and removal, linear and non-linear color transformations, and spatially-varying adjustment brushes. Given an input image pair, we first apply a dense correspondence method between them to match edited image regions with their sources. For each edited region, we determine geometric and semantic appearance operations that have been applied. Finally, we compute an optimal editing path from the region-level editing operations, based on predefined semantic constraints. The recovered history can be used in various applications such as image re-editing, edit transfer, and image revision control. A user study suggests that the editing histories generated from our system are semantically comparable to the ones generated by artists.

References:


    1. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3, 24:1–24:11.
    2. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized patchmatch correspondence algorithm. In Proc. of ECCV, 29–43.
    3. Berthouzoz, F., Li, W., Dontcheva, M., and Agrawala, M. 2011. A framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations. ACM Trans. Graph. 30, 5, 120:1–120:14.
    4. Bleyer, M., Rhemann, C., and Rother, C. 2011. Patchmatch stereo – stereo matching with slanted support windows. In Proceedings of the British Machine Vision Conference, 14.1–14.11.
    5. Brox, T., Bregler, C., and Malik, J. 2009. Large displacement optical flow. In Proc. of CVPR, 41–48.
    6. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input / output image pairs. In Proc. of CVPR, 97–104.
    7. Chen, H.-T., Wei, L.-Y., and Chang, C.-F. 2011. Nonlinear revision control for images. ACM Trans. Graph. 30, 4, 105:1–105:10.
    8. Chen, H.-T., Wei, L.-Y., Hartmann, B., and Agrawala, M. 2012. Data-driven adaptive history for image editing. Technical Report.
    9. Cheng, M.-M., Zhang, F.-L., Mitra, N. J., Huang, X., and Hu, S.-M. 2010. Repfinder: finding approximately repeated scene elements for image editing. ACM Trans. Graph. 29 (July), 83:1–83:8.
    10. Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603–619.
    11. Fu, H., Zhou, S., Liu, L., and Mitra, N. J. 2011. Animated construction of line drawings. ACM Trans. Graph. 30, 6 (Dec.), 133:1–133:10.
    12. Grabler, F., Agrawala, M., Li, W., Dontcheva, M., and Igarashi, T. 2009. Generating photo manipulation tutorials by demonstration. ACM Trans. Graph. 28, 3 (July), 66:1–66:9.
    13. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4, 70:1–70:10.
    14. Heer, J., Mackinlay, J., Stolte, C., and Agrawala, M. 2008. Graphical histories for visualization: Supporting analysis, communication, and evaluation. IEEE Transactions on Visualization and Computer Graphics 14, 6, 1189–1196.
    15. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proc. of Siggraph, 327–340.
    16. Hu, S.-M., Chen, T., Xu, K., Cheng, M.-M., and Martin, R. R. 2013. Internet visual media processing: a survey with graphics and vision applications. The Visual Computer 29, 5, 393–405.
    17. Kee, E., O’brien, J., and Farid, H. 2013. Exposing photo manipulation with inconsistent shadows. ACM Transactions on Graphics 32, 3, 28:1–28:12.
    18. Kong, N., Grossman, T., Hartmann, B., Agrawala, M., and Fitzmaurice, G. W. 2012. Delta: a tool for representing and comparing workflows. In Proc. of CHI, 1027–1036.
    19. Kurlander, D., and Feiner, S. 1988. Editable graphical histories. In IEEE Workshop on Visual Languages, 127–134.
    20. Lai, Y.-K., Hu, S.-M., and Martin, R. R. 2009. Automatic and topology-preserving gradient mesh generation for image vectorization. ACM Trans. Graph. 28, 3, 85:1–85:8.
    21. Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2, 228–242.
    22. Liao, Z., Hoppe, H., Forsyth, D., and Yu, Y. 2012. A subdivision-based representation for vector image editing. IEEE Transactions on Visualization and Computer Graphics 18, 11, 1858–1867.
    23. Liu, C., Yuen, J., Torralba, A., Sivic, J., and Freeman, W. T. 2008. Sift flow: Dense correspondence across different scenes. In Proc. of ECCV, 28–42.
    24. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91–110.
    25. Ma, L.-Q., Xu, K., Wong, T.-T., Jiang, B.-Y., and Hu, S.-M. 2013. Change blindness images. IEEE Transactions on Visualization and Computer Graphics, to appear.
    26. O’Brien, J. F., and Farid, H. 2012. Exposing photo manipulation with inconsistent reflections. ACM Transactions on Graphics 31, 1, 4:1–4:11.
    27. Rother, C., Kolmogorov, V., and Blake, A. 2004. “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 3, 309–314.
    28. Su, S. L., Paris, S., Aliaga, F., Scull, C., Johnson, S., and Durand, F. 2009. Interactive visual histories for vector graphics. Tech Report, MIT-CSAIL-TR-2009-031.
    29. Xiao, C., Liu, M., Yongwei, N., and Dong, Z. 2011. Fast exact nearest patch matching for patch-based image editing and processing. IEEE Transactions on Visualization and Computer Graphics 17, 8, 1122–1134.
    30. Yang, G., Stewart, C., Sofka, M., and Tsai, C.-L. 2007. Registration of challenging image pairs: Initialization, estimation, and decision. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 11, 1973–1989.
    31. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176:1–176:9.
    32. Zhang, F.-L., Cheng, M.-M., Jia, J., and Hu, S.-M. 2012. Imageadmixture: Putting together dissimilar objects from groups. IEEE Transactions on Visualization and Computer Graphics 18, 11, 1849–1857.
    33. Zimmer, H., Bruhn, A., and Weickert, J. 2011. Optic flow in harmony. Int. J. Comput. Vision 93, 3, 368–388.

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org