“Image completion using planar structure guidance” by Huang, Kang, Ahuja and Kopf

  • ©Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf

Conference:


Type:


Title:

    Image completion using planar structure guidance

Session/Category Title: Image Tricks

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We propose a method for automatically guiding patch-based image completion using mid-level structural cues. Our method first estimates planar projection parameters, softly segments the known region into planes, and discovers translational regularity within these planes. This information is then converted into soft constraints for the low-level completion algorithm by defining prior probabilities for patch offsets and transformations. Our method handles multiple planes, and in the absence of any detected planes falls back to a baseline fronto-parallel image completion algorithm. We validate our technique through extensive comparisons with state-of-the-art algorithms on a variety of scenes.

References:


    1. Aiger, D., Cohen-Or, D., and Mitra, N. J. 2012. Repetition maximization based texture rectification. Computer Graphics Forum (EUROGRAPHICS) 31, 2pt2, 439–448. Google ScholarDigital Library
    2. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., and Verdera, J. 2001. Filling-in by joint interpolation of vector fields and gray levels. IEEE TIP 10, 8, 1200–1211. Google ScholarDigital Library
    3. Barinova, O., Konushin, V., Yakubenko, A., Lee, K., Lim, H., and Konushin, A. 2008. Fast automatic single-view 3-d reconstruction of urban scenes. In ECCV. Google ScholarDigital Library
    4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. 2009. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. on Graphics (Proc. of Siggraph) 28, 3, 24. Google ScholarDigital Library
    5. Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. 2000. Image inpainting. ACM Trans. on Graphics (Proc. of Siggraph) 19, 3, 417–424. Google ScholarDigital Library
    6. Bertalmio, M., Vese, L., Sapiro, G., and Osher, S. 2003. Simultaneous structure and texture image inpainting. IEEE TIP 12, 8, 882–889. Google ScholarDigital Library
    7. Chum, O., and Matas, J. 2010. Planar affine rectification from change of scale. In ACCV. Google ScholarDigital Library
    8. Comaniciu, D., and Meer, P. 2002. Mean Shift: A robust approach toward feature space analysis. IEEE TPAMI 24, 5, 603–619. Google ScholarDigital Library
    9. Criminisi, A., Pérez, P., and Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE TIP 13, 9, 1200–1212. Google ScholarDigital Library
    10. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image Melding: Combining Inconsistent Images using Patch-based Synthesis. ACM Trans. on Graphics (Proc. of Siggraph) 31, 4. Google ScholarDigital Library
    11. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. ACM Trans. on Graphics (Proc. of Siggraph) 20, 3, 341–346. Google ScholarDigital Library
    12. Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV. Google ScholarDigital Library
    13. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press. Google ScholarDigital Library
    14. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Trans. on Graphics (Proc. of Siggraph) 26, 3, 4. Google ScholarDigital Library
    15. He, K., and Sun, J. 2012. Statistics of patch offsets for image completion. In ECCV. Google ScholarDigital Library
    16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. ACM Trans. on Graphics (Proc. of Siggraph) 20, 3, 327–340. Google ScholarDigital Library
    17. Huang, H., K. Yin, Gong, M., Lischinski, D., Cohen-Or, D., Ascher, U., and Chen, B. 2013. Mind the gap: Tele-registration for structure-driven image completion. ACM Trans. on Graphics (Proc. of Siggraph Asia) 32, 174:1–174:10. Google ScholarDigital Library
    18. Huang, J.-B., Kopf, J., Ahuja, N., and Kang, S. B. 2013. Transformation guided image completion. In ICCP.Google Scholar
    19. Jia, J., and Tang, C. 2003. Image repairing: Robust image synthesis by adaptive nd tensor voting. In CVPR. Google ScholarDigital Library
    20. Komodakis, N., and Tziritas, G. 2007. Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE TIP 16, 11, 2649–2661. Google ScholarDigital Library
    21. Kopf, J., Kienzle, W., Drucker, S., and Kang, S. B. 2012. Quality prediction for image completion. ACM Trans. on Graphics (Proc. of Siggraph Asia) 31, 6. Google ScholarDigital Library
    22. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Trans. on Graphics (Proc. of Siggraph) 24, 3, 795–802. Google ScholarDigital Library
    23. Liu, Y., Lin, W.-C., and Hays, J. 2004. Near-regular texture analysis and manipulation. ACM Trans. on Graphics (Proc. of Siggraph) 23, 3, 368–376. Google ScholarDigital Library
    24. Liu, Y., Hel-Or, H., and Kaplan, C. 2010. Computational symmetry in computer vision and computer graphics. Now Publishers.Google Scholar
    25. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91–110. Google ScholarDigital Library
    26. Mansfield, A., Prasad, M., Rother, C., Sharp, T., Kohli, P., and Van Gool, L. 2011. Transforming image completion. In BMVC.Google Scholar
    27. Pavić, D., Schönefeld, V., and Kobbelt, L. 2006. Interactive image completion with perspective correction. The Visual Computer 22, 9, 671–681. Google ScholarDigital Library
    28. Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In ICCV.Google Scholar
    29. Sun, J., Yuan, L., Jia, J., and Shum, H. 2005. Image completion with structure propagation. ACM Trans. on Graphics (Proc. of Siggraph) 24, 3, 861–868. Google ScholarDigital Library
    30. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE TPAMI 29, 3, 463–476. Google ScholarDigital Library
    31. Whyte, O., Sivic, J., and Zisserman, A. 2009. Get out of my picture! internet-based inpainting. In BMVC.Google Scholar
    32. Zhang, Z., Ganesh, A., Liang, X., and Ma, Y. 2012. TILT: transform invariant low-rank textures. International Journal of Computer Vision 99, 1, 1–24. Google ScholarDigital Library
    33. Zhang, Y., Xiao, J., Hays, J., and Tan, P. 2013. FrameBreak: Dramatic image extrapolation by guided shift-maps. In CVPR. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: