“Automating Image Morphing Using Structural Similarity on a Halfway Domain” by Liao, Lima, Nehab, Hoppe, Sander, et al. …

  • ©Jing Liao, Rodolfo S. Lima, Diego Nehab, Hugues Hoppe, Pedro V. Sander, and Jinhui Yu

Conference:


Type:


Title:

    Automating Image Morphing Using Structural Similarity on a Halfway Domain

Session/Category Title: Image Tricks

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.

References:


    1. S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski. 2011. A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92, 1, 1–31.
    2. P. N. Belhumeur and D. Mumford. 1992. A bayesian treatment of the stereo correspondence problem using half-occluded regions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 506–512.
    3. D. Beymer. 1996. Feature correspondences by interleaving shape and texture computations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 921–928.
    4. M. Bichsel. 1996. Automatic interpolation and recognition of face images by morphing. In Proceedings of the International Conference on Automatic Face and Gesture Recognition. 128–135.
    5. M. Covell. 1996. Eigen-points: Control-point location using principal component analyses. In Proceedings of the International Conference on Automatic Face and Gesture Recognition. 122–127.
    6. S. Darabi, E. Shechtman, C. Barnes, D. B. Goldman, and P. Sen. 2012. Image melding: Combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31, 4, 82.
    7. M. Finch, J. Snyder, and H. Hoppe. 2011. Freeform vector graphics with controlled thin-plate splines. ACM Trans. Graph. 30, 6, 166.
    8. P. Gao and T. W. Sederberg. 1998. A work minimization approach to image morphing. The Visual Comput. 14, 8–9, 390–400.
    9. J. Gomes, L. Darsa, B. Costa, and L. Velho. 1999. Warping and Morphing of Graphical Objects. Morgan Kaufmann, San Fransisco.
    10. S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. 2003. High dynamic range video. ACM Trans. Graph. 22, 3, 319–325.
    11. M. Kass, A. Witkin, and D. Terzopoulos. 1988. Snakes: Active contour models. Int. J. Comput. Vis. 1, 4, 321–331.
    12. I. Kemelmacher-Shlizerman, E. Shechtman, R. Garg, and S. M. Seitz. 2011. Exploring photobios. ACM Trans. Graph. 30, 4, 61.
    13. J. Kiefer. 1953. Sequential minimax search for a maximum. Proc. Amer. Math. Soc. 4, 3, 502–506.
    14. S. Lee, K. Y. Chwa, J. K. Hahn, and S. Y. Shin. 1994. Image morphing using deformable surfaces. In Proceedings of the Conference on Computer Animation. 31–39.
    15. S. Lee, K. Y. Chwa, J. K. Hahn, and S. Y. Shin. 1996. Image morphing using deformation techniques. J. Visual. Comput. Animat. 7, 1, 3–23.
    16. Z. Liu, C. Liu, H. Shum, and Y. Yu. 2002. Pattern-based texture metamorphosis. In Proceedings of the Pacific Conference on Computer Graphics and Applications. 184–191.
    17. D. Mahajan, F.-C. Huang, W. Matusik, R. Ramamoorthi, and P. Belhumeur. 2009. Moving gradients: A path-based method for plausible image interpolation. ACM Trans. Graph. 28, 3, 42.
    18. W. Matusik, M. Zwicker, and F. Durand. 2005. Texture design using a simplicial complex of morphable textures. ACM Trans. Graph. 24, 3, 787–794.
    19. E. Mortensen and W. Barrett. 1995. Intelligent scissors for image composition. In Proceedings of the Annual ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques. 191–198.
    20. T. Nishita, T. Fujii, and E. Nakamae. 1993. Metamorphosis using Bezier clipping. In Proceedings of the Pacific Conference on Computer Graphics and Applications. 162–173.
    21. P. Perez, M. Gangnet, and A. Blake. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313–318.
    22. J. Rossignac and A. Vinacua. 2011. Steady affine motions and morphs. ACM Trans. Graph. 30, 5, 116.
    23. M. Ruzon and C. Tomasi. 2001. Edge, junction, and corner detection using color distributions. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11, 1281–1295.
    24. D. Scharstein and R. Szeliski. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 1, 7–42.
    25. E. Shechtman, A. Rav-Acha, M. Irani, and S. M. Seitz. 2010. Regenerative morphing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 615–622.
    26. H. Shum, S. Kang, and S. Chan. 2003. Survey of image-based representations and compression techniques. IEEE Trans. Circ. Syst. Video Technol. 13, 11, 1020–1037.
    27. D. B. Smythe. 1990. A two-pass mesh warping algorithm for object transformation and image interpolation. Tech. rep. 1030, ILM.
    28. Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4, 600–612.
    29. G. Wolberg. 1990. Digital Image Warping. IEEE Computer Society Press.
    30. G. Wolberg. 1998. Image morphing: A survey. The Visual Comput. 14, 8, 360–372.
    31. E. Wu and F. Liu. 2013. Robust image metamorphosis immune from ghost and blur. The Visual Comput. 29, 4, 311–321.
    32. F. Yang, E. Shechtman, J. Wang, L. Bourdev, and D. Metaxas. 2012. Face morphing using 3D-aware appearance optimization. In Proceedings of the Graphics Interface Conference. 93–99.
    33. L. Yang, Y.-C. Tse, P. V. Sander, J. D. Lawrence, D. Nehab, H. Hoppe, and C. L. Wilkins. 2011. Image-based bidirectional scene reprojection. ACM Trans. Graph. 30, 6, 150.
    34. K. Yucer, A. Jacobson, A. Hornung, and O. Sorkine. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176.
    35. Y. Zhang. 1996. A fuzzy approach to digital image warping. IEEE Comput. Graph. Appl. 16, 4, 34–41.
    36. B. Zitova and J. Flusser. 2003. Image registration methods: A survey. Image Vis. Comput. 21, 11, 977–1000.

ACM Digital Library Publication:


Overview Page: