“Regional foremost matching for internet scene images”
Conference:
Type(s):
Title:
- Regional foremost matching for internet scene images
Session/Category Title: All About Seeing
Presenter(s)/Author(s):
Abstract:
We analyze the dense matching problem for Internet scene images based on the fact that commonly only part of images can be matched due to the variation of view angle, motion, objects, etc. We thus propose regional foremost matching to reject outlier matching points while still producing dense high-quality correspondence in the remaining foremost regions. Our system initializes sparse correspondence, propagates matching with model fitting and optimization, and detects foremost regions robustly. We apply our method to several applications, including time-lapse sequence generation, Internet photo composition, automatic image morphing, and automatic rephotography.
References:
1. Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., and Szeliski, R. 2009. Building rome in a day. In ICCV, 72–79.
2. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S. M., Colburn, A., Curless, B., Salesin, D., and Cohen, M. F. 2004. Interactive digital photomontage. ACM Trans. Graph. 23, 3, 294–302.
3. Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., and Silva, C. T. 2003. Computing and rendering point set surfaces. IEEE Transactions on Visualization and Computer Graphics 9, 1, 3–15.
4. Bailer, C., Taetz, B., and Stricker, D. 2015. Flow fields: Dense correspondence fields for highly accurate large displacement optical flow estimation. In ICCV, 4015–4023.
5. Bao, L., Yang, Q., and Jin, H. 2014. Fast edge-preserving patchmatch for large displacement optical flow. IEEE Trans. Image Processing 23, 12, 4996–5006. Cross Ref
6. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3.
7. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized patchmatch correspondence algorithm. In ECCV, 29–43.
8. Bay, H., Tuytelaars, T., and Gool, L. J. V. 2006. Surf: Speeded up robust features. In ECCV, 404–417.
9. Black, M. J., and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63, 1, 75–104.
10. Brox, T., and Malik, J. 2011. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 3, 500–513.
11. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV, 25–36.
12. Bruhn, A., and Weickert, J. 2005. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In ICCV, 749–755.
13. Chen, T., Cheng, M., Tan, P., Shamir, A., and Hu, S. 2009. Sketch2photo: internet image montage. ACM Trans. Graph. 28, 5.
14. Chia, A. Y. S., Zhuo, S., Gupta, R. K., Tai, Y., Cho, S., Tan, P., and Lin, S. 2011. Semantic colorization with internet images. ACM Trans. Graph. 30, 6, 156.
15. Cho, M., Shin, Y. M., and Lee, K. M. 2008. Co-recognition of image pairs by data-driven monte carlo image exploration. In ECCV, 144–157.
16. Dale, K., Johnson, M. K., Sunkavalli, K., Matusik, W., and Pfister, H. 2009. Image restoration using online photo collections. In ICCV, 2217–2224.
17. Dosovitskiy, A., Springenberg, J. T., Riedmiller, M. A., and Brox, T. 2014. Discriminative unsupervised feature learning with convolutional neural networks. In NIPS, 766–774.
18. Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2010. Towards internet-scale multi-view stereo. In CVPR, 1434–1441.
19. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4, 70.
20. HaCohen, Y., Shechtman, E., and Lischinski, D. 2013. Deblurring by example using dense correspondence. In ICCV, 2384–2391.
21. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Trans. Graph. 26, 3, 4.
22. Horn, B. K. P., and Schunck, B. G. 1981. Determining optical flow. Artif. Intell. 17, 1-3, 185–203.
23. Johnson, M. K., Dale, K., Avidan, S., Pfister, H., Freeman, W. T., and Matusik, W. 2011. Cg2real: Improving the realism of computer generated images using a large collection of photographs. IEEE Transactions on Visualization and Computer Graphics 17, 9, 1273–1285.
24. Joshi, N., Matusik, W., Adelson, E. H., and Kriegman, D. J. 2010. Personal photo enhancement using example images. ACM Trans. Graph. 29, 2.
25. Kim, J., Liu, C., Sha, F., and Grauman, K. 2013. Deformable spatial pyramid matching for fast dense correspondences. In CVPR, 2307–2314.
26. Krähenbühl, P., and Koltun, V. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. In NIPS, 109–117.
27. Liao, J., Lima, R. S., Nehab, D., Hoppe, H., Sander, P. V., and Yu, J. 2014. Automating image morphing using structural similarity on a halfway domain. ACM Trans. Graph. 33, 5, 168:1–168:12.
28. Lipman, Y., Yagev, S., Poranne, R., Jacobs, D. W., and Basri, R. 2014. Feature matching with bounded distortion. ACM Trans. Graph. 33, 3, 26.
29. Liu, C., Yuen, J., and Torralba, A. 2011. Sift flow: Dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33, 5, 978–994.
30. Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Trans. Graph. 30, 1, 4.
31. Liu, S., Yuan, L., Tan, P., and Sun, J. 2013. Bundled camera paths for video stabilization. ACM Trans. Graph. 32, 4.
32. Liu, Z., Yuan, L., Tang, X., Uyttendaele, M., and Sun, J. 2014. Fast burst images denoising. ACM Trans. Graph. 33, 6, 232.
33. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal on Computer Vision 60, 2, 91–110.
34. Martin-Brualla, R., Gallup, D., and Seirz, S. M. 2015. Time-lapse mining from internet photos. In ACM SIGGRAPH, vol. 34, 62.
35. Martin-Brualla, R., Gallup, D., and Seitz, S. M. 2015. 3d time-lapse reconstruction from internet photos. In ICCV, 1332–1340.
36. Noah, S. 2011. Scene reconstruction and visualization from internet photo collections: A survey. Transactions on Computer Vision and Applications 3, 44–66. Cross Ref
37. Revaud, J., Weinzaepfel, P., Harchaoui, Z., and Schmid, C. 2015. Epicflow: Edge-preserving interpolation of correspondences for optical flow. In CVPR, 1164–1172.
38. Revaud, J., Weinzaepfel, P., Harchaoui, Z., and Schmid, C. 2016. Deepmatching: Hierarchical deformable dense matching. CoRR abs/1506.07656.
39. Rhemann, C., Hosni, A., Bleyer, M., Rother, C., and Gelautz, M. 2011. Fast cost-volume filtering for visual correspondence and beyond. In CVPR, 3017–3024.
40. Sen, P., Kalantari, N. K., Yaesoubi, M., Darabi, S., Goldman, D. B., and Shechtman, E. 2012. Robust patch-based hdr reconstruction of dynamic scenes. ACM Trans. Graph. 31, 6, 203.
41. Shen, L., and Tan, P. 2009. Photometric stereo and weather estimation using internet images. In CVPR, 1850–1857.
42. Shepard, D. 1968. A two-dimensional interpolation function for irregularly-spaced data. In Proceedings of the 23rd ACM National Conference, ACM ’68, 517–524.
43. Shi, B., Inose, K., Matsushita, Y., Tan, P., Yeung, S., and Ikeuchi, K. 2014. Photometric stereo using internet images. In International Conference on 3D Vision, 361–368.
44. Shih, Y., Paris, S., Barnes, C., Freeman, W. T., and Durand, F. 2014. Style transfer for headshot portraits. ACM Trans. Graph. 33, 4, 148:1–148:14.
45. Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., and Moreno-Noguer, F. 2015. Discriminative learning of deep convolutional feature point descriptors. In ICCV, 118–126.
46. Simonyan, K., Vedaldi, A., and Zisserman, A. 2014. Learning local feature descriptors using convex optimisation. IEEE Trans. Pattern Anal. Mach. Intell. 36, 8, 1573–1585.
47. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835–846.
48. Snavely, N., Seitz, S. M., and Szeliski, R. 2008. Modeling the world from internet photo collections. International Journal of Computer Vision 80, 2, 189–210.
49. Tau, M., and Hassner, T. 2016. Dense correspondences across scenes and scales. IEEE Trans. Pattern Anal. Mach. Intell. 38, 5, 875–888.
50. Tola, E., V.Lepetit, and Fua, P. 2008. A fast local descriptor for dense matching. In CVPR.
51. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In ICCV, 839–846.
52. Wasserman, L. 2013. All of statistics: a concise course in statistical inference. Springer.
53. Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., and Cremers, D. 2008. Efficient dense scene flow from sparse or dense stereo data. In ECCV, 739–751.
54. Weinzaepfel, P., Revaud, J., Harchaoui, Z., and Schmid, C. 2013. DeepFlow: Large displacement optical flow with deep matching. In ICCV, 1385–1392.
55. Xu, L., Jia, J., and Matsushita, Y. 2012. Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 9, 1744–1757.
56. Yan, Z., Zhang, H., Wang, B., Paris, S., and Yu, Y. 2016. Automatic photo adjustment using deep neural networks. ACM Trans. Graph. 35, 2, 11.
57. Yang, H., Lin, W., and Lu, J. 2014. DAISY filter flow: A generalized discrete approach to dense correspondences. In CVPR, 3406–3413.
58. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176.
59. Zach, C., Pock, T., and Bischof, H. 2007. A duality based approach for realtime TV-L1 optical flow. Pattern Recognition, 214–223.
60. Zaragoza, J., Chin, T., Tran, Q., Brown, M. S., and Suter, D. 2014. As-projective-as-possible image stitching with moving DLT. IEEE Trans. Pattern Anal. Mach. Intell. 36, 7, 1285–1298.
61. Zhang, C., Gao, J., Wang, O., Georgel, P., Yang, R., Davis, J., Frahm, J., and Pollefeys, M. 2014. Personal photograph enhancement using internet photo collections. IEEE Transactions on Visualization and Computer Graphics 20, 2, 262–275.
