“Quality prediction for image completion” by Kopf, Kienzle, Drucker and Kang
Conference:
Type(s):
Title:
- Quality prediction for image completion
Session/Category Title: Operating on Images
Presenter(s)/Author(s):
Abstract:
We present a data-driven method to predict the quality of an image completion method. Our method is based on the state-of-the-art non-parametric framework of Wexler et al. [2007]. It uses automatically derived search space constraints for patch source regions, which lead to improved texture synthesis and semantically more plausible results. These constraints also facilitate performance prediction by allowing us to correlate output quality against features of possible regions used for synthesis. We use our algorithm to first crop and then complete stitched panoramas. Our predictive ability is used to find an optimal crop shape before the completion is computed, potentially saving significant amounts of computation. Our optimized crop includes as much of the original panorama as possible while avoiding regions that can be less successfully filled in. Our predictor can also be applied for hole filling in the interior of images. In addition to extensive comparative results, we ran several user studies validating our predictive feature, good relative quality of our results against those of other state-of-the-art algorithms, and our automatic cropping algorithm.
References:
1. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. on Graphics (Proceedings of Siggraph) 28, 24:1–24:11.
2. Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. 2000. Image inpainting. ACM Trans. on Graphics (Proceedings of Siggraph) 19.
3. Bornard, R., Lecan, E., Laborelli, L., and Chenot, J.-H. 2002. Missing data correction in still images and image sequences. Proceedings of the tenth ACM international conference on Multimedia, 355–361.
4. Criminisi, A., Perez, P., and Toyama, K. 2003. Object removal by exemplar-based inpainting. In CVPR, 417–424.
5. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image melding: Combining inconsistent images using patch-based synthesis. ACM Trans. on Graphics (Proceedings of Siggraph) 31, 4.
6. Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Trans. on Graphics (Proceedings of Siggraph) 22, 303–312.
7. Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In CVPR, 1033–1038.
8. Felzenszwalb, P., and Huttenlocher, D. 2004. Efficient graph-based image segmentation. IJCV 59, 2, 167–181.
9. Friedman, J., Hastie, T., and Tibshirani, R. 2000. Additive logistic regression: a statistical view of boosting. Annals of Statistics 28, 2, 337–407.
10. Galassi, M., Davies, J., Theiler, J., Gough, B., and Jungman, G. 2009. GNU Scientific Library — Reference Manual, Third Edition. Network Theory Ltd.
11. Jia, J., and Tang, C.-K. 2003. Image repairing: robust image synthesis by adaptive nd tensor voting. Proc. CVPR 2003, 643–650.
12. Kawai, N., Sato, T., and Yokoya, N. 2008. Image inpainting considering brightness change and spatial locality of textures and its evaluation. In PSIVT ’09, 271–282.
13. Komodakis, N., and Tziritas, G. 2007. Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. Image Processing 16, 2649–2661.
14. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Trans. on Graphics (Proceedings of Siggraph) 24, 795–802.
15. Mansfield, A., Prasad, M., Rother, C., Sharp, T., Kohli, P., and Van Gool, L. 2011. Transforming image completion. In British Machine Vision Conf. (BMVC).
16. Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28, 7 (July), 1150–1163.
17. Nelder, J., and Mead, R. 1965. A simplex method for function minimization. Computer Journal 7, 308–313.
18. Nocedal, J., and Wright, S. J. 2000. Numerical Optimization. Springer.
19. Pérez, P., Gangnet, M., and Blake, A. 2004. Patchworks: example-based region tiling for image editing. Tech. Rep. MSR-TR-2004-04, Microsoft Research.
20. Platt, J. C. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in large margin classifiers, MIT Press, 61–74.
21. Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In ICCV’09, 151–158.
22. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In CVPR.
23. Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Trans. on Graphics (Proceedings of Siggraph) 24, 861–868.
24. Swamy, D., Chandler, D., Butler, K., and Hemami, S. 2011. Parametric quality assessment of synthesized textures. Proc. Human Vision and Electronic Imaging 2011.
25. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time video completion. TPAMI.
26. Yu, S., and Shi, J. 2003. Multiclass spectral clustering. 313–319 vol. 1.
27. Zhang, M., Zhang, L., Sun, Y., Feng, L., and Ma, W. 2005. Auto cropping for digital photographs. In ICME.
