“Image upsampling via imposed edge statistics” by Fattal

In this paper we propose a new method for upsampling images which is capable of generating sharp edges with reduced input-resolution grid-related artifacts. The method is based on a statistical edge dependency relating certain edge features of two different resolutions, which is generically exhibited by real-world images. While other solutions assume some form of smoothness, we rely on this distinctive edge dependency as our prior knowledge in order to increase image resolution. In addition to this relation we require that intensities are conserved; the output image must be identical to the input image when downsampled to the original resolution. Altogether the method consists of solving a constrained optimization problem, attempting to impose the correct edge relation and conserve local intensities with respect to the low-resolution input image. Results demonstrate the visual importance of having such edge features properly matched, and the method’s capability to produce images in which sharp edges are successfully reconstructed.

1. Aly, H., and Dubois, E. 2005. Image up-sampling using total-variation regularization with a new observation model. In IEEE Transactions on Image Processing, vol. 14, 1647–1659. Google ScholarDigital Library
2. Benzi, M., Golub, G. H., and Liesen, J. 2005. Numerical solution of saddle point problems. In Acta Numer., vol. 14, 1–137.Google ScholarCross Ref
3. Calle, D., and Montanvert, A. 1998. Super-resolution inducing of an image. In International Conference on Image Processing, vol. 3, 232–236.Google Scholar
4. Capel, D., and Zisserman, A. 1998. Automatic mosaicing with super-resolution zoom. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Washington, DC, USA, 885. Google ScholarDigital Library
5. Carrato, S., Ramponi, G., and Marsi, S. 1996. A simple edge-sensitive image interpolation filter. In Proceedings of the IEEE International Conference on Image Processing, 1996, vol. 3, 711–714.Google ScholarCross Ref
6. Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. In IEEE Computer Graphics and Applications, vol. 22, 56–65. Google ScholarDigital Library
7. Greenspan, H., Anderson, C. H., and Akber, S. 2000. Image enhancement by non-linear extrapolation in frequency space. In IEEE Transactions on Image Processing, vol. 9, 1035-1048-202. Google ScholarDigital Library
8. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In ACM Transactions on Graphics, ACM Press, New York, NY, USA, 327–340. Google ScholarDigital Library
9. Huang, J., and Mumford, D. 1999. Statistics of natural images and models. In Computer Vision and Pattern Recognition, 1541–1547.Google Scholar
10. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. In ACM Transactions on Graphphics, ACM Press, New York, NY, USA, vol. 24, 795–802. Google ScholarDigital Library
11. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. In ACM Transactions on Graphics, ACM Press, New York, NY, USA, vol. 23, 689–694. Google ScholarDigital Library
12. Li, X., and Orchard, M. T. 2001. New edge-directed interpolation. In IEEE Transactions on Image Processing, vol. 10, 1521–1527. Google ScholarDigital Library
13. Morse, B. S., and Schwartzwald, D. 2001. Image magnification using level-set reconstruction. In Computer Vision and Pattern Recognition, 333–340.Google Scholar
14. Osher, S., Sole, A., and Vese, L. 2003. Image decomposition and restoration using total variation minimization and the h
-1. Multiscale Modeling & Simulation 1, 3, 349–370.Google Scholar
15. Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P. 1993. Numerical Recipes in C: The Art of Scientific Computing, 2nd edition ed. Cambridge University Press, Cambridge, UK. Google ScholarDigital Library
16. Prez, P. 1998. Markov random fields and images. In CWI Quarterly, vol. 11, 413–437.Google Scholar
17. Ramanarayanan, G., Bala, K., and Walter, B. 2004. Feature-based textures. In Rendering Techniques, 265–274. Google ScholarCross Ref
18. Ratakonda, K., and Ahuja, N. 1998. Pocs based adaptive image magnification. In ICIP (3), 203–207.Google Scholar
19. Reinhard, E., Shirley, P., Ashikhmin, M., and Troscianko, T. 2004. Second order image statistics in computer graphics. In APGV ’04: Proceedings of the 1st Symposium on Applied perception in graphics and visualization, ACM Press, New York, NY, USA, 99–106. Google ScholarDigital Library
20. Schaaf, A. V. D. 1998. Natural Image Statistics and Visual Processing. PhD thesis, Rijksuniversiteit Groningen, The Netherlands.Google Scholar
21. Simoncelli, E. P. 1997. Statistical models for images: Compression, restoration and synthesis. In 31st Asilomar Conf on Signals, Systems and Computers, IEEE Computer Society, Pacific Grove, CA, 673–678.Google ScholarCross Ref
22. Su, D., and Willis, P. 2004. Image interpolation by pixel-level data-dependent triangulation. In Computer Graphics Forum, vol. 23, 189–202.Google ScholarCross Ref
23. Tappen, M. F., Russell, B. C., and Freeman, W. T. 2004. Efficient graphical models for processing images. In Computer Vision and Pattern Recognition, 673–680. Google ScholarDigital Library
24. Thvenaz, P., Blu, T., and Unser, M. 2000. Image interpolation and resampling. In Handbook of Medical Imaging, Processing and Analysis, Academic Press, San Diego CA, USA, I. Bankman, Ed., 393–420. Google ScholarDigital Library
25. Tumblin, J., and Choudhury, P. 2004. Bixels: Picture samples with sharp embedded boundaries. In Rendering Techniques, 255–264. Google ScholarDigital Library
26. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: From error measurement to structural similarity. IEEE Trans. Image Processing 13, 4, 600–612. Google ScholarDigital Library
27. Zomet, A., and Peleg, S. 2002. Multi-sensor super-resolution. In 6th IEEE Workshop on Applications of Computer Vision, 27–31. Google ScholarDigital Library