“Domain transform for edge-aware image and video processing” by Gastal and Oliveira

  • ©Eduardo Simões Lopes Gastal and Manuel M. Oliveira

Conference:


Type:


Title:

    Domain transform for edge-aware image and video processing

Presenter(s)/Author(s):



Abstract:


    We present a new approach for performing high-quality edge-preserving filtering of images and videos in real time. Our solution is based on a transform that defines an isometry between curves on the 2D image manifold in 5D and the real line. This transform preserves the geodesic distance between points on these curves, adaptively warping the input signal so that 1D edge-preserving filtering can be efficiently performed in linear time. We demonstrate three realizations of 1D edge-preserving filters, show how to produce high-quality 2D edge-preserving filters by iterating 1D-filtering operations, and empirically analyze the convergence of this process. Our approach has several desirable features: the use of 1D operations leads to considerable speedups over existing techniques and potential memory savings; its computational cost is not affected by the choice of the filter parameters; and it is the first edge-preserving filter to work on color images at arbitrary scales in real time, without resorting to subsampling or quantization. We demonstrate the versatility of our domain transform and edge-preserving filters on several real-time image and video processing tasks including edge-preserving filtering, depth-of-field effects, stylization, recoloring, colorization, detail enhancement, and tone mapping.

References:


    1. Adams, A., Gelfand, N., Dolson, J., and Levoy, M. 2009. Gaussian kd-trees for fast high-dimensional filtering. In SIGGRAPH. Google Scholar
    2. Adams, A., Baek, J., and Davis, M. A. 2010. Fast high-dimensional filtering using the permutohedral lattice. CGF 29, 2, 753–762.Google ScholarCross Ref
    3. Adobe Systems Inc., 2010. Photoshop CS5. Computer software.Google Scholar
    4. Barash, D. 2002. A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE TPAMI 24, 844–847. Google ScholarDigital Library
    5. Belkin, M., and Niyogi, P. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 6 (June), 1373–1396. Google ScholarDigital Library
    6. Black, M., Sapiro, G., Marimont, D., and Heeger, D. 1998. Robust anisotropic diffusion. IEEE TIP 7, 3, 421–432. Google Scholar
    7. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM TOG 26, 3, 103. Google ScholarDigital Library
    8. Criminisi, A., Sharp, T., Rother, C., and P’erez, P. 2010. Geodesic image and video editing. ACM TOG 29, 5, 134. Google ScholarDigital Library
    9. D’Almeida, F., 2004. Nonlinear Diff. Toolbox (mathworks.com/matlabcentral/fileexchange/3710-nonlinear-diffusion-toolbox).Google Scholar
    10. Dougherty, E. 1994. Digital Image Processing Methods. Optical engineering. CRC Press. Google Scholar
    11. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In SIGGRAPH ’02, 257–266. Google Scholar
    12. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM TOG 27, 3, 67. Google ScholarDigital Library
    13. Farbman, Z., Fattal, R., and Lischinski, D. 2010. Diffusion maps for edge-aware image editing. ACM TOG 29, 6, 145. Google ScholarDigital Library
    14. Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM TOG 28, 3, 22. Google ScholarDigital Library
    15. Gastal, E. S. L., and Oliveira, M. M. 2010. Shared sampling for real-time alpha matting. CGF 29, 2, 575–584.Google ScholarCross Ref
    16. Grewenig, S., Weickert, J., and Bruhn, A. 2010. From box filtering to fast explicit diffusion. Pattern Recognition, 533–542. Google Scholar
    17. Kimball, S., Mattis, P., and GIMP Development Team, 2011. GNU Image Manipulation Program. Computer software.Google Scholar
    18. Kimmel, R., Sochen, N. A., and Malladi, R. 1997. From high energy physics to low level vision. In Scale-Space Theory in Computer Vision, Springer-Verlag, 236–247. Google Scholar
    19. Knutsson, H., and Westin, C.-F. 1993. Normalized and differential convolution: Methods for interpolation and filtering of incomplete and uncertain data. In CVPR, 515–523.Google Scholar
    20. Lee, J., and Verleysen, M. 2010. Nonlinear Dimensionality Reduction. Springer. Google Scholar
    21. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM TOG 23, 689–694. Google ScholarDigital Library
    22. Lévy, B., Petitjean, S., Ray, N., and Maillo t, J. 2002. Least squares conformal maps for automatic texture atlas generation. In ACM SIGGRAPH, 362–371. Google Scholar
    23. Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM TOG 25, 3, 646–653. Google ScholarDigital Library
    24. Loeve, M. 1977. Probability Theory, vol. 1. Springer.Google Scholar
    25. Oliveira, M. M., Bishop, G., and McAllister, D. 2000. Relief texture mapping. In ACM SIGGRAPH, 359–368. Google Scholar
    26. O’Neill, B. 2006. Elementary Differential Geometry. AP.Google Scholar
    27. Paris, S., and Durand, F. 2009. A fast approximation of the bilateral filter using a signal processing approach. IJCV 81, 1, 24–52. Google ScholarDigital Library
    28. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE TPAMI 12, 7, 629–639. Google ScholarDigital Library
    29. Pham, T., and van Vliet, L. 2005. Separable bilateral filtering for fast video preprocessing. IEEE Intl. Conf. on Multimedia and Expo 0, 4 pp.Google Scholar
    30. Piroddi, R., and Petrou, M. 2004. Analysis of irregularly sampled data: A review. Advances in Imaging and Electron Physics 132, 109–165.Google ScholarCross Ref
    31. Porikli, F. 2008. Constant time O(1) bilateral filtering. In CVPR, 1–8.Google Scholar
    32. Smith, A. R. 1987. Planar 2-pass texture mapping and warping. In Proc. SIGGRAPH 87, 263–272. Google ScholarDigital Library
    33. Smith, J. O. 2007. Introduction to Digital Filters with Audio Applications. W3K Publishing.Google Scholar
    34. Sochen, N., Kimmel, R., and Bruckstein, A. 2001. Diffusions and confusions in signal and image processing. Journal of Mathematical Imaging and Vision 14, 3, 195–209. Google ScholarCross Ref
    35. Subr, K., Soler, C., and Durand, F. 2009. Edge-preserving multiscale image decomposition based on local extrema. ACM TOG 28, 147:1–147:9. Google Scholar
    36. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In ICCV, 839–846. Google Scholar
    37. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing 13, 4, 600–612. Google ScholarDigital Library
    38. Weickert, J., Romeny, B., and Viergever, M. 1998. Efficient and reliable schemes for nonlinear diffusion filtering. IEEE TIP 7, 3, 398–410. Google Scholar
    39. Yang, Q., Tan, K. H., and Ahuja, N. 2009. Real-time O(1) bilateral filtering. In CVPR, 557–564.Google Scholar


ACM Digital Library Publication:



Overview Page: