“Domain transform for edge-aware image and video processing” by Gastal and Oliveira

We present a new approach for performing high-quality edge-preserving filtering of images and videos in real time. Our solution is based on a transform that defines an isometry between curves on the 2D image manifold in 5D and the real line. This transform preserves the geodesic distance between points on these curves, adaptively warping the input signal so that 1D edge-preserving filtering can be efficiently performed in linear time. We demonstrate three realizations of 1D edge-preserving filters, show how to produce high-quality 2D edge-preserving filters by iterating 1D-filtering operations, and empirically analyze the convergence of this process. Our approach has several desirable features: the use of 1D operations leads to considerable speedups over existing techniques and potential memory savings; its computational cost is not affected by the choice of the filter parameters; and it is the first edge-preserving filter to work on color images at arbitrary scales in real time, without resorting to subsampling or quantization. We demonstrate the versatility of our domain transform and edge-preserving filters on several real-time image and video processing tasks including edge-preserving filtering, depth-of-field effects, stylization, recoloring, colorization, detail enhancement, and tone mapping.

1. Adams, A., Gelfand, N., Dolson, J., and Levoy, M. 2009. Gaussian kd-trees for fast high-dimensional filtering. In SIGGRAPH. Google Scholar
2. Adams, A., Baek, J., and Davis, M. A. 2010. Fast high-dimensional filtering using the permutohedral lattice. CGF 29, 2, 753–762.Google ScholarCross Ref
3. Adobe Systems Inc., 2010. Photoshop CS5. Computer software.Google Scholar
4. Barash, D. 2002. A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE TPAMI 24, 844–847. Google ScholarDigital Library
5. Belkin, M., and Niyogi, P. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 6 (June), 1373–1396. Google ScholarDigital Library
6. Black, M., Sapiro, G., Marimont, D., and Heeger, D. 1998. Robust anisotropic diffusion. IEEE TIP 7, 3, 421–432. Google Scholar
7. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM TOG 26, 3, 103. Google ScholarDigital Library
8. Criminisi, A., Sharp, T., Rother, C., and P’erez, P. 2010. Geodesic image and video editing. ACM TOG 29, 5, 134. Google ScholarDigital Library
9. D’Almeida, F., 2004. Nonlinear Diff. Toolbox (mathworks.com/matlabcentral/fileexchange/3710-nonlinear-diffusion-toolbox).Google Scholar
10. Dougherty, E. 1994. Digital Image Processing Methods. Optical engineering. CRC Press. Google Scholar
11. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In SIGGRAPH ’02, 257–266. Google Scholar
12. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM TOG 27, 3, 67. Google ScholarDigital Library
13. Farbman, Z., Fattal, R., and Lischinski, D. 2010. Diffusion maps for edge-aware image editing. ACM TOG 29, 6, 145. Google ScholarDigital Library
14. Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM TOG 28, 3, 22. Google ScholarDigital Library
15. Gastal, E. S. L., and Oliveira, M. M. 2010. Shared sampling for real-time alpha matting. CGF 29, 2, 575–584.Google ScholarCross Ref
16. Grewenig, S., Weickert, J., and Bruhn, A. 2010. From box filtering to fast explicit diffusion. Pattern Recognition, 533–542. Google Scholar
17. Kimball, S., Mattis, P., and GIMP Development Team, 2011. GNU Image Manipulation Program. Computer software.Google Scholar
18. Kimmel, R., Sochen, N. A., and Malladi, R. 1997. From high energy physics to low level vision. In Scale-Space Theory in Computer Vision, Springer-Verlag, 236–247. Google Scholar
19. Knutsson, H., and Westin, C.-F. 1993. Normalized and differential convolution: Methods for interpolation and filtering of incomplete and uncertain data. In CVPR, 515–523.Google Scholar
20. Lee, J., and Verleysen, M. 2010. Nonlinear Dimensionality Reduction. Springer. Google Scholar
21. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM TOG 23, 689–694. Google ScholarDigital Library
22. Lévy, B., Petitjean, S., Ray, N., and Maillo t, J. 2002. Least squares conformal maps for automatic texture atlas generation. In ACM SIGGRAPH, 362–371. Google Scholar
23. Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM TOG 25, 3, 646–653. Google ScholarDigital Library
24. Loeve, M. 1977. Probability Theory, vol. 1. Springer.Google Scholar
25. Oliveira, M. M., Bishop, G., and McAllister, D. 2000. Relief texture mapping. In ACM SIGGRAPH, 359–368. Google Scholar
26. O’Neill, B. 2006. Elementary Differential Geometry. AP.Google Scholar
27. Paris, S., and Durand, F. 2009. A fast approximation of the bilateral filter using a signal processing approach. IJCV 81, 1, 24–52. Google ScholarDigital Library
28. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE TPAMI 12, 7, 629–639. Google ScholarDigital Library
29. Pham, T., and van Vliet, L. 2005. Separable bilateral filtering for fast video preprocessing. IEEE Intl. Conf. on Multimedia and Expo 0, 4 pp.Google Scholar
30. Piroddi, R., and Petrou, M. 2004. Analysis of irregularly sampled data: A review. Advances in Imaging and Electron Physics 132, 109–165.Google ScholarCross Ref
31. Porikli, F. 2008. Constant time O(1) bilateral filtering. In CVPR, 1–8.Google Scholar
32. Smith, A. R. 1987. Planar 2-pass texture mapping and warping. In Proc. SIGGRAPH 87, 263–272. Google ScholarDigital Library
33. Smith, J. O. 2007. Introduction to Digital Filters with Audio Applications. W3K Publishing.Google Scholar
34. Sochen, N., Kimmel, R., and Bruckstein, A. 2001. Diffusions and confusions in signal and image processing. Journal of Mathematical Imaging and Vision 14, 3, 195–209. Google ScholarCross Ref
35. Subr, K., Soler, C., and Durand, F. 2009. Edge-preserving multiscale image decomposition based on local extrema. ACM TOG 28, 147:1–147:9. Google Scholar
36. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In ICCV, 839–846. Google Scholar
37. Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing 13, 4, 600–612. Google ScholarDigital Library
38. Weickert, J., Romeny, B., and Viergever, M. 1998. Efficient and reliable schemes for nonlinear diffusion filtering. IEEE TIP 7, 3, 398–410. Google Scholar
39. Yang, Q., Tan, K. H., and Ahuja, N. 2009. Real-time O(1) bilateral filtering. In CVPR, 557–564.Google Scholar