“An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition”

  • ©Sai Bi, Xiaoguang Han, and Yizhou Yu




    An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition

Session/Category Title: Image Processing




    Identifying sparse salient structures from dense pixels is a longstanding problem in visual computing. Solutions to this problem can benefit both image manipulation and understanding. In this paper, we introduce an image transform based on the L1 norm for piecewise image flattening. This transform can effectively preserve and sharpen salient edges and contours while eliminating insignificant details, producing a nearly piecewise constant image with sparse structures. A variant of this image transform can perform edge-preserving smoothing more effectively than existing state-of-the-art algorithms. We further present a new method for complex scene-level intrinsic image decomposition. Our method relies on the above image transform to suppress surface shading variations, and perform probabilistic reflectance clustering on the flattened image instead of the original input image to achieve higher accuracy. Extensive testing on the Intrinsic-Images-in-the-Wild database indicates our method can perform significantly better than existing techniques both visually and numerically. The obtained intrinsic images have been successfully used in two applications, surface retexturing and 3D object compositing in photographs.


    1. An, X., and Pellacini, F. 2008. Appprop: all-pairs appearance-space edit propagation. ACM Trans. Graph. 27, 3, 40. Google ScholarDigital Library
    2. Bao, L., Song, Y., Yang, Q., Yuan, H., and Wang, G. 2014. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Transactions on Image Processing 23, 2, 555–569. Google ScholarDigital Library
    3. Barron, J. T., and Malik, J. 2012. Color constancy, intrinsic images, and shape estimation. In 12th European Conference on Computer Vision, 57–70. Google ScholarDigital Library
    4. Barron, J. T., and Malik, J. 2012. Shape, albedo, and illumination from a single image of an unknown object. CVPR. Google ScholarDigital Library
    5. Barrow, H. G., and Tenenbaum, J. M. 1978. Recovering intrinsic scene characteristics from images. In CVS78, 3–26.Google Scholar
    6. Bell, S., Bala, K., and Snavely, N. 2014. Intrinsic images in the wild. ACM Trans. Graph 33, 4, 159. Google ScholarDigital Library
    7. Blei, D. M., and Jordan, M. I. 2006. Variational inference for dirichlet process mixtures. Bayesian Analysis 1, 1, 121–143.Google ScholarCross Ref
    8. Bonneel, N., Sunkavalli, K., Tompkin, J., Sun, D., Paris, S., and Pfister, H. 2014. Interactive intrinsic video editing. ACM Trans. Graph 33, 6, 197. Google ScholarDigital Library
    9. Bousseau, A., Paris, S., and Durand, F. 2009. User-assisted intrinsic images. ACM Trans. Graph 28, 5. Google ScholarDigital Library
    10. Chang, J., Cabezas, R., and III, J. F. 2014. Bayesian nonparametric intrinsic image decomposition. In European Conference on Computer Vision.Google ScholarCross Ref
    11. Chen, Q., and Koltun, V. 2013. A simple model for intrinsic image decomposition with depth cues. In ICCV, IEEE, 241–248. Google ScholarDigital Library
    12. Donoho, D., and Logan, B. 1992. Signal recovery and the large sieve. SIAM J. Appl. Math. 52, 2, 577–591. Google ScholarDigital Library
    13. Donoho, D., and Stark, P. 1989. Uncertainty principles and signal recovery. SIAM J. Appl. Math. 49, 3, 906–931. Google ScholarDigital Library
    14. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics 27, 3 (Aug.), 67:1–67:?? Google ScholarDigital Library
    15. Farbman, Z., Fattal, R., and Lischinski, D. 2010. Diffusion maps for edge-aware image editing. ACM Trans. Graph 29, 6, 145. Google ScholarDigital Library
    16. Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM Trans. Graph 28, 3. Google ScholarDigital Library
    17. Felzenszwalb, P. F., and Huttenlocher, D. P. 2004. Efficient graph-based image segmentation. International Journal of Computer Vision 59, 2 (Sept.), 167–181. Google ScholarDigital Library
    18. Ferguson, T. S. 1973. A bayesian analysis of some nonparametric problems. The Annals of Statistics 1, 2, 209–230.Google ScholarCross Ref
    19. Garces, E., Muñoz, A., Lopez-Moreno, J., and Gutierrez, D. 2012. Intrinsic images by clustering. Comput. Graph. Forum 31, 4, 1415–1424. Google ScholarDigital Library
    20. Gehler, P. V., Rother, C., Kiefel, M., Zhang, L., and Schölkopf, B. 2011. Recovering intrinsic images with a global sparsity prior on reflectance. In 25th Annual Conference on Neural Information Processing Systems., J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. C. N. Pereira, and K. Q. Weinberger, Eds., 765–773.Google Scholar
    21. Goldstein, T., and Osher, S. J. 2009. The split bregman method for L1-regularized problems. Journal on Imaging Sciences 2, 2, 323–343. Google ScholarDigital Library
    22. Grosse, R. B., Johnson, M. K., Adelson, E. H., and Freeman, W. T. 2009. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV, IEEE, 2335–2342.Google Scholar
    23. Karsch, K., Hedau, V., Forsyth, D., and Hoiem, D. 2011. Rendering synthetic objects into legacy photographs. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA ’11, 157:1–157:12. Google ScholarDigital Library
    24. Karsch, K., Sunkavalli, K., Hadap, S., Carr, N., Jin, H., Fonte, R., Sittig, M., and Forsyth, D. A. 2014. Automatic scene inference for 3d object compositing. ACM Trans. Graph., 32. Google ScholarDigital Library
    25. Krähenbühl, P., and Koltun, V. 2013. Parameter learning and convergent inference for dense random fields. In International Conference on Machine Learning.Google Scholar
    26. Laffont, P.-Y., Bousseau, A., Paris, S., Durand, F., and Drettakis, G. 2012. Coherent intrinsic images from photo collections. ACM Trans. Graph 31, 6, 202. Google ScholarDigital Library
    27. Land, E. H., and McCann, J. J. 1971. Lightness and retinex theory. Journal of the Optical Society of America 61, 1, 1–11.Google ScholarCross Ref
    28. Min, D., Choi, S., Lu, J., Ham, B., Sohn, K., and Do, M. N. 2014. Fast global image smoothing based on weighted least squares. IEEE Transactions on Image Processing 23, 12, 5638–5653.Google ScholarCross Ref
    29. Omer, I., and Werman, M. 2004. Color lines: Image specific color representation. In CVPR (2), 946–953. Google ScholarDigital Library
    30. Paris, S., Hasinoff, S., and Kautz, J. 2011. Local laplacian filters: Edge-aware image processing with a laplacian pyramid. ACM Transactions on Graphics (Aug.). Google ScholarDigital Library
    31. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 7, 629639. Google ScholarDigital Library
    32. Rudin, L. I., Osher, S. J., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena 60, 259–268. Google ScholarDigital Library
    33. Shen, L., and Yeo, C. 2011. Intrinsic images decomposition using a local and global sparse representation of reflectance. In CVPR, IEEE Computer Society, 697–704. Google ScholarDigital Library
    34. Shen, L., Tan, P., and Lin, S. 2008. Intrinsic image decomposition with non-local texture cues. In CVPR, 1–7.Google Scholar
    35. Shen, J., Yang, X., Jiang, Y., and Li, X. 2011. Intrinsic images using optimization. Institute of Electrical and Electronics Engineers.Google Scholar
    36. Tappen, M., Freeman, W., and Adelson, E. 2005. Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell 27, 1459–1472. Google ScholarDigital Library
    37. Tomasi. 1998. Bilateral filtering for gray and color images. In ICCV, 839–846. Google ScholarDigital Library
    38. Tu, Z. W. 2005. Probabilistic boosting-tree: Learning discriminative models for classification, recognition, and clustering. In ICCV, II: 1589–1596. Google ScholarDigital Library
    39. Xu, L., Lu, C., Xu, Y., and Jia, J. 2011. Image smoothing via L0 gradient minimization. ACM Transactions on Graphics 30, 6 (Dec.), 174:1–174:?? Google ScholarDigital Library
    40. Zabih, R., Veksler, O., and Boykov, Y. Y. 1999. Fast approximate energy minimization via graph cuts. In ICCV, 377–384.Google Scholar
    41. Zhao, Q., Tan, P., Dai, Q., Shen, L., Wu, E., and Lin, S. 2012. A closed-form solution to retinex with nonlocal texture constraints. IEEE Trans. Pattern Anal. Mach. Intell 34, 7, 1437–1444. Google ScholarDigital Library

ACM Digital Library Publication:

Overview Page: