“Multiscale shape and detail enhancement from multi-light image collections” by Fattal, Agrawala and Rusinkiewicz

  • ©Raanan Fattal, Maneesh Agrawala, and Szymon Rusinkiewicz




    Multiscale shape and detail enhancement from multi-light image collections



    We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varying lighting conditions. For each image we compute a multiscale decomposition based on the bilateral filter and then reconstruct an enhanced image that combines detail information at each scale across all the input images. Our approach does not require any information about light source positions, or camera calibration, and can produce good results with 3 to 5 input images. In addition our system provides a few high-level parameters for controlling the amount of enhancement and does not require pixel-level user input. We show that the bilateral filter is a good choice for our multiscale algorithm because it avoids the halo artifacts commonly associated with the traditional Laplacian image pyramid. We also develop a new scheme for computing our multiscale bilateral decomposition that is simple to implement, fast O(N2 log N) and accurate.


    1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., AND Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 294–302. Google ScholarDigital Library
    2. Akers, D., Losasso, F., Klingner, J., Agrawala, M., Rick, J., and Hanrahan, P. 2003. Conveying shape and features with image-based relighting. In Proc. IEEE Visualization, 349–254. Google ScholarDigital Library
    3. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 637–645. Google ScholarDigital Library
    4. Barash, D., and Comaniciu, D. 2004. A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift. Journal of Image and Vision Computing 22, 1, 73–81.Google ScholarCross Ref
    5. Burt, P. J., and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Comm. 31, 4, 532–540.Google ScholarCross Ref
    6. Burt, P., and Kolczynski, R. 1993. Enhanced image capture through fusion. Proc. ICCV, 173–182.Google Scholar
    7. Burt, P. J. 1981. Fast filter transforms for image processing. Computer Graphics, Image Processing 6, 20–51.Google ScholarCross Ref
    8. Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. Eurographics Symposium on Rendering, 186–196. Google ScholarDigital Library
    9. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proc. SIGGRAPH, 145–156. Google ScholarDigital Library
    10. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In Proc. SIGGRAPH, 257–266. Google ScholarDigital Library
    11. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 673–678. Google ScholarDigital Library
    12. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proc. SIGGRAPH, 249–256. Google ScholarDigital Library
    13. Finalyson, G. D., Hordley, S. D., Lu, C., and Drew, M. S. 2006. On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1, 59–68. Google ScholarDigital Library
    14. Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, P. 1989. A real-time algorithms for signal analysis with the help of the wavelet transform. Wavelets, Time-Frequency Methods and Phase Space, 286–297.Google Scholar
    15. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 654–663. Google ScholarDigital Library
    16. Li, H., Manjunath, B., and Mitra, S. 1994. Multi-sensor image fusion using the wavelet transform. Proc. ICIP 1, 51–55.Google Scholar
    17. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3, 836–844. Google ScholarDigital Library
    18. Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press. Google ScholarDigital Library
    19. Malzbender, T., Gelb, D., and Wolters, H. 2001. Polynomial texture maps. In Proc. SIGGRAPH, 519–528. Google ScholarDigital Library
    20. Masselus, V., Dutré, P., and Anrys, F. 2002. The free-form light stage. In Proc. Eurographics Symposium on Rendering, 247–256. Google ScholarDigital Library
    21. Mohan, A., Tumblin, J., Bodenheimer, B., Grimm, C., and Bailey, R. 2005. Table-top computed lighting for practical digital photography. In Eurographics Symposium on Rendering, 165–172. Google ScholarDigital Library
    22. Mudge, M., Voutaz, J.-P., Schroer, C., and Lum, M. 2005. Reflection transformation imaging and virtual representations of coins from the Hospice of the Grand St. Bernard. In Proc. VAST. Google ScholarCross Ref
    23. Mudge, M., Malzbender, T., Schroer, C., and Lum, M. 2006. New reflection transformation imaging methods for rock art and multiple-viewpoint display. In Proc. VAST. Google ScholarCross Ref
    24. Ogden, J., Adelson, E., Bergen, J., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4–15.Google Scholar
    25. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. SIGGRAPH, 433–442. Google ScholarDigital Library
    26. Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. Tech. Rep. MIT-CSAIL-TR-2006-073.Google Scholar
    27. Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. SIGGRAPH, 287–298. Google ScholarDigital Library
    28. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 664–672. Google ScholarDigital Library
    29. Pham, T., and van Vliet, L. 2005. Separable Bilateral Filtering for Fast Video Preprocessing. Proc. ICME, 454–457.Google ScholarCross Ref
    30. Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. Proc. NPAR, 85–152. Google ScholarDigital Library
    31. Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 679–688. Google ScholarDigital Library
    32. Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. In Proc. SIGGRAPH, 267–276. Google ScholarDigital Library
    33. Rusinkiewicz, S., Burns, M., and DeCarlo, D. 2006. Exaggerated shading for depicting shape and detail. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1199–1205. Google ScholarDigital Library
    34. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839–846. Google ScholarDigital Library
    35. Tumblin, J., and Turk, G. 1999. LCIS: A boundary hierarchy for detail-preserving contrast reduction. In Proc. SIGGRAPH, 83–90. Google ScholarDigital Library
    36. Weiss, Y. 2001. Deriving intrinsic images from image sequences. In ICCV, 68–75.Google Scholar
    37. Weiss, B. 2006. Fast median and bilateral filtering. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 519–526. Google ScholarDigital Library
    38. Winnemöller, H., Mohan, A., Tumblin, J., and Gooch, B. 2005. Light waving: Estimating light positions from photographs alone. Computer Graphics Forum 24, 3, 433–438.Google ScholarCross Ref
    39. Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1221–1226. Google ScholarDigital Library
    40. Woodham, R. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139–144.Google ScholarCross Ref

ACM Digital Library Publication:

Overview Page: