“Multiscale shape and detail enhancement from multi-light image collections” by Fattal, Agrawala and Rusinkiewicz

We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varying lighting conditions. For each image we compute a multiscale decomposition based on the bilateral filter and then reconstruct an enhanced image that combines detail information at each scale across all the input images. Our approach does not require any information about light source positions, or camera calibration, and can produce good results with 3 to 5 input images. In addition our system provides a few high-level parameters for controlling the amount of enhancement and does not require pixel-level user input. We show that the bilateral filter is a good choice for our multiscale algorithm because it avoids the halo artifacts commonly associated with the traditional Laplacian image pyramid. We also develop a new scheme for computing our multiscale bilateral decomposition that is simple to implement, fast O(N2 log N) and accurate.

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., AND Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 294–302. Google ScholarDigital Library
2. Akers, D., Losasso, F., Klingner, J., Agrawala, M., Rick, J., and Hanrahan, P. 2003. Conveying shape and features with image-based relighting. In Proc. IEEE Visualization, 349–254. Google ScholarDigital Library
3. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 637–645. Google ScholarDigital Library
4. Barash, D., and Comaniciu, D. 2004. A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift. Journal of Image and Vision Computing 22, 1, 73–81.Google ScholarCross Ref
5. Burt, P. J., and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Comm. 31, 4, 532–540.Google ScholarCross Ref
6. Burt, P., and Kolczynski, R. 1993. Enhanced image capture through fusion. Proc. ICCV, 173–182.Google Scholar
7. Burt, P. J. 1981. Fast filter transforms for image processing. Computer Graphics, Image Processing 6, 20–51.Google ScholarCross Ref
8. Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. Eurographics Symposium on Rendering, 186–196. Google ScholarDigital Library
9. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proc. SIGGRAPH, 145–156. Google ScholarDigital Library
10. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In Proc. SIGGRAPH, 257–266. Google ScholarDigital Library
11. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 673–678. Google ScholarDigital Library
12. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proc. SIGGRAPH, 249–256. Google ScholarDigital Library
13. Finalyson, G. D., Hordley, S. D., Lu, C., and Drew, M. S. 2006. On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1, 59–68. Google ScholarDigital Library
14. Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, P. 1989. A real-time algorithms for signal analysis with the help of the wavelet transform. Wavelets, Time-Frequency Methods and Phase Space, 286–297.Google Scholar
15. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 654–663. Google ScholarDigital Library
16. Li, H., Manjunath, B., and Mitra, S. 1994. Multi-sensor image fusion using the wavelet transform. Proc. ICIP 1, 51–55.Google Scholar
17. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3, 836–844. Google ScholarDigital Library
18. Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press. Google ScholarDigital Library
19. Malzbender, T., Gelb, D., and Wolters, H. 2001. Polynomial texture maps. In Proc. SIGGRAPH, 519–528. Google ScholarDigital Library
20. Masselus, V., Dutré, P., and Anrys, F. 2002. The free-form light stage. In Proc. Eurographics Symposium on Rendering, 247–256. Google ScholarDigital Library
21. Mohan, A., Tumblin, J., Bodenheimer, B., Grimm, C., and Bailey, R. 2005. Table-top computed lighting for practical digital photography. In Eurographics Symposium on Rendering, 165–172. Google ScholarDigital Library
22. Mudge, M., Voutaz, J.-P., Schroer, C., and Lum, M. 2005. Reflection transformation imaging and virtual representations of coins from the Hospice of the Grand St. Bernard. In Proc. VAST. Google ScholarCross Ref
23. Mudge, M., Malzbender, T., Schroer, C., and Lum, M. 2006. New reflection transformation imaging methods for rock art and multiple-viewpoint display. In Proc. VAST. Google ScholarCross Ref
24. Ogden, J., Adelson, E., Bergen, J., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4–15.Google Scholar
25. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. SIGGRAPH, 433–442. Google ScholarDigital Library
26. Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. Tech. Rep. MIT-CSAIL-TR-2006-073.Google Scholar
27. Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. SIGGRAPH, 287–298. Google ScholarDigital Library
28. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 664–672. Google ScholarDigital Library
29. Pham, T., and van Vliet, L. 2005. Separable Bilateral Filtering for Fast Video Preprocessing. Proc. ICME, 454–457.Google ScholarCross Ref
30. Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. Proc. NPAR, 85–152. Google ScholarDigital Library
31. Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3, 679–688. Google ScholarDigital Library
32. Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. In Proc. SIGGRAPH, 267–276. Google ScholarDigital Library
33. Rusinkiewicz, S., Burns, M., and DeCarlo, D. 2006. Exaggerated shading for depicting shape and detail. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1199–1205. Google ScholarDigital Library
34. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839–846. Google ScholarDigital Library
35. Tumblin, J., and Turk, G. 1999. LCIS: A boundary hierarchy for detail-preserving contrast reduction. In Proc. SIGGRAPH, 83–90. Google ScholarDigital Library
36. Weiss, Y. 2001. Deriving intrinsic images from image sequences. In ICCV, 68–75.Google Scholar
37. Weiss, B. 2006. Fast median and bilateral filtering. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 519–526. Google ScholarDigital Library
38. Winnemöller, H., Mohan, A., Tumblin, J., and Gooch, B. 2005. Light waving: Estimating light positions from photographs alone. Computer Graphics Forum 24, 3, 433–438.Google ScholarCross Ref
39. Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3, 1221–1226. Google ScholarDigital Library
40. Woodham, R. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139–144.Google ScholarCross Ref