“Removing photography artifacts using gradient projection and flash-exposure sampling” by Agrawal, Raskar, Nayar and Li

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. We propose to use a flash and no-flash (ambient) image pair to produce better flash images. We present a novel gradient projection scheme based on a gradient coherence model that allows removal of reflections and highlights from flash images. We also present a brightness-ratio based algorithm that allows us to compensate for the falloff in the flash image brightness due to depth. In several practical scenarios, the quality of flash/no-flash images may be limited in terms of dynamic range. In such cases, we advocate using several images taken under different flash intensities and exposures. We analyze the flash intensity-exposure space and propose a method for adaptively sampling this space so as to minimize the number of captured images for any given scene. We present several experimental results that demonstrate the ability of our algorithms to produce improved flash images.

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3 (Aug.), 294–302. Google ScholarDigital Library
2. Canon. http://www.cps.canon-europe.com/kb/.Google Scholar
3. Chen, H., Belhumeur, P., and Jacobs, D. 2000. In search of illumination invariants. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 254–261.Google Scholar
4. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proc. of the 24th annual conference on Computer graphics and interactive techniques, 369–378. Google ScholarDigital Library
5. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (Aug.). 673–678. Google ScholarDigital Library
6. Farid, H., and Adelson, E. H. 1999. Separating reflections and lighting using independent components analysis. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 1262–1267.Google Scholar
7. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Transactions on Graphics 21, 3, 249–256. Google ScholarDigital Library
8. Finlayson, G., Hordley, S., and Drew, M. 2002. Removing shadows from images. In Proc. of European Conf. on Computer Vision, vol. 4, 823–836. Google ScholarDigital Library
9. Finlayson, G. D., Drew, M. S., and Lu, C. 2004. Intrinsic images by entropy minimization. In Proc. of European Conf. on Computer Vision, vol. 3, 582–595.Google Scholar
10. Grossberg, M. D., and Nayar, S. K. 2003. High dynamic range from multiple images: which exposures to combine? In Proc. of ICCV Workshop on Color and Photometric Methods in Computer Vision.Google Scholar
11. Jahne, B. 1993. Spatio-temporal image processing, theory and scientific applications, vol. 751 of Lecture Notes in Computer Vision. Springer-Verlag. Google ScholarDigital Library
12. Kodak. 2001. CCD image sensor noise sources. Application note MPT/PS-0233.Google Scholar
13. Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 306–313.Google Scholar
14. Lichtenauer, J., Reinders, M., and Hendriks, E. 2004. Influence of the observation likelihood function on particle filtering performance in tracking applications. In Sixth IEEE Int’l. Conf. on Automatic Face and Gesture Recognition, 767–772. Google ScholarDigital Library
15. Mann, S., and Picard, R. W. 1995. Being undigital with digital cameras: extending dynamic range by combining differently exposed pictures. In Proc. of IS&T 48th annual conference, 422–428.Google Scholar
16. Nayar, S. K., and Mitsunaga, T. 2000. High dynamic range imaging: spatially varying pixel exposures. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 472–479.Google Scholar
17. Nayar, S. K., Fang, X.-S., and Boult, T. 1997. Separation of reflection components using color and polarization. Int’l. Journal of Computer Vision 21, 3 (Feb.), 163–186. Google ScholarDigital Library
18. Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3, 313–318. Google ScholarDigital Library
19. Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (Aug.), 664–672. Google ScholarDigital Library
20. Press, W. H., Teukolsky, S., Vetterling, W. T., and Flannery, B. P. 1992. Numerical recipes in C: the art of scientific computing. Pearson Education. Google ScholarDigital Library
21. Raskar, R., Tan, K., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Transactions on Graphics 23, 3, 679–688. Google ScholarDigital Library
22. Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. In Proc. of NPAR, 85–95. Google ScholarDigital Library
23. Schechner, Y. Y., Kiryati, N., and Basri, R. 2000. Separation of transparent layers using focus. Int’l. Journal of Computer Vision 39, 1 (Aug.), 25–39. Google ScholarDigital Library
24. Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Transactions on Graphics 23, 3, 315–321. Google ScholarDigital Library
25. Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 246–243.Google Scholar