“Image deblurring with blurred/noisy image pairs” by Yuan, Sun, Quan and Shum

Taking satisfactory photos under dim lighting conditions using a hand-held camera is challenging. If the camera is set to a long exposure time, the image is blurred due to camera shake. On the other hand, the image is dark and noisy if it is taken with a short exposure time but with a high camera gain. By combining information extracted from both blurred and noisy images, however, we show in this paper how to produce a high quality image that cannot be obtained by simply denoising the noisy image, or deblurring the blurred image alone.Our approach is image deblurring with the help of the noisy image. First, both images are used to estimate an accurate blur kernel, which otherwise is difficult to obtain from a single blurred image. Second, and again using both images, a residual deconvolution is proposed to significantly reduce ringing artifacts inherent to image deconvolution. Third, the remaining ringing artifacts in smooth image regions are further suppressed by a gain-controlled deconvolution process. We demonstrate the effectiveness of our approach using a number of indoor and outdoor images taken by off-the-shelf hand-held cameras in poor lighting environments.

1. Bardsley, J., Jefferies, S., Nagy, J., and Plemmons, R. 2006. Blind iterative restoration of images with spatially-varying blur. In Optics Express, 1767–1782.Google Scholar
2. Bascle, B., Blake, A., and Zisserman, A. 1996. Motion deblurring and super-resolution from an image sequence. In Processings of ECCV, vol. II, 573–582. Google ScholarDigital Library
3. Ben-Ezra, M., and Nayar, S. K. 2003. Motion deblurring using hybrid imaging. In Processings of CVPR, vol. I, 657–664. Google ScholarDigital Library
4. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Trans. Graph. 24, 3, 845–852. Google ScholarDigital Library
5. Buades, A., Coll, B., and Morel, J. M. 2005. A non-local algorithm for image denoising. In Proceedings of CVPR, vol. II, 60–65. Google ScholarDigital Library
6. Canny, J. 1986. A computational approach to edge detection. IEEE Trans. on PAMI. 8, 6, 679–698. Google ScholarDigital Library
7. Caron, J. N., M., N. N., and J., R. C. 2002. Noniterative blind data restoration by use of an extracted filter function. Applied optics (Appl. opt.) 41, 32, 68–84.Google Scholar
8. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH, 369–378. Google ScholarDigital Library
9. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. In Proceedings of SIGGRAPH, 257–266. Google ScholarDigital Library
10. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 3, 673–678. Google ScholarDigital Library
11. Engl, H. W., Hanke, M., and Neubauer, A. 2000. Regularization of Inverse Problems. Kluwer Academic.Google Scholar
12. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In Proceedings of SIGGRAPH, 249–256. Google ScholarDigital Library
13. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing camera shake from a single photograph. In ACM Trans. Graph., vol. 25, 787–794. Google ScholarDigital Library
14. Geman, D., and Reynolds, G. 1992. Constrained restoration and the recovery of discontinuities. IEEE Trans. on PAMI. 14, 3, 367–383. Google ScholarDigital Library
15. H. Richardson, W. 1972. Bayesian-based iterative method of image restoration. JOSA, A 62, 1, 55–59.Google Scholar
16. Jalobeanu, A., Blanc-Feraud, L., and Zerubia, J. 2002. Estimation of blur and noise parameters in remote sensing. In Proceedings of ICASSP, 249–256.Google Scholar
17. Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Bayesian correction of image intensity with spatial consideration. In Proceedings of ECCV, 342–354.Google Scholar
18. Kundur, D., and Hatzinakos, D. 1996. Blind image deconvolution. IEEE Signal Processing Magazine. 13, 3, 43–64.Google ScholarCross Ref
19. Levin, A. 2006. Blind motion deblurring using image statistics. In Advances in Neural Information Processing Systems (NIPS).Google Scholar
20. Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. 24, 3, 836–844. Google ScholarDigital Library
21. Lim, S. H., and Silverstein, D. A. 2006. Method for deblurring an image. US Patent Application, Pub. No. US2006/0187308 A1, Aug 24, 2006.Google Scholar
22. Liu, X., and Gamal, A. 2001. Simultaneous image formation and motion blur restoration via multiple capture. Proceedings of ICASSP.. Google ScholarDigital Library
23. Liu, C., Freeman, W., Szeliski, R., and Kang, S. 2006. Noise estimation from a single image. In Proceedings of CVPR, vol. I, 901–908. Google ScholarDigital Library
24. Neelamani, R., Choi, H., and Baraniuk, R. 2004. ForWaRd: Fourier-wavelet regularized deconvolution for ill-conditioned systems. IEEE Trans. on Signal Processing 52, 2, 418–433. Google ScholarDigital Library
25. Nikon. 2005. http://www.nikon.co.jp/main/eng/portfolio/about/technology/nikon_technology/vr_e/index.htm.Google Scholar
26. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on PAMI 12, 7, 629–639. Google ScholarDigital Library
27. Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., and Toyama., K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 3, 664–672. Google ScholarDigital Library
28. Portilla, J., Strela, V., Wainwright, M., and Simoncelli., E. P. 2003. Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans. on Image Processing 12, 11, 1338–1351. Google ScholarDigital Library
29. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3, 795–804. Google ScholarDigital Library
30. Rav-Acha, A., and Peleg, S. 2000. Restoration of multiple images with motion blur in different directions. IEEE Workshop on Applications of Computer Vision.Google ScholarCross Ref
31. Rav-Acha, A., and Peleg, S. 2005. Two motion-blurred images are better than one. Pattern Recogn. Lett. 26, 3, 311–317. Google ScholarDigital Library
32. Reeves, S. J., and Mersereau, R. M. 1992. Blur identification by the method of generalized cross-validation. IEEE Trans. on Image Processing. 1, 3, 301–311.Google ScholarDigital Library
33. Roth, S., and Black, M. J. 2005. Fields of experts: A framework for learning image priors. In Proceedings of CVPR, vol. II, 860–867. Google ScholarDigital Library
34. Rudin, L., Osher, S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Phys. D. 60, 259–268. Google ScholarDigital Library
35. Simoncelli, E. P., and Adelson, E. H. 1996. Noise removal via bayesian wavelet coring. In Proceedings of ICIP, vol. I, 379–382.Google Scholar
36. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proceedings of ICCV, 839–846. Google ScholarDigital Library
37. Y. Yitzhaky, I. Mor, A. L., and Kopeika., N. 1998. Direct method for restoration of motion blurred images. J. Opt. Soc. Am., A 15, 6, 1512–1519.Google ScholarCross Ref
38. Zarowin, C. B. 1994. Robust, noniterative, and computationally efficient modification of vab cittert deconvolution optical figuring. JOSA, A 11, 10, 2571–2583.Google Scholar