“Invertible motion blur in video” by Agrawal, Xu and Raskar

We show that motion blur in successive video frames is invertible even if the point-spread function (PSF) due to motion smear in a single photo is non-invertible. Blurred photos exhibit nulls (zeros) in the frequency transform of the PSF, leading to an ill-posed deconvolution. Hardware solutions to avoid this require specialized devices such as the coded exposure camera or accelerating sensor motion. We employ ordinary video cameras and introduce the notion of null-filling along with joint-invertibility of multiple blur-functions. The key idea is to record the same object with varying PSFs, so that the nulls in the frequency component of one frame can be filled by other frames. The combined frequency transform becomes null-free, making deblurring well-posed. We achieve jointly-invertible blur simply by changing the exposure time of successive frames. We address the problem of automatic deblurring of objects moving with constant velocity by solving the four critical components: preservation of all spatial frequencies, segmentation of moving parts, motion estimation of moving parts, and non-degradation of the static parts of the scene. We demonstrate several challenging cases of object motion blur including textured backgrounds and partial occluders.

1. Agrawal, A., and Raskar, R. 2007. Resolving Objects at Higher Resolution from a Single Motion-blurred Image. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
2. Bascle, B., Blake, A., and Zisserman, A. 1996. Motion Deblurring and Super-resolution from an Image Sequence. In Proc. European Conf. Computer Vision, vol. 2, 573–582. Google ScholarDigital Library
3. Ben-Ezra, M., and Nayar, S. 2004. Motion-based Motion Deblurring. IEEE Trans. Pattern Anal. Machine Intell. 26, 6 (Jun), 689–698. Google ScholarDigital Library
4. Berenstein, C., and Patrick, E. 1990. Exact Deconvolution for Multiple Convolution Operators-an Overview, plus Performance Characterizations for Imaging Sensors. Proc. of the IEEE 78 (Apr.), 723–734.Google ScholarCross Ref
5. Chen, W.-G., Nandhakumar, N., and Martin, W. N. 1996. Image Motion Estimation from Motion Smear-A New Computational Model. IEEE Trans. Pattern Anal. Mach. Intell. 18, 4 (Apr.), 412–425. Google ScholarDigital Library
6. Chen, J., Yuan, L., Tang, C.-K., and Quan, L. 2008. Robust Dual Motion Deblurring. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
7. Cho, S., Matsushita, Y., and Lee, S. 2007. Removing Non-Uniform Motion Blur from Images. In Proc. Int’l Conf. Computer Vision, 1–8.Google Scholar
8. Dai, S., and Wu, Y. 2008. Motion from Blur. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
9. Debevec, P. E., and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proc. SIGGRAPH 97, 369–378. Google ScholarDigital Library
10. Dowski, E. R., and Cathey, W. 1995. Extended Depth of Field through Wavefront Coding. Appl. Optics 34, 11 (Apr.), 1859–1866.Google ScholarCross Ref
11. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing Camera Shake from a Single Photograph. ACM Trans. Graph. 25, 3 (jul), 787–794. Google ScholarDigital Library
12. Grossberg, M., and Nayar, S. 2003. High Dynamic Range from Multiple Images: Which Exposures to Combine? In ICCV Workshop on Color and Photometric Methods in Computer Vision (CPMCV).Google Scholar
13. Jansson, P. 1997. Deconvolution of Image and Spectra, 2nd ed. Academic Press. Google ScholarDigital Library
14. Jia, J. 2007. Single Image Motion Deblurring using Transparency. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google ScholarCross Ref
15. Joshi, N., Szeliski, R., and Kriegman, D. 2008. PSF Estimation using Sharp Edge Prediction. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
16. Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and Depth from a Conventional Camera with a Coded Aperture. ACM Trans. Graph. 26, 3 (Jul.), 70. Google ScholarDigital Library
17. Levin, A., Lischinski, D., and Weiss, Y. 2008. A Closed-Form Solution to Natural Image Matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2, 228–242. Google ScholarDigital Library
18. Levin, A., Sand, P., Cho, T. S., Durand, F., and Freeman, W. T. 2008. Motion-Invariant Photography. ACM Trans. Graph. 27, 3 (Aug.), 71. Google ScholarDigital Library
19. Lucy, L. 1974. An iterative technique for the rectification of observed distributions. J. Astronomy 79, 745–754.Google ScholarCross Ref
20. Mann, S., and Picard, R. W. 1995. Being Undigital with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures. In Proc. of IS&T 48th Annual Conference, 422–428.Google Scholar
21. Nagahara, H., Kuthirummal, S., Zhou, C., and Nayar, S. 2008. Flexible Depth of Field Photography. In Proc. European Conf. Computer Vision, 60–73. Google ScholarDigital Library
22. Piccardi, M. 2004. Background Subtraction Techniques: a Review. In Proc. IEEE SMC Intl. Conf. Systems, Man and Cybernetics.Google ScholarCross Ref
23. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded Rxposure Photography: Motion Deblurring using Fluttered Shutter. ACM Trans. Graph. 25, 3 (Jul.), 795–804. Google ScholarDigital Library
24. Rav-Acha, A., and Peleg, S. 2005. Two Motion-blurred Images are Better than One. Pattern Recognition Letters 26, 3, 311–317. Google ScholarDigital Library
25. Richardson, W. 1972. Bayesian-based iterative method of image restoration. J. Opt. Soc. of America 62, 1, 55–59.Google ScholarCross Ref
26. Schultz, R. R., and Stevenson, R. L. 1996. Extraction of High-Resolution Frames from Video Sequences. IEEE Trans. Image Processing 5 (jun), 996–1011. Google ScholarDigital Library
27. Sellent, A., Eisemann, M., and Magnor, M. 2008. Calculating Motion Fields from Images with Two Different Exposure Times. Tech. rep., Computer Graphics Lab, Technical University of Braunschweig, 5.Google Scholar
28. Shan, Q., Jia, J., and Agarwala, A. 2008. High-Quality Motion Deblurring from a Single Image. ACM Trans. Graph. 27, 3 (Aug.), 73. Google ScholarDigital Library
29. Shechtman, E., Caspi, Y., and Irani, M. 2002. Increasing Space-Time Resolution in Video. In Proc. European Conf. Computer Vision, 753–768. Google ScholarDigital Library
30. Tai., W. Y., Hao, D., Brown, M. S., and Lin, S. 2008. Image/Video Deblurring using a Hybrid Camera. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
31. Telleen, J., Sullivan, A., Yee, J., Gunawardane, P., Wang, O., Collins, I., and Davis, J. 2007. Synthetic Shutter Speed Imaging. In Proc. Eurographics, 591–598.Google Scholar
32. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. 24, 3 (Jul.), 765–776. Google ScholarDigital Library
33. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2007. Image Deblurring with Blurred/Noisy Image Pairs. ACM Trans. Graph. 26, 3 (Jul.), 1. Google ScholarDigital Library
34. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2008. Progressive Inter-Scale and Intra-Scale Non-Blind Image Deconvolution. ACM Trans. Graph. 27, 3 (Aug.), 74. Google ScholarDigital Library