“Invertible motion blur in video” by Agrawal, Xu and Raskar

  • ©Amit Agrawal, Yi Xu, and Ramesh Raskar

Conference:


Type:


Title:

    Invertible motion blur in video

Presenter(s)/Author(s):



Abstract:


    We show that motion blur in successive video frames is invertible even if the point-spread function (PSF) due to motion smear in a single photo is non-invertible. Blurred photos exhibit nulls (zeros) in the frequency transform of the PSF, leading to an ill-posed deconvolution. Hardware solutions to avoid this require specialized devices such as the coded exposure camera or accelerating sensor motion. We employ ordinary video cameras and introduce the notion of null-filling along with joint-invertibility of multiple blur-functions. The key idea is to record the same object with varying PSFs, so that the nulls in the frequency component of one frame can be filled by other frames. The combined frequency transform becomes null-free, making deblurring well-posed. We achieve jointly-invertible blur simply by changing the exposure time of successive frames. We address the problem of automatic deblurring of objects moving with constant velocity by solving the four critical components: preservation of all spatial frequencies, segmentation of moving parts, motion estimation of moving parts, and non-degradation of the static parts of the scene. We demonstrate several challenging cases of object motion blur including textured backgrounds and partial occluders.

References:


    1. Agrawal, A., and Raskar, R. 2007. Resolving Objects at Higher Resolution from a Single Motion-blurred Image. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
    2. Bascle, B., Blake, A., and Zisserman, A. 1996. Motion Deblurring and Super-resolution from an Image Sequence. In Proc. European Conf. Computer Vision, vol. 2, 573–582. Google ScholarDigital Library
    3. Ben-Ezra, M., and Nayar, S. 2004. Motion-based Motion Deblurring. IEEE Trans. Pattern Anal. Machine Intell. 26, 6 (Jun), 689–698. Google ScholarDigital Library
    4. Berenstein, C., and Patrick, E. 1990. Exact Deconvolution for Multiple Convolution Operators-an Overview, plus Performance Characterizations for Imaging Sensors. Proc. of the IEEE 78 (Apr.), 723–734.Google ScholarCross Ref
    5. Chen, W.-G., Nandhakumar, N., and Martin, W. N. 1996. Image Motion Estimation from Motion Smear-A New Computational Model. IEEE Trans. Pattern Anal. Mach. Intell. 18, 4 (Apr.), 412–425. Google ScholarDigital Library
    6. Chen, J., Yuan, L., Tang, C.-K., and Quan, L. 2008. Robust Dual Motion Deblurring. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
    7. Cho, S., Matsushita, Y., and Lee, S. 2007. Removing Non-Uniform Motion Blur from Images. In Proc. Int’l Conf. Computer Vision, 1–8.Google Scholar
    8. Dai, S., and Wu, Y. 2008. Motion from Blur. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
    9. Debevec, P. E., and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proc. SIGGRAPH 97, 369–378. Google ScholarDigital Library
    10. Dowski, E. R., and Cathey, W. 1995. Extended Depth of Field through Wavefront Coding. Appl. Optics 34, 11 (Apr.), 1859–1866.Google ScholarCross Ref
    11. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing Camera Shake from a Single Photograph. ACM Trans. Graph. 25, 3 (jul), 787–794. Google ScholarDigital Library
    12. Grossberg, M., and Nayar, S. 2003. High Dynamic Range from Multiple Images: Which Exposures to Combine? In ICCV Workshop on Color and Photometric Methods in Computer Vision (CPMCV).Google Scholar
    13. Jansson, P. 1997. Deconvolution of Image and Spectra, 2nd ed. Academic Press. Google ScholarDigital Library
    14. Jia, J. 2007. Single Image Motion Deblurring using Transparency. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google ScholarCross Ref
    15. Joshi, N., Szeliski, R., and Kriegman, D. 2008. PSF Estimation using Sharp Edge Prediction. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
    16. Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and Depth from a Conventional Camera with a Coded Aperture. ACM Trans. Graph. 26, 3 (Jul.), 70. Google ScholarDigital Library
    17. Levin, A., Lischinski, D., and Weiss, Y. 2008. A Closed-Form Solution to Natural Image Matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2, 228–242. Google ScholarDigital Library
    18. Levin, A., Sand, P., Cho, T. S., Durand, F., and Freeman, W. T. 2008. Motion-Invariant Photography. ACM Trans. Graph. 27, 3 (Aug.), 71. Google ScholarDigital Library
    19. Lucy, L. 1974. An iterative technique for the rectification of observed distributions. J. Astronomy 79, 745–754.Google ScholarCross Ref
    20. Mann, S., and Picard, R. W. 1995. Being Undigital with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures. In Proc. of IS&T 48th Annual Conference, 422–428.Google Scholar
    21. Nagahara, H., Kuthirummal, S., Zhou, C., and Nayar, S. 2008. Flexible Depth of Field Photography. In Proc. European Conf. Computer Vision, 60–73. Google ScholarDigital Library
    22. Piccardi, M. 2004. Background Subtraction Techniques: a Review. In Proc. IEEE SMC Intl. Conf. Systems, Man and Cybernetics.Google ScholarCross Ref
    23. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded Rxposure Photography: Motion Deblurring using Fluttered Shutter. ACM Trans. Graph. 25, 3 (Jul.), 795–804. Google ScholarDigital Library
    24. Rav-Acha, A., and Peleg, S. 2005. Two Motion-blurred Images are Better than One. Pattern Recognition Letters 26, 3, 311–317. Google ScholarDigital Library
    25. Richardson, W. 1972. Bayesian-based iterative method of image restoration. J. Opt. Soc. of America 62, 1, 55–59.Google ScholarCross Ref
    26. Schultz, R. R., and Stevenson, R. L. 1996. Extraction of High-Resolution Frames from Video Sequences. IEEE Trans. Image Processing 5 (jun), 996–1011. Google ScholarDigital Library
    27. Sellent, A., Eisemann, M., and Magnor, M. 2008. Calculating Motion Fields from Images with Two Different Exposure Times. Tech. rep., Computer Graphics Lab, Technical University of Braunschweig, 5.Google Scholar
    28. Shan, Q., Jia, J., and Agarwala, A. 2008. High-Quality Motion Deblurring from a Single Image. ACM Trans. Graph. 27, 3 (Aug.), 73. Google ScholarDigital Library
    29. Shechtman, E., Caspi, Y., and Irani, M. 2002. Increasing Space-Time Resolution in Video. In Proc. European Conf. Computer Vision, 753–768. Google ScholarDigital Library
    30. Tai., W. Y., Hao, D., Brown, M. S., and Lin, S. 2008. Image/Video Deblurring using a Hybrid Camera. In Proc. Conf. Comp. Vision and Pattern Recognition, 1–8.Google Scholar
    31. Telleen, J., Sullivan, A., Yee, J., Gunawardane, P., Wang, O., Collins, I., and Davis, J. 2007. Synthetic Shutter Speed Imaging. In Proc. Eurographics, 591–598.Google Scholar
    32. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. 24, 3 (Jul.), 765–776. Google ScholarDigital Library
    33. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2007. Image Deblurring with Blurred/Noisy Image Pairs. ACM Trans. Graph. 26, 3 (Jul.), 1. Google ScholarDigital Library
    34. Yuan, L., Sun, J., Quan, L., and Shum, H.-Y. 2008. Progressive Inter-Scale and Intra-Scale Non-Blind Image Deconvolution. ACM Trans. Graph. 27, 3 (Aug.), 74. Google ScholarDigital Library


ACM Digital Library Publication: