“Bundled camera paths for video stabilization” by Liu, Yuan, Tan and Sun

  • ©Shuaicheng Liu, Lu Yuan, Ping Tan, and Jian Sun

Conference:


Type:


Title:

    Bundled camera paths for video stabilization

Session/Category Title: Video & Warping

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We present a novel video stabilization method which models camera motion with a bundle of (multiple) camera paths. The proposed model is based on a mesh-based, spatially-variant motion representation and an adaptive, space-time path optimization. Our motion representation allows us to fundamentally handle parallax and rolling shutter effects while it does not require long feature trajectories or sparse 3D reconstruction. We introduce the ‘as-similar-as-possible’ idea to make motion estimation more robust. Our space-time path smoothing adaptively adjusts smoothness strength by considering discontinuities, cropping size and geometrical distortion in a unified optimization framework. The evaluation on a large variety of consumer videos demonstrates the merits of our method.

References:


    1. Baker, S., Bennett, E. P., Kang, S. B., and Szeliski, R. 2010. Removing rolling shutter wobble. In Proc. CVPR.Google Scholar
    2. Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. 2008. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110, 3, 346–359. Google ScholarDigital Library
    3. Bronshtein, I. N., and Semendyayev, K. A. 1997. Handbook of Mathematics. Springer-Verlag, New York, NY, USA. Google ScholarDigital Library
    4. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV.Google Scholar
    5. Buehler, C., Bosse, M., and McMillan, L. 2001. Non-metric image-based rendering for video stabilization. In Proc. CVPR.Google ScholarCross Ref
    6. Chen, B.-Y., Lee, K.-Y., Huang, W.-T., and Lin, J.-S. 2008. Capturing intention-based full-frame video stabilization. Computer Graphics Forum 27, 7, 1805–1814.Google ScholarCross Ref
    7. Cho, S., Wang, J., and Lee, S. 2012. Video deblurring for hand-held cameras using patch-based synthesis. ACM Trans. Graph. (Proc. of SIGGRAPH) 31, 4. Google ScholarDigital Library
    8. Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
    9. Forssén, P.-E., and Ringaby, E. 2010. Rectifying rolling shutter video from hand-held devices. In CVPR.Google Scholar
    10. Gao, J., Kim, S. J., and Brown, M. S. 2011. Constructing image panoramas using dual-homography warping. In Proc. CVPR. Google ScholarDigital Library
    11. Gleicher, M. L., and Liu, F. 2007. Re-cinematography: Improving the camera dynamics of casual video. In Proc. of ACM Multimedia. Google ScholarDigital Library
    12. Goldstein, A., and Fattal, R. 2012. Video stabilization using epipolar geometry. ACM Trans. Graph. (TOG) 31, 5, 126:1–126:10. Google ScholarDigital Library
    13. Grundmann, M., Kwatra, V., and Essa, I. 2011. Auto-directed video stabilization with robust l1 optimal camera paths. In Proc. CVPR. Google ScholarDigital Library
    14. Grundmann, M., Kwatra, V., Castro, D., and Essa, I. 2012. Calibration-free rolling shutter removal. In Proc. ICCP.Google Scholar
    15. Hartley, R., and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2 ed. Cambridge University Press, New York, NY, USA. Google ScholarDigital Library
    16. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Trans. Graph. (Proc. of SIGGRAPH) 24, 3, 1134–1141. Google ScholarDigital Library
    17. Karpenko, A., Jacobs, D., Baek, J., and Levoy, M. 2011. Digital video stabilization and rolling shutter correction using gyroscopes. In Stanford CS Tech Report.Google Scholar
    18. Lee, K.-Y., Chuang, Y.-Y., Chen, B.-Y., and Ouhyoung, M. 2009. Video stabilization using robust feature trajectories. In Proc. ICCV.Google Scholar
    19. Liang, C.-K., Chang, L.-W., and Chen, H. H. 2008. Analysis and compensation of rolling shutter effect. In IEEE Trans. on Image Processing. Google ScholarDigital Library
    20. Lin, W.-Y., Liu, S., Matsushita, Y., Ng, T.-T., and Cheong, L.-F. 2011. Smoothly varying affine stitching. In Proc. CVPR. Google ScholarDigital Library
    21. Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Trans. Graph. (Proc. of SIGGRAPH) 28. Google ScholarDigital Library
    22. Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Trans. Graph. 30. Google ScholarDigital Library
    23. Liu, S., Wang, Y., Yuan, L., Bu, J., Tan, P., and Sun, J. 2012. Video stabilization with a depth camera. In Proc. CVPR. Google ScholarDigital Library
    24. Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. of the International Joint Conference on Artificial Intelligence (IJCAI), 674–679. Google ScholarDigital Library
    25. Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1163. Google ScholarDigital Library
    26. Morimoto, C., and Chellappa, R. 1998. Evaluation of image stabilization algorithms. In Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, 2789–2792.Google Scholar
    27. Nakamura, J. 2005. Image Sensors and Signal Processing for Digital Still Cameras. CRC Press, Inc. Google ScholarDigital Library
    28. Nir, T., Bruckstein, A. M., and Kimmel, R. 2008. Over-parameterized variational optical flow. Int. J. Comput. Vision (IJCV) 76, 2, 205–216. Google ScholarDigital Library
    29. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. Graph. (Proc. of SIGGRAPH) 25, 3, 533–540. Google ScholarDigital Library
    30. Shum, H.-Y., and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. Int. J. Comput. Vision (IJCV) 36, 2, 101–130. Google ScholarDigital Library
    31. Smith, B. M., Zhang, L., Jin, H., and Agarwala, A. 2009. Light field video stabilization. In Proc. ICCV.Google Scholar
    32. Szeliski, R. 1996. Motion estimation with quadtree splines. IEEE Trans. Pattern Anal. Mach. Intell. 18, 12, 1199–1210. Google ScholarDigital Library
    33. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839–846. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: