“Bundled camera paths for video stabilization” by Liu, Yuan, Tan and Sun
Conference:
Type(s):
Title:
- Bundled camera paths for video stabilization
Session/Category Title: Video & Warping
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a novel video stabilization method which models camera motion with a bundle of (multiple) camera paths. The proposed model is based on a mesh-based, spatially-variant motion representation and an adaptive, space-time path optimization. Our motion representation allows us to fundamentally handle parallax and rolling shutter effects while it does not require long feature trajectories or sparse 3D reconstruction. We introduce the ‘as-similar-as-possible’ idea to make motion estimation more robust. Our space-time path smoothing adaptively adjusts smoothness strength by considering discontinuities, cropping size and geometrical distortion in a unified optimization framework. The evaluation on a large variety of consumer videos demonstrates the merits of our method.
References:
1. Baker, S., Bennett, E. P., Kang, S. B., and Szeliski, R. 2010. Removing rolling shutter wobble. In Proc. CVPR.Google Scholar
2. Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. 2008. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110, 3, 346–359. Google ScholarDigital Library
3. Bronshtein, I. N., and Semendyayev, K. A. 1997. Handbook of Mathematics. Springer-Verlag, New York, NY, USA. Google ScholarDigital Library
4. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV.Google Scholar
5. Buehler, C., Bosse, M., and McMillan, L. 2001. Non-metric image-based rendering for video stabilization. In Proc. CVPR.Google ScholarCross Ref
6. Chen, B.-Y., Lee, K.-Y., Huang, W.-T., and Lin, J.-S. 2008. Capturing intention-based full-frame video stabilization. Computer Graphics Forum 27, 7, 1805–1814.Google ScholarCross Ref
7. Cho, S., Wang, J., and Lee, S. 2012. Video deblurring for hand-held cameras using patch-based synthesis. ACM Trans. Graph. (Proc. of SIGGRAPH) 31, 4. Google ScholarDigital Library
8. Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
9. Forssén, P.-E., and Ringaby, E. 2010. Rectifying rolling shutter video from hand-held devices. In CVPR.Google Scholar
10. Gao, J., Kim, S. J., and Brown, M. S. 2011. Constructing image panoramas using dual-homography warping. In Proc. CVPR. Google ScholarDigital Library
11. Gleicher, M. L., and Liu, F. 2007. Re-cinematography: Improving the camera dynamics of casual video. In Proc. of ACM Multimedia. Google ScholarDigital Library
12. Goldstein, A., and Fattal, R. 2012. Video stabilization using epipolar geometry. ACM Trans. Graph. (TOG) 31, 5, 126:1–126:10. Google ScholarDigital Library
13. Grundmann, M., Kwatra, V., and Essa, I. 2011. Auto-directed video stabilization with robust l1 optimal camera paths. In Proc. CVPR. Google ScholarDigital Library
14. Grundmann, M., Kwatra, V., Castro, D., and Essa, I. 2012. Calibration-free rolling shutter removal. In Proc. ICCP.Google Scholar
15. Hartley, R., and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2 ed. Cambridge University Press, New York, NY, USA. Google ScholarDigital Library
16. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Trans. Graph. (Proc. of SIGGRAPH) 24, 3, 1134–1141. Google ScholarDigital Library
17. Karpenko, A., Jacobs, D., Baek, J., and Levoy, M. 2011. Digital video stabilization and rolling shutter correction using gyroscopes. In Stanford CS Tech Report.Google Scholar
18. Lee, K.-Y., Chuang, Y.-Y., Chen, B.-Y., and Ouhyoung, M. 2009. Video stabilization using robust feature trajectories. In Proc. ICCV.Google Scholar
19. Liang, C.-K., Chang, L.-W., and Chen, H. H. 2008. Analysis and compensation of rolling shutter effect. In IEEE Trans. on Image Processing. Google ScholarDigital Library
20. Lin, W.-Y., Liu, S., Matsushita, Y., Ng, T.-T., and Cheong, L.-F. 2011. Smoothly varying affine stitching. In Proc. CVPR. Google ScholarDigital Library
21. Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Trans. Graph. (Proc. of SIGGRAPH) 28. Google ScholarDigital Library
22. Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Trans. Graph. 30. Google ScholarDigital Library
23. Liu, S., Wang, Y., Yuan, L., Bu, J., Tan, P., and Sun, J. 2012. Video stabilization with a depth camera. In Proc. CVPR. Google ScholarDigital Library
24. Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. of the International Joint Conference on Artificial Intelligence (IJCAI), 674–679. Google ScholarDigital Library
25. Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1163. Google ScholarDigital Library
26. Morimoto, C., and Chellappa, R. 1998. Evaluation of image stabilization algorithms. In Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, 2789–2792.Google Scholar
27. Nakamura, J. 2005. Image Sensors and Signal Processing for Digital Still Cameras. CRC Press, Inc. Google ScholarDigital Library
28. Nir, T., Bruckstein, A. M., and Kimmel, R. 2008. Over-parameterized variational optical flow. Int. J. Comput. Vision (IJCV) 76, 2, 205–216. Google ScholarDigital Library
29. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. Graph. (Proc. of SIGGRAPH) 25, 3, 533–540. Google ScholarDigital Library
30. Shum, H.-Y., and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. Int. J. Comput. Vision (IJCV) 36, 2, 101–130. Google ScholarDigital Library
31. Smith, B. M., Zhang, L., Jin, H., and Agarwala, A. 2009. Light field video stabilization. In Proc. ICCV.Google Scholar
32. Szeliski, R. 1996. Motion estimation with quadtree splines. IEEE Trans. Pattern Anal. Mach. Intell. 18, 12, 1199–1210. Google ScholarDigital Library
33. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839–846. Google ScholarDigital Library
