“Phase-based video motion processing” by Wadhwa, Rubinstein, Durand and Freeman

  • ©Neal Wadhwa, Michael Rubinstein, Frédo Durand, and William T. Freeman

Conference:


Type:


Title:

    Phase-based video motion processing

Session/Category Title: Video & Warping

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.

References:


    1. Bai, J., Agarwala, A., Agrawala, M., and Ramamoorthi, R. 2012. Selectively de-animating video. ACM Transactions on Graphics. Google ScholarDigital Library
    2. Bojanic, S., Simpson, T., and Bolger, C. 2001. Ocular microtremor: a tool for measuring depth of anaesthesia? British Journal of Anaesthesia 86, 4, 519–522.Google ScholarCross Ref
    3. Buades, A., Coll, B., and Morel, J.-M. 2008. Nonlocal image and movie denoising. International Journal of Computer Vision 76, 123–139. Google ScholarDigital Library
    4. Dabov, K., Foi, A., and Egiazarian, K. 2007. Video denoising by sparse 3d transform-domain collaborative filtering. In Proc. 15th European Signal Processing Conference, vol. 1, 7.Google Scholar
    5. Fleet, D. J., and Jepson, A. D. 1990. Computation of component image velocity from local phase information. Int. J. Comput. Vision 5, 1 (Sept.), 77–104. Google ScholarDigital Library
    6. Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1991. Motion without movement. SIGGRAPH Comput. Graph. 25 (Jul), 27–30. Google ScholarDigital Library
    7. Fuchs, M., Chen, T., Wang, O., Raskar, R., Seidel, H.-P., and Lensch, H. P. 2010. Real-time temporal shaping of high-speed video streams. Computers & Graphics 34, 5, 575–584. Google ScholarDigital Library
    8. Gautama, T., and Van Hulle, M. 2002. A phase-based approach to the estimation of the optical flow field using spatial filtering. Neural Networks, IEEE Transactions on 13, 5 (sep), 1127–1136. Google ScholarDigital Library
    9. Liu, C., and Freeman, W. 2010. A high-quality video denoising algorithm based on reliable motion estimation. In Computer Vision ECCV 2010, K. Daniilidis, P. Maragos, and N. Paragios, Eds., vol. 6313 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 706–719. Google ScholarDigital Library
    10. Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24 (Jul), 519–526. Google ScholarDigital Library
    11. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40, 1 (Oct.), 49–70. Google ScholarDigital Library
    12. Rolfs, M. 2009. Microsaccades: Small steps on a long way. Vision Research 49, 20, 2415–2441.Google ScholarCross Ref
    13. Rubinstein, M., Liu, C., Sand, P., Durand, F., and Freeman, W. T. 2011. Motion denoising with application to time-lapse photography. IEEE Computer Vision and Pattern Recognition (CVPR) (June), 313–320. Google ScholarDigital Library
    14. Simoncelli, E. P., and Freeman, W. T. 1995. The steerable pyramid: a flexible architecture for multi-scale derivative computation. In Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3 – Volume 3, IEEE Computer Society, Washington, DC, USA, ICIP ’95, 3444–. Google ScholarDigital Library
    15. Simoncelli, E. P., Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1992. Shiftable multi-scale transforms. IEEE Trans. Info. Theory 2, 38, 587–607. Google ScholarDigital Library
    16. Wang, J., Drucker, S. M., Agrawala, M., and Cohen, M. F. 2006. The cartoon animation filter. ACM Trans. Graph. 25, 1169–1173. Google ScholarDigital Library
    17. Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH) 31 (aug). Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: