“Animating pictures with stochastic motion textures” by Chuang, Goldman, Zheng, Curless, Salesin, et al. …

In this paper, we explore the problem of enhancing still pictures with subtly animated motions. We limit our domain to scenes containing passive elements that respond to natural forces in some fashion. We use a semi-automatic approach, in which a human user segments the scene into a series of layers to be individually animated. Then, a “stochastic motion texture” is automatically synthesized using a spectral method, i.e., the inverse Fourier transform of a filtered noise spectrum. The motion texture is a time-varying 2D displacement map, which is applied to each layer. The resulting warped layers are then recomposited to form the animated frames. The result is a looping video texture created from a single still image, which has the advantages of being more controllable and of generally higher image quality and resolution than a video texture created from a video source. We demonstrate the technique on a variety of photographs and paintings.

1. Aoki, M., Shinya, M., Tsutsuguchi, K., and Kotani, N. 1999. Dynamic texture: Physically-based 2D animation. In ACM SIGGRAPH 1999 Conference Sketches and Applications, 239. Google ScholarDigital Library
2. Barrett, W. A., and Cheney, A. S. 2002. Object-based image editing. ACM Transactions on Graphics 21, 3, 777–784. Google ScholarDigital Library
3. Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. 2000. Image inpainting. In Proceedings of ACM SIGGRAPH 2000, 417–424. Google ScholarDigital Library
4. Chuang, Y.-Y., Curless, B., Salesin, D. H., and Szeliski, R. 2001. A Bayesian approach to digital matting. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2001, vol. II, 264–271.Google Scholar
5. Criminisi, A., Reid, I. D., and Zisserman, A. 2000. Single view metrology. International Journal of Computer Vision 40, 2, 123–148. Google ScholarDigital Library
6. Criminisi, A., Perez, P., and Toyama, K. 2003. Object removal by exemplar-based inpainting. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2003, vol. II, 721–728.Google Scholar
7. Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Transactions on Graphics 22, 3, 303–312. Google ScholarDigital Library
8. Feynman, R. P., Leighton, R. B., and Sands, M. 1964. The Feynman Lectures On Physics, Volume II: Mainly Electromagnetism and Matter. Addison Wesley, Reading, Mass.Google Scholar
9. Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1991. Motion without movement. Computer Graphics (Proceedings of ACM SIGGRAPH 91) 25, 4, 27–30. Google ScholarDigital Library
10. Griffiths, D., 1997. Lake java applet. http://www.jaydax.co.uk/tutorials/laketutorial/dgclassfiles.html.Google Scholar
11. Hathaway, T., Bowers, D., Pease, D., and Wendel, S., 2003. http://www.mechanicalmusicpress.com/history/pianella/p40.htm.Google Scholar
12. Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make a nimation from a single image. In Proceedings of ACM SIGGRAPH 1997, 225–232. Google ScholarDigital Library
13. Jia, J., and Tang, C.-K. 2003. Image repairing: Robust image synthesis by adaptive ND tensor voting. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2003, vol. I, 643–650. Google ScholarDigital Library
14. Li, Y., Wang, T., and Shum, H.-Y. 2002. Motion texture: a two-level statistical model for character motion synthesis. ACM Transactions on Graphics 21, 3, 465–472. Google ScholarDigital Library
15. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Transactions on Graphics 23, 3, 303–308. Google ScholarDigital Library
16. Litwinowicz, P., and Williams, L. 1994. Animating images with drawings. In Proceedings of ACM SIGGRAPH 1994, 409–412. Google ScholarDigital Library
17. Mastin, G. A., Watterberg, P. A., and Mareda, J. F. 1987. Fourier synthesis of ocean scenes. IEEE Computer Graphics and Applications 7, 3, 16–23. Google ScholarDigital Library
18. Mortensen, E. N., and Barrett, W. A. 1995. Intelligent scissors for image composition. In Proceedings of ACM SIGGRAPH 1995, 191–198. Google ScholarDigital Library
19. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proceedings of ACM SIGGRAPH 2001, 433–442. Google ScholarDigital Library
20. Porter, T., and Duff, T. 1984. Compositing digital images. Computer Graphics (Proceedings of ACM SIGGRAPH 84) 18, 4, 253–259. Google ScholarDigital Library
21. Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut — interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23, 3, 309–314. Google ScholarDigital Library
22. Ruzon, M. A., and Tomasi, C. 2000. Alpha estimation in natural images. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2000, 18–25.Google Scholar
23. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of ACM SIGGRAPH 2000, 489–498. Google ScholarDigital Library
24. Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. In Proceedings of ACM SIGGRAPH 1998, 231–242. Google ScholarDigital Library
25. Shinya, M., and Fournier, A. 1992. Stochastic motion — motion under the influence of wind. Computer Graphics Forum 11, 3, 119–128.Google ScholarCross Ref
26. Shinya, M., Mori, T., and Osumi, N. 1998. Periodic motion synthesis and Fourier compression. The Journal of Visualization and Computer Animation 9, 3, 95–107.Google ScholarCross Ref
27. Simiu, E., and Scanlan, R. H. 1986. Wind Effects on Structures. John Wiley & Sons.Google Scholar
28. Soatto, S., Doretto, G., and Wu, Y. N. 2001. Dynamic textures. In Proceedings of IEEE International Conference on Computer Vision (ICCV) 2001, 439–446.Google Scholar
29. Stam, J., and Fiume, E. 1993. Turbulent wind fields for gaseous phenomena. In Proceedings of ACM SIGGRAPH 1993, 369–376. Google ScholarDigital Library
30. Stam, J. 1995. Multi-Scale Stochastic Modelling of Complex Natural Phenomena. PhD thesis, Dept. of Computer Science, University of Toronto. Google ScholarDigital Library
31. Stam, J. 1997. Stochastic dynamics: Simulating the effects of turbulence on flexible structures. Computer Graphics Forum 16, 3, 159–164.Google ScholarCross Ref
32. Sun, M., Jepson, A. D., and Fiume, E. 2003. Video input driven animation (VIDA). In Proceedings of IEEE International Conference on Computer Vision (ICCV) 2003, 96–103. Google ScholarDigital Library
33. Szummer, M., and Picard, R. W. 1996. Temporal texture modeling. In Proceedings of IEEE International Conference on Image Processing (ICIP) 1996, vol. 3, 823–826.Google ScholarCross Ref
34. Tessendorf, J. 2001. Simulating ocean water. ACM SIGGRAPH 2001 course notes No. 47 Simulating Nature: Realistic and Interactive Techniques.Google Scholar
35. Treuille, A., McNamara, A., Popović, Z., and Stam, J. 2003. Keyframe control of smoke simulations. ACM Trans. Graph. 22, 3, 716–723. Google ScholarDigital Library
36. Wang, Y., and Zhu, S. C. 2003. Modeling textured motion: Particle, wave and sketch. In Proceedings of IEEE International Conference on Computer Vision (ICCV) 2003, 213–220. Google ScholarDigital Library
37. Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Proceedings of ACM SIGGRAPH 2000, 479–488. Google ScholarDigital Library