“Motion texture: a two-level statistical model for character motion synthesis”

In this paper, we describe a novel technique, called motion texture, for synthesizing complex human-figure motion (e.g., dancing) that is statistically similar to the original motion captured data. We define motion texture as a set of motion textons and their distribution, which characterize the stochastic and dynamic nature of the captured motion. Specifically, a motion texton is modeled by a linear dynamic system (LDS) while the texton distribution is represented by a transition matrix indicating how likely each texton is switched to another. We have designed a maximum likelihood algorithm to learn the motion textons and their relationship from the captured dance motion. The learnt motion texture can then be used to generate new animations automatically and/or edit animation sequences interactively. Most interestingly, motion texture can be manipulated at different levels, either by changing the fine details of a specific motion at the texton level or by designing a new choreography at the distribution level. Our approach is demonstrated by many synthesized sequences of visually compelling dance motion.

1. O. Arikan and D. Forsyth. Interactive motion generation from examples. In Proceedings of ACM SIGGRAPH 02, 2002. Google Scholar
2. Z. Bar-Joseph, R. El-Yaniv, D. Lischiniski, and M. Werman. Texture movies: Statistical learning of time-varying textures. IEEE Transactions on Visualization and Computer Graphics, 7(1):120-135, 2001. Google Scholar
3. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995. Google Scholar
4. B. Bodenheimer, A. Shleyfman, and J. Hodgins. The effects of noise on the perception of animated human running. In Computer Animation and Simulation ’99, Eurographics Animation Workshop, pages 53-63, 1999.Google Scholar
5. R. Bowden. Learning statistical models of human motion. In IEEE Workshop on Human Modeling, Analysis and Synthesis, CVPR, 2000.Google Scholar
6. M. Brand and A. Hertzmann. Style machines. In Proceedings of ACM SIGGRAPH 00, pages 183-192, 2000. Google Scholar
7. C. Bregler. Learning and recognizing human dynamics in video sequences. In Int. Conf. on Computer Vision and Pattern Rocognition, pages 568-574, 1997. Google Scholar
8. A. Bruderlin and L. Williams. Motion signal processing. In Proceedings of ACM SIGGRAPH 95, pages 97-104, 1995. Google Scholar
9. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. The MIT Press, 1997. Google Scholar
10. J. de Bonet. Multiresolution sampling procedure for analysis and synthesis of texture images. In Proceedings of ACM SIGGRAPH 97, pages 361-368, 1997. Google Scholar
11. N. M. Dempster, A. P. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. B, 39:185-197, 1977.Google Scholar
12. A. Efros and W. Freeman. Image quilting for texture synthesis and transfer. In Proceedings of ACM SIGGRAPH 01, pages 341-346, 2001. Google Scholar
13. A. W. Fitzgibbon. Stochastic rigidity: Image registration for nowhere-static scenes. In IEEE Int. Conf. on Computer Vision, pages 662-669, 2001.Google Scholar
14. A. Galata, N. Johnson, and D. Hogg. Learning variable length Markov models of behaviour. Computer Vision and Image Understanding, 81(3):398-413, 2001. Google Scholar
15. M. Gleicher. Retargetting motion to new characters. In Proceedings of ACM SIGGRAPH 98, pages 33-42, 1998. Google Scholar
16. F. S. Grassia. Practical parameterization of rotations using the exponential map. Journal of Graphics Tools, 3(3):29-48, 1998. Google Scholar
17. C. E. Guo, S. C. Zhu, and Y. N. Wu. Visual learning by integrating descriptive and generative methods. In Int. Conf. Computer Vision, 2001.Google Scholar
18. J. K. Hodgins, W. L. Wooten, D. C. Brogan, and J. F. O’Brien. Animating human athletics. In Proceedings of ACM SIGGRAPH 95, pages 71-78, 1995. Google Scholar
19. M. Isard and A. Blake. Condensation – conditional density propagation for visual tracking. Int’l J. Computer Vision, 28(1):5-28, 1998. Google Scholar
20. B. Julesz. Textons, the elements of texture perception and their interactions. Nature, 290:91-97, 1981.Google Scholar
21. L. Kovar, M. Gleicher, and F. Pighin. Motion graphs. In Proceedings of ACM SIGGRAPH 02, 2002. Google Scholar
22. J. Lee, J. Chai, P. Reisma, and J. Hodgins. Interactive control of avartas animated with human motion data. In Proceedings of ACM SIGGRAPH 02, 2002. Google Scholar
23. J. Lee and S. Y. Shin. A hierarchical approach to interactive motion editing for human-like figures. In Proceedings of ACM SIGGRAPH 99, pages 39-48, 1999. Google Scholar
24. T. Leung and J. Malik. Recognizing surfaces using three-dimensional textons. In Int. Conf. on Computer Vision, 1999. Google Scholar
25. L. Liang, C. Liu, Y. Xu, B. Guo, and H.-Y. Shum. Real-time texture synthesis by patch-based sampling. Technical Report MSR-TR-2001-40, Microsoft Research, 2001.Google Scholar
26. L. Ljung. System Identification – Theory for the User. Prentice Hall, Upper Saddle River, N.J., 2nd edition, 1999. Google Scholar
27. J. Malik, S. Belongie, J. Shi, and T. Leung. Textons, contours and regions: Cue integration in image segmentation. In Int. Conf. Computer Vision, pages 918-925, 1999. Google Scholar
28. B. North, A. Blake, M. Isard, and J. Rittscher. Learning and classification of complex dynamics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9):1016-1034, Sep. 2000. Google Scholar
29. V. Pavlović, J. M. Rehg, T. J. Cham, and K. P. Murphy. A dynamic Bayesian network approach to figure tracking using learned dynamic models. In IEEE International Conference on Computer Vision, 1999. Google Scholar
30. V. Pavlović, J. M. Rehg, and J. MacCormick. Impact of dynamic model learning on classification of human motion. In IEEE International Conference on Computer Vision and Pattern Recognition, 2000.Google Scholar
31. K. Perlin. An image synthesizer. In Computer Graphics (Proceedings of ACM SIGGRAPH 85), pages 287-296, 1985. Google Scholar
32. K. Perlin and A. Goldberg. Improv: A system for scripting interactive actors in virtual worlds. In Proceedings of ACM SIGGRAPH 96, pages 205-216, 1996. Google Scholar
33. Z. Popović and A. Witkin. Physically based motion transformation. In Proceedings of ACM SIGGRAPH 99, pages 11-20, 1999. Google Scholar
34. K. Pullen and C. Bregler. http://graphics.stanford.edu/~pullen/motion_texture.Google Scholar
35. K. Pullen and C. Bregler. Motion capture assisted animation: Texturing and synthesis. In Proceedings of ACM SIGGRAPH 02, 2002. Google Scholar
36. L. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-285, February 1989.Google Scholar
37. A. Schödl, R. Szeliski, D. H. Salesin, and I. Essa. Video textures. In Proceedings of ACM SIGGRAPH 00, pages 489-498, 2000. Google Scholar
38. H. J. Shin, J. Lee, M. Gleicher, and S. Y. Shin. Computer puppetry: An importance-based approach. ACM Transactions on Graphics, 20(2):67-94, April 2001. Google Scholar
39. K. Shoemake. Animating rotation with quaternion curves. In Computer Graphics (Proceedings of ACM SIGGRAPH 85), pages 245-254, 1985. Google Scholar
40. S. Soatto, G. Doretto, and Y. N. Wu. Dynamic textures. In IEEE International Conference on Computer Vision, pages 439-446, 2001.Google Scholar
41. L. M. Tanco and A. Hilton. Realistic synthesis of novel human movements from a database of motion capture examples. In IEEE Workshop on Human Motion, 2000. Google Scholar
42. M. Unuma, K. Anjyo, and R. Takeuchi. Fourier principles for emotion-based human figure animation. In Proceedings of ACM SIGGRAPH 95, pages 91-96, 1995. Google Scholar
43. A. Witkin and Z. Popović. Motion warping. In Proceedings of ACM SIGGRAPH 95, pages 105-108, 1995. Google Scholar
44. S. C. Zhu, C. E. Guo, Y. Wu, and Y. Wang. What are textons. In Proc. of European Conf. on Computer Vision (ECCV), 2002. Google Scholar