“Style machines” by Brand and Hertzmann

We approach the problem of stylistic motion synthesis by learning motion patterns from a highly varied set of motion capture sequences. Each sequence may have a distinct choreography, performed in a distinct sytle. Learning identifies common choreographic elements across sequences, the different styles in which each element is performed, and a small number of stylistic degrees of freedom which span the many variations in the dataset. The learned model can synthesize novel motion data in any interpolation or extrapolation of styles. For example, it can convert novice ballet motions into the more graceful modern dance of an expert. The model can also be driven by video, by scripts or even by noise to generate new choreography and synthesize virtual motion-capture in many styles.

1. L. Baum. An inequality and associated maximization technique in statistical estimation of probabilistic functions of Markov processes. Inequalities, 3:1-8, 1972.
2. M. Brand. Pattern discovery via entropy minimization. In D. Heckerman and C. Whittaker, editors, Artificial Intelligence and Statistics #7. Morgan Kaufmann., January 1999.
3. M. Brand. Exploring variational structure by cross-entropy optimization. In P. Langley, editor, P1vceedings, International Conference on Machine Learning, 2000.
4. M. Brand. Voice puppetry. P1vceedings of SIGGRAPH 99, pages 21-28, August 1999.
5. M. Brand. Shadow puppetry. P1vceedings oflCCV 99, September 1999.
6. C. Bregler. Learning and recognizing human dynamics in video sequences. P1vceedings of CVPR 97, 1997.
7. A. Bruderlin and L. Williams. Motion signal processing. P1vceedings of SIG- GRAPH 95, pages 97-104, August 1995.
8. J. Buhmann. Empirical risk approximation: An induction principle for unsupervised leaming. Technical Report IAI-TR-98-3, Institut fiir Intbrmatik III, Universitfit Bonn. 1998., 1998.
9. R.M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and D. E. Knuth. On the Lambert W function. Advances in Computational Mathematics, 5:329-359, 1996.
10. W.T. Freeman and J. B. Tenenbaum. Learning bilinear models for two-thctor problems in vision. In P~vceedings, Conf. on Computer Vision and Pattern Recognition, pages 554-560, San Juan, PR, 1997.
11. M. Gleicher. Motion editing with spacetime constraints. 1997 Symposium on Interactive 3D Graphics, pages 139-148, April 1997.
12. M. Gleicher. Retargeting motion to new characters. P~vceedings of SIGGRAPH 98, pages 33-42, July 1998.
13. R. Grzeszczuk, D. Terzopoulos, and G. Hinton. Neuroanimator: Fast neural network emulation and control of physics-based models. P~vceedings of SIG- GRAPH 98, pages 9-20, July 1998.
14. N.R. Howe, M. E. Leventon, and W. T. Freeman. Bayesian reconstruction of 3d human motion from single-camera video. In S. Solla, T. Leend, and K. Muller, editors, Advances in Neural Information P~vcessing Systems, volume 10. MIT Press, 2000.
15. J. Lee and S. Y. Shin. A hierarchical approach to interactive motion editing for human-like figures. P1vceedings of SIGGRAPH 99, pages 39-48, August 1999.
16. Z. Popovi6 and A. Witkin. Physically based motion transformation. P~vceedings of SIGGRAPH 99, pages 11-20, August 1999.
17. L.R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. P1vceedings of the IEEE, 77(2):257-286, Feb. 1989.
18. C. Rose, M. F. Cohen, and B. Bodenheimer. Verbs and adverbs: Multidimensional motion interpolation. IEEE Computer Graphics & Applications, 18 (5): 32-40, September – October 1998.
19. J.B. Tenenbaum and W. T. Freeman. Separating style and content. In M. Mozer, M. Jordan, and T. Petsche, editors, Advances in Neural Information P~vcessing Systems, volume 9, pages 662-668. MIT Press, 1997.
20. M. Unuma, K. Anjyo, and R. Takeuchi. Fourier principles for emotion-based human figure animation. P~vceedings of SIGGRAPH 95, pages 91-96, August 1995.
21. A. Wilson and A. Bobick. Parametric hidden markov models for gesture recognition. IEEE Trans. Pattern Analysis and Machine Intelligence, 21(9), 1999.
22. A. Witkin and Z. Popovi6. Motion warping. Pivceedings of SIGGRAPH 95, pages 105-108, August 1995.
23. S. C. Zhu, Y. Wu, and D. Mumford. Minimax entropy principle and its applications to texture modeling. Neural Computation, 9(8), 1997.