“Rhythmic-motion synthesis based on motion-beat analysis” by Kim, Park and Shin

Real-time animation of human-like characters is an active research area in computer graphics. The conventional approaches have, however, hardly dealt with the rhythmic patterns of motions, which are essential in handling rhythmic motions such as dancing and locomotive motions. In this paper, we present a novel scheme for synthesizing a new motion from unlabelled example motions while preserving their rhythmic pattern. Our scheme first captures the motion beats from the example motions to extract the basic movements and their transitions. Based on those data, our scheme then constructs a movement transition graph that represents the example motions. Given an input sound signal, our scheme finally synthesizes a novel motion in an on-line manner while traversing the motion transition graph, which is synchronized with the input sound signal and also satisfies kinematic constraints given explicitly and implicitly. Through experiments, we have demonstrated that our scheme can effectively produce a variety of rhythmic motions.

1. ARIKAN, O., AND FORSYTH, D. A. 2002. Interactive Motion Generation from Examples. ACM Transactions on Graphics (Proc. SIGGRAPH 2002) 21, 3 (July), 483–490.]] Google Scholar
2. BEZDEK, J. C. 1981. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York.]] Google Scholar
3. BINDIGANAVALE, R., AND BADLER, N. I. 1998. Motion Abstraction and Mapping with Spatial Constraints. In Proc. International Workshop, CAPTECH ’98, 70–82.]] Google Scholar
4. BLOM, L., AND CHAPLIN, L. T. 1982. The Intimate Act of Choreography. Pittsburgh: University of Pittsburgh Press.]]Google Scholar
5. BOTTOMER, P. 1998. Let’s Dance! Black Dog & Leventhal Publishers.]]Google Scholar
6. BRAND, M., AND HERTZMANN, A. 2000. Style Machines. Computer Graphics (Proc. SIGGRAPH 2000) 34 (July), 183–192.]] Google Scholar
7. BRUDERLIN, A., AND WILLIAMS, L. 1995. Motion Signal Processing. Computer Graphics (Proc. SIGGRAPH ’95) 29 (Aug.), 97–104.]] Google Scholar
8. CASSELL, J., VILHJÁLMSSON, H. H., AND BICKMORE, T. 2001. BEAT: the Behavior Expression Animation Toolkit. Computer Graphics (Proc. SIGGRAPH 2001) 35 (Aug.), 477–486.]] Google Scholar
9. CHIU, S. 1994. Fuzzy Model Identification Based on Cluster Estimation. Journal of Intelligent & Fuzzy Systems 2, 3 (Sept.).]]Google Scholar
10. DUBES, R., AND JAIN, A. K. 1976. Clutering Techniques: The User’s Dilemma. Pattern Recognition 8, 247–260.]]Google ScholarCross Ref
11. ECK, D., GASSER, M., AND PORT, R. 2000. Dynamics and Embodiment in Beat Induction. In Rhythm Perception and Production, 157–170.]]Google Scholar
12. ESSENS, P. J., AND POVEL, D. 1985. Metrical and Nonmetrical Representation of Temporal Patterns. Perception and Psychophysics 37, 1–7.]]Google ScholarCross Ref
13. FOGEL, E., AND GAVISH, M. 1988. Parameter Estimation of Quasiperiodic Sequences. In IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, 2348–2351.]]Google Scholar
14. FORGY, E. 1965. Cluster Analysis of Multivariate Data: Efficiency vs. Interpretability of Classifications. Biometrics 21, 768.]]Google Scholar
15. GALATA, A., JOHNSON, N., AND HOGG, D. 2001. Learning Variable-Length Markov Models of Behavior. Computer Vision and Image Understanding 81, 398–413.]] Google ScholarDigital Library
16. GLEICHER, M. 1998. Retargetting Motion to New Characters. Computer Graphics (Proc. SIGGRAPH ’98) 32 (July), 33–42.]] Google Scholar
17. GOMES, J., AND VELHO, L. 1999. From Fourier Analysis to Wavelets. In Course Notes #5 (SIGGRAPH ’99).]]Google Scholar
18. GUO, S., AND ROBERGÉ, J. 1996. A High-level Control Mechanism for Human Locomotion Based on Parametric Frame Space Interpolation. In Proc. Eurographics Workshop on Computer Animation and Simulation 96, 97–107.]] Google Scholar
19. HAWKINS, A. 1988. Creating through dance, revised ed. Princeton.]]Google Scholar
20. HELBING, D., AND MOLNAR, P. 1995. Social force model for pedestrian dynamics. Physical Review E 51, 4282–4286.]]Google ScholarCross Ref
21. HELBING, D., FARKAS, I. J., AND T. VICSEK. 2000. Simulating dynamic features of escape panic. Nature 407, 487–490.]]Google ScholarCross Ref
22. HUBER, P. J. 1981. Robust Statistics. John Wiley & Sons, New York.]]Google Scholar
23. J VAN CAMP. 1994. Copyright of Choreographic Works. 1994–1995 Entertainment, Publishing and the Arts Handbook, 59–92.]]Google Scholar
24. JONES, M. R., AND BOLTZ, M. 1989. Dynamic Attending and Responses to Time. Psychological Review 96, 3, 459–491.]]Google ScholarCross Ref
25. KIM, D., KIM, H. K., AND SHIN, S. Y. 2003. An Event-driven Approach to Crowd Simulation with Example Motions. Tech. Rep. CS-TR-2002-186, Dept. Comput. Sci., KAIST, Taejon, Korea (Republic of).]]Google Scholar
26. KIM, M. J. 1996. General Schemes for Unit Quaternion Curve Construction. PhD thesis, DCS88343, Department of Computer Science, KAIST.]]Google Scholar
27. KIM, H. K. 2002. Efficient Collision Detection Among Moving Spheres with Unknown Trajectories. PhD thesis, DCS02009, Department of Computer Science, KAIST.]]Google Scholar
28. KOVAR, L., GLEICHER, M., AND PIGHIN, F. 2002. Motion Graphs. ACM Transactions on Graphics (Proc. SIGGRAPH 2002) 21, 3 (July), 473–482.]] Google Scholar
29. LARGE, E. W., AND KOLEN, J. F. 1994. Resonance and the Perception of Musical Meter. Connection Science 6, 2/3, 177–208.]]Google Scholar
30. LAYBOURNE, K. 1998. The Animation Book. Three Rivers Press.]]Google Scholar
31. LEE, J., AND SHIN, S. Y. 1999. A Hierarchical Approach to Interactive Motion Editing for Human-like Figures. Computer Graphics (Proc. SIGGRAPH ’99) 33 (Aug.), 395–408.]] Google Scholar
32. LEE, J., CHAI, J., REITSMA, P. S. A., HODGINS, J. K., AND POLLARD, N. S. 2002. Interactive Control of Avatars Animated with Human Motion Data. ACM Transactions on Graphics (Proc. SIGGRAPH 2002) 21, 3 (July), 491–500.]] Google Scholar
33. LI, X., AND PARIZEAU, M. 2000. Training Hidden Markov Models with Multiple Observations – A Combinatorial Method. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 4 (Apr.), 371–377.]] Google Scholar
34. LI, Y., WANG, T., AND SHUM, H. 2002. Motion Texture: A Two-Level Statistical Model for Character Motion Synthesis. ACM Transactions on Graphics (Proc. SIGGRAPH 2002) 21, 3 (July), 465–472.]] Google Scholar
35. MINTON, S. C. 1997. Choreography: A Basic Approach Using Improvisation, 2nd ed. Human Kinetics.]]Google Scholar
36. PARK, S. I., SHIN, H. J., AND SHIN, S. Y. 2002. On-line Locomotion Generation Based on Motion Blending. In Proc. ACM SIGGRAPH Symposium on Computer Animation, 105–111.]] Google Scholar
37. PAUSCH, R., SNODDY, J., TAYLOR, R., WATSON, S., AND HASELTINE, E 1996. Disney’s Aladdin: First Steps Toward Storytelling in Virtual Reality. Computer Graphics (Proc. SIGGRAPH ’96) 30 (Aug.), 193–203.]] Google Scholar
38. POPOVIć, Z., AND WITKIN, A. 1999. Physically-based Motion Transformation. Computer Graphics (Proc. SIGGRAPH ’99) 33, 11–20.]] Google Scholar
39. PRESS, W. H., TEUKOLSKY, S. A., VETTERLING, W. T., AND FLANNERY, B. P. 1999. Numerical Recipes in C – The Art of Scientific Computing, second ed. Cambridge University Press.]] Google Scholar
40. PROAKIS, J. G., AND MANOLAKIS, D. G. 1996. Digital Signal Processing: Principles, algorithms, and applications, third ed. Prentice Hall International Editions.]] Google Scholar
41. PULLEN, K., AND BREGLER, C. 2002. Motion Capture Assisted Animation: Texturing and Synthesis. ACM Transactions on Graphics (Proc. SIGGRAPH 2002) 21, 3 (July), 501–508.]] Google Scholar
42. RABINER, L. R. 1989. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proc. the IEEE 77, 2 (Feb.), 257–286.]]Google ScholarCross Ref
43. ROSE, C., GUENTER, B., BODENHEIMER, B., AND COHEN, M. F. 1996. Efficient Generation of Motion Transitions using Spacetime Constraints. Computer Graphics (Proc. SIGGRAPH ’96) 30 (Aug.), 147–154.]] Google Scholar
44. ROSE, C., BODENHEIMER, B., AND COHEN, M. 1998. Verbs and Adverbs: Multidimensional Motion Interpolation. IEEE Computer Graphics and Applications 18, 5, 32–40.]] Google ScholarDigital Library
45. SADLER, B. M., AND CASEY, S. D. 1998. On Periodic Pulse Interval Analysis with Outliers and Missing Observations. IEEE Transactions on Signal Processing 46, 11 (November), 2990–3002.]]Google ScholarDigital Library
46. SCARBOROUGH, D., MILLER, P., AND JONES, J. A. 1992. On the perception of Meter. In M. Balaban, K. Ebcioglu, and O. Laske (Eds.) Understanding Music with AI: Perspectives in Music Cognition, Cambridge:MIT Press, 427–447.]] Google Scholar
47. SCHÖDL, A., SZELISKI, R., SALESIN, D. H., AND ESSA, I. 2000. Video Textures. Computer Graphics (Proc. SIGGRAPH 2000) 34 (July), 489–498.]] Google Scholar
48. SHIN, H. J., LEE, J., GLEICHER, M., AND SHIN, S. Y. 2001. Computer Puppetry: An Importance-Based Approach. ACM Transactions On Graphics 20.]] Google Scholar
49. SHOEMAKE, K. 1985. Animating Rotation with Quaternion Curves. Computer Graphics (Proc. SIGGRAPH ’85) 19 (Aug.), 245–54.]] Google Scholar
50. SILVESTER, V. 1993. Modern Ballroom Dancing, new rev. ed. Trafalgar Square.]]Google Scholar
51. SLOAN, P., ROSE, C. F., AND COHEN, M. F. 2001. Shape by Example. In Proc. of 2001 ACM Symposium on Interactive 3D Graphics, 135–144.]] Google Scholar
52. TANCO, L. M., AND HILTON, A. 2000. Realistic Synthesis of Novel Human Movements from a Database of Motion Capture Examples. In Proc. the IEEE Workshop on Human Motion, 137–142.]] Google ScholarCross Ref
53. UNUMA, M., ANJYO, K., AND TAKEUCHI, R. 1995. Fourier Principles for Emotion-based Human Figure Animation. Computer Graphics (Proc. SIGGRAPH ’95) 29 (Aug.), 91–96.]] Google Scholar
54. WILEY, D., AND HAHN, J. K. 1997. Interpolation Synthesis of Articulated Figure Motion. IEEE Computer Graphics and Applications 17, 6, 39–45.]] Google ScholarDigital Library
55. WITKIN, A., AND POPOVIĆ, Z. 1995. Motion Warping. Computer Graphics (Proc. SIGGRAPH ’95) 29 (Aug.), 105–108.]] Google Scholar