“Performance animation from low-dimensional control signals” by Chai and Hodgins

  • ©Jinxiang Chai and Jessica K. Hodgins




    Performance animation from low-dimensional control signals



    This paper introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user’s performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation. We demonstrate the power of this approach with real-time control of six different behaviors using two video cameras and a small set of retro-reflective markers. We compare the resulting animation to animation from commercial motion capture equipment with a full set of markers.


    1. Aha, D. 1997. Editorial, special issue on lazy learning. In Artificial Intelligence Review. 11(1–5):1–6. Google ScholarDigital Library
    2. Arikan, O., and Forsyth, D. A. 2002. Interactive motion generation from examples. In ACM Transactions on Graphics. 21(3):483–490. Google ScholarDigital Library
    3. Arikan, O., Forsyth, D. A., and O’Brien, J. F. 2003. Motion synthesis from annotations. In ACM Transactions on Graphics. 22(3):402–408. Google ScholarDigital Library
    4. Atkeson, C. G., Moore, A. W., and Schaal, S. 1997a. Locally weighted learning. In Artificial Intelligence Review. 11(1–5):11–73. Google ScholarDigital Library
    5. Atkeson, C. G., Moore, A. W., and Schaal, S. 1997b. Locally weighted learning for control. In Artificial Intelligence Review. 11(1–5):75–113. Google ScholarDigital Library
    6. Badler, N. I., Hollick, M., and Granieri, J. 1993. Real-time control of a virtual human using minimal sensors. In Presence. 2(1):82–86.Google ScholarDigital Library
    7. Bazaraa, M. S., Sherali, H. D., and Shetty, C. 1993. Nonlinear Programming: Theory and Algorithms. John Wiley and Sons Ltd. 2nd Edition.Google Scholar
    8. Bishop, C. 1996. Neural Network for Pattern Recognition. Cambridge University Press. Google ScholarDigital Library
    9. Brand, M., and Hertzmann, A. 2000. Style machines. In Proceedings of ACM SIGGRAPH 2000. 183–192. Google ScholarDigital Library
    10. Brand, M. 1999. Shadow puppetry. In Proceedings of IEEE International Conference on Computer Vision. 1237–1244. Google ScholarDigital Library
    11. Bregler, C., and Omohundro, S. 1995. Nonlinear image interpolation using manifold learning. In Advances in Neural Information Processing Systems 7. 973–980.Google Scholar
    12. Cheung, G., Baker, S., and Kanade, T. 2003. Shape-from-silhouette of articulated object and its use for human body kinematics estimation and motion capture. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 77–84. Google ScholarDigital Library
    13. Daelemans, W., and Van De Bosch. A. 2001. Treetalk: Memory-based word phonemisation. In Data-Driven Techniques in Speech Synthesis, Kluwer. 149–172.Google Scholar
    14. Dontcheva, M., Yngve, G., and Popovic, Z. 2003. Layered acting for character animation. In ACM Transactions on Graphics. 22(3):409–416. Google ScholarDigital Library
    15. Freeman, W. T., Anderson, D., Beardsley, P., Dodge, C., Kage, H., Kyuma, K., Miyake, Y., Roth, M., Tanaka, K., Weissman, C., and Yerazunis, W. 1998. Computer vision for interactive computer graphics. In IEEE Computer Graphics and Applications. 18(3):42–53. Google ScholarDigital Library
    16. Fukunaga, K., and Olsen. D. 1971. An algorithm for finding intrinsic dimensionality of data. In IEEE Transactions on Computers. C-20:176–183.Google Scholar
    17. Grochow. K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Style-based inverse kinematics. In ACM Transactions on Graphics. 23(3):522–531. Google ScholarDigital Library
    18. Guo. S., and Roberge, J. 1996. A high level control mechanism for human locomotion based on parametric frame space interpolation. In Eurographics Workshop on Computer Animation and Simulation ’96. 95–107. Google ScholarDigital Library
    19. Hinton. G., Revow, M., and Dayan, P. 1995. Recognizing handwritten digits using mixtures of linear models. In Advances in Neural Information Processing Systems 7. 1015–1022.Google Scholar
    20. Howe. N., Leventon, M., and Freeman, W. 1999. Bayesian reconstruction of 3d human motion from single-camera video. In Advances in Neural Information Processing Systems 12. 820–826.Google Scholar
    21. Konami Boxing and Police 911 GAME, 2001. http://www.konami.com.Google Scholar
    22. Kovar. L., and Gleicher, M. 2004. Automated extraction and parameterization of motions in large data sets. In ACM Transactions on Graphics. 23(3):559–568. Google ScholarDigital Library
    23. Kovar. L., Gleicher, M., and Pighin, F. 2002. Motion graphs. In ACM Transactions on Graphics. 21(3):473–482. Google ScholarDigital Library
    24. Lawrence, N. D. 2004. Gaussian process latent variable models for visualization of high dimensional data. In Advances in Neural Information Processing Systems 16. 329–336.Google Scholar
    25. Lee, J., Chai, J., Reitsma, P., Hodgins, J., and Pollard, N. 2002. Interactive control of avatars animated with human motion data. In ACM Transactions on Graphics. 21(3):491–500. Google ScholarDigital Library
    26. Li, Y., Wang, T., and Shum, H.-Y. 2002. Motion texture: A two-level statistical model for character synthesis. In ACM Transactions on Graphics. 21(3):465–472. Google ScholarDigital Library
    27. Mardia, K., Kent, J., and Bibby, M. 1979. Multivariate Analysis. Academy Press.Google Scholar
    28. MicroStrain 3DM-G, 2004. http://www.microstrain.com.Google Scholar
    29. Oore, S., Terzopoulos, D., and Hinton, G. 2002. A desktop input device and interface for interactive 3d character. In Proceedings of Graphics Interface 2002. 133–140.Google Scholar
    30. Pullen, K., and Bregler, C. 2002. Motion capture assisted animation: Texturing and synthesis. In ACM Transactions on Graphics. 21(3):501–508. Google ScholarDigital Library
    31. Ren, L., Shakhnarovich, G., Hodgins, J. K., Pfister, H., and Viola, P. A. 2004. Learning silhouette features for control of human motion. In Computer Science Technical Reports 2004, Carnegie Mellon University. CMU-CS-04-165.Google Scholar
    32. Rose, C., Cohen, M. F., and Bodenheimer, B. 1998. Verbs and adverbs: Multidimensional motion interpolation. In IEEE Computer Graphics and Applications. 18(5):32–40. Google ScholarDigital Library
    33. Roweis, S., and Saul, L. 2000. Nonlinear dimensionality reduction by locally linear embedding. In Science. 290(5500):2323–2326.Google ScholarCross Ref
    34. Safonova, A., Hodgins, J., and Pollard, N. 2004. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. In ACM Transactions on Graphics. 23(3):514–521. Google ScholarDigital Library
    35. Scholkopf, B., Smola, A., and Muller, K.-R. 1999. Kernel principal component analysis. In Advances in Kernel Methods-SV Learning, MIT Press. 327–352. Google ScholarDigital Library
    36. Semwal, S., Hightower, R., and Stansfield, S. 1998. Mapping algorithms for real-time control of an avatar using eight sensors. In Presence. 7(1):1–21. Google ScholarDigital Library
    37. Shin, H. J., Lee, J., Gleicher, M., and Shin, S. Y. 2001. Computer puppetry: An importance-based approach. In ACM Transactions on Graphics. 20(2):67–94. Google ScholarDigital Library
    38. Sidenbladh, H., Black, M. J., and Sigal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In European Conference on Computer Vision. 784–800. Google ScholarDigital Library
    39. Sony Eye Toy Systems, 2003. http://www.eyetoy.com.Google Scholar
    40. Stone, M. 1974. Cross-validatory choice and assessment of statistical predictions. In Journal of the Royal Statistical Society. 36:111–147.Google Scholar
    41. Tenenbaum, J., Silva, V., and Langford, J. 2000. A global geometric framework for nonlinear dimensionality reduction. In Science. 290(5500):2319–2323.Google ScholarCross Ref
    42. Vicon Systems, 2004. http://www.vicon.com.Google Scholar
    43. Wiley, D. J., and Hahn, J. K. 1997. Interpolation synthesis of articulated figure motion. In IEEE Computer Graphics and Applications. 17(6):39–45. Google ScholarDigital Library
    44. Xsens Mt-9, 2004. http://www.xsens.com.Google Scholar
    45. Xu, G., and Zhang, Z. 1996. Epipolar Geometry in Stereo, Motion, and Object Recognition: A Unified Approach. Kluwer. Google ScholarDigital Library
    46. Yamane, K., and Nakamura, Y. 2003. Natural motion animation through constraining and deconstraining at will. In IEEE Transactions on Visualization and Computer Graphics. 9(3):352–360. Google ScholarDigital Library
    47. Yamane, K., Kuffner, J. J., and Hodgins, J. K. 2004. Synthesizing animations of human manipulation tasks. In ACM Transactions on Graphics. 23(3):532–539. Google ScholarDigital Library
    48. Yin, K., and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 329–338. Google ScholarDigital Library
    49. Zhang, Z. 1999. Flexible camera calibration by viewing a plane from unknown orientations. In Proceedings of the International Conference on Computer Vision. 666–673.Google ScholarCross Ref

ACM Digital Library Publication:

Overview Page: