“Integrated learning for interactive synthetic characters”

  • ©Bruce Blumberg, Marc Downie, Yuri A. Ivanov, Matt Berlin, Michael Patrick Johnson, and Bill Tomlinson

  • ©Bruce Blumberg, Marc Downie, Yuri A. Ivanov, Matt Berlin, Michael Patrick Johnson, and Bill Tomlinson

Conference:


Type:


Title:

    Integrated learning for interactive synthetic characters

Presenter(s)/Author(s):



Abstract:


    The ability to learn is a potentially compelling and important quality for interactive synthetic characters. To that end, we describe a practical approach to real-time learning for synthetic characters. Our implementation is grounded in the techniques of reinforcement learning and informed by insights from animal training. It simplifies the learning task for characters by (a) enabling them to take advantage of predictable regularities in their world, (b) allowing them to make maximal use of any supervisory signals, and (c) making them easy to train by humans.We built an autonomous animated dog that can be trained with a technique used to train real dogs called “clicker training”. Capabilities demonstrated include being trained to recognize and use acoustic patterns as cues for actions, as well as to synthesize new actions from novel paths through its motion space.A key contribution of this paper is to demonstrate that by addressing the three problems of state, action, and state-action space discovery at the same time, the solution for each becomes easier. Finally, we articulate heuristics and design principles that make learning practical for synthetic characters.

References:


    1. BALLARD, D. 1997. An Introduction to Natural Computation. MIT Press, Cambridge, MA. Google Scholar
    2. BLUMBERG, B., AND GAYLEAN, T. 1995. Multi-level direction of autonomous creatures for real-time virtual environments. In Proceedings of SIGGRAPH 1995, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    3. BURKE, R., ISLA, D., DOWNIE, M., IVANOV, Y., AND BLUMBERG, B. 2001. Creature smarts: The art and architecture of a virtual brain. In Proceedings of the Computer Game Developers Conference.Google Scholar
    4. BURKE, R. 2001. Its about Time:Temporal Representation for Synthetic Characters. Master’s thesis, The Media Lab, MIT.Google Scholar
    5. COPPINGER, R., AND COPPINGER, L. 2001. Dogs: A Startling New Understanding of Canine Origin, Behavior, and Evolution. Scribner, New York, NY.Google Scholar
    6. DOWNIE, M. 2000. behavior, animation, music: the music and movement of synthetic characters. Master’s thesis, The Media Lab, MIT.Google Scholar
    7. DRESCHER, G. 1991. Made-Up Minds:A Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge MA. Google Scholar
    8. EVANS, R. 2002. Varieties of learning. In AI Game Programming Wisdom, E. Rabin, Ed. Charles River Media, Hingham MA. Google Scholar
    9. FALOUTSOS, P., VAN DE PANNE, M., AND TERZOPOLOUS, D. 2001. Composible controllers for physics-based character animation. In Proceedings of SIGGRAPH 2001, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    10. FUNGE, J., TU, X., AND TERZOPOLOUS, D. 1999. Cognitive modeling: Knowledge, reasoning and planning for intelligent characters. In Proceedings of SIGGRAPH 1999, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    11. GALLISTEL, C. R., AND GIBBON, J. 2000. Time, rate and conditioning. Psychological Review 107.Google Scholar
    12. GLEICHER, M. 1998. Retargetting motion to new characters. In Proceedings of SIGGRAPH 1998, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    13. GOULD, J., AND GOULD, C. 1999. The Animal Mind. W.H. Freeman, New York, NY.Google Scholar
    14. GRAND, S., CLIFF, D., AND MALHOTRA, A. 1996. Creatures: Artificial life autonomous agents for home entertainment. In Proceedings of the Autonomous Agents ’97 Conference. Google Scholar
    15. GRZESZCZUK, R., AND TERZOPOULOS, D. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proceedings of SIGGRAPH 1995, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    16. GRZESZCZUK, R., TERZOPOULOS, D., AND HINTON, G. 1998. Neuroanimator: Fast neural network emulation and control of physics-based models. In Proceedings of SIGGRAPH 1998, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    17. HODGINS, J., AND POLLARD, N. 1997. Adapting simulated behaviors for new characters. In Proceedings of SIGGRAPH 1997, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    18. ISLA, D., BURKE, R., DOWNIE, M., AND BLUMBERG, B. 2001. A layered brain architecture for synthetic creatures. In Proceedings of The International Joint Conference on Artificial Intelligence. Google Scholar
    19. ISLA, D. 2001. The Virtual Hippocampus: Spatial Common Sense for Synthetic Creatures. Master’s thesis, The Media Lab, MIT.Google Scholar
    20. IVANOV, Y., BLUMBERG, B., AND PENTLAND, A. 2001. Expectation maximization for weakly labeled data. In Proceedings of the 18th International Conference on Machine Learning. Google Scholar
    21. IVANOV, Y. 2001. State Discovery for Autonomous Creatures. PhD thesis, The Media Lab, MIT.Google Scholar
    22. KAELBLING, L. 1990. Learning in embedded systems. PhD thesis, Stanford University. Google Scholar
    23. KAPLAN, F., OUDEYER, P.-Y., KUBINYI, E., AND MIKLOSI, A. 2001. Taming robots with clicker training : a solution for teaching complex behaviors. In Proceedings of the 9th European workshop on learning robots, LNAI, Springer, M. Quoy, P. Gaussier, and J. L. Wyatt, Eds.Google Scholar
    24. LINDSAY, S. 2000. Applied Dog Behavior and Training. Iowa State University Press, Ames, IA.Google Scholar
    25. LORENZ, K., AND LEYAHUSEN, P. 1973. Motivation of Human and Animal Behavior: An Ethological View. Van Nostrand Reinhold Co., New York, NY.Google Scholar
    26. LORENZ, K. 1981. The Foundations of Ethology. Springer-Verlag, New York, NY.Google Scholar
    27. MITCHELL, K. 1997. Machine Learning. McGraw Hill, New York, NY. Google Scholar
    28. PERLIN, K., AND GOLDBERG, A. 1996. Improv: A system for scripting interactive actors in virtual worlds. In Proceedings of SIGGRAPH 1996, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    29. PRYOR, K. 1999. Clicker Training for Dogs. Sunshine Books, Inc., Waltham, MA.Google Scholar
    30. RABINER, L., AND JUANG, B.-H. 1993. Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs, NJ. Google Scholar
    31. RAMIREZ, K. 1999. Animal Training:Successful Animal Management Through Positive Reinforcement. Shedd Aquarium, Chicago, IL.Google Scholar
    32. RESNER, B., STERN, A., AND FRANK, A. 1997. The truth about catz and dogz. In The Computer Games Developer Conference.Google Scholar
    33. REYNOLDS, C. 1987. Flocks, herds and schools: A distributed behavioral model. In Proceedings of SIGGRAPH 1987, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    34. ROSE, C., COHEN, M., AND BODENHEIMER, B. 1999. Verbs and adverbs: Multidimensional motion interpolation. IEEE Computer Graphics And Applications 18, 5. Google Scholar
    35. SHETTLEWORTH, S. J. 1998. Cognition, Evolution and Behavior. Oxford University Press, New York, NY.Google Scholar
    36. SUTTON, R., AND BARTO, A. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge MA. Google Scholar
    37. SUTTON, R. 1991. Reinforcement learning architectures for animats. In The First International Conference on Simulation of Adaptive Behavior, MIT Press, Paris, Fr. Google Scholar
    38. THERRIEN, C. 1989. Decision Estimation and Classification: An Introduction to Pattern Recognition and Related Topics. John Wiley and Sons, New York, NY. Google Scholar
    39. TOMLINSON, B., AND BLUMBERG, B. 2002. Alphawolf: Social learning, emotion and development in autonomous virtual agents. In First GSFC/JPL Workshop on Radical Agent Concepts.Google Scholar
    40. TU, X., AND TERZOPOULOS, D. 1994. Artificial fishes: Physics, locomotion, perception, behavior. In Proceedings of SIGGRAPH 1994, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    41. VAN DE PANNE, M., AND FIUME., E. 1993. Sensor-actuator networks. In Proceedings of SIGGRAPH 1993, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM. Google Scholar
    42. VAN DE PANNE, M., KIM, R., AND FIUME., E. 1994. Synthesizing parameterized motions. In 5th Eurographics Workshop on Simulation and Animation.Google Scholar
    43. WATKINS, C. J., AND DAYAN, P. 1992. Q-learning. Machine Learning 8. Google Scholar
    44. WILKES, G. 1995. Click and Treat Training Kit. Click and Treat Inc., Mesa, AZ.Google Scholar
    45. YOON, S., BLUMBERG, B., AND SCHNEIDER, G. 2000. Motivation-driven learning for interactive synthetic characters. In Proceedings of the Fourth International Conference on Autonomous Agents. Google Scholar
    46. YOON, S., BURKE, R., AND BLUMBERG, B. 2000. Interactive training for synthetic characters. In Proceedings of AAAI 2000. Google Scholar


ACM Digital Library Publication: