Joseph Bates, James Altucher, Alexander Hauptman, Mark Kantrowitz, A. Bryan Loyall, Koichi Murakami, Paul Olbrich, Zoran Popovic, W. Scott Reilly, Phoebe Sengers, William Welch, Paul Weyhrauch, Andrew Witkin: Edge of Intention

  • ©,

Artist Statement:


    Computing and art are becoming increasingly intertwined. Clear evidence of this appears regularly in the film industry, the music industry, and in exhibits such as this one. Artificial intelligence (Al) technologies and fundamental Al research have an essential role to play in this convergence, especially in the domain of interactive art.

    A work of interactive art creates and responds to details of the interaction that the artist could not precisely anticipate when the work was constructed. The more sophisticated the interactivity, the more responsible the work is for carrying out the artist’s intent. This means the artist must strive to create an active entity with a sense of aesthetics and of meaning with the ability to understand and respond to human reactions.

    In constructing these subtle, behaving entities, traditional technical concerns of Al arise. These include how to transfer knowledge into the machine, how to represent that knowledge, and how to process it in ways usually attributed only to humans.

    The work shown here is an attempt to create Al-based art. It is intended to be an emotionally compelling glimpse of how new kinds of machines ultimately may affect our ethics and our culture.

    This work appears as a real-time, animated, simulated world displayed on a monitor. The 3D space, presented from a fixed point of view, contains three creatures that interact with each other and with a fourth usercontrolled creature.

    Each creature is an autonomous entity that seems to perceive its environment, act in response to those perceptions, tries to achieve its desires, react emotionally to events that occur, and form simple relationships with other creatures. Each creature has a distinct personality. The behavior of the creatures is not random, but it is not easily predictable either.

    We call these creatures “woggles.” The particular creatures are named “Bear” (blue), “Wolf” (red), and “Shrimp” (orange), to suggest their personalities. The woggles are autonomous. They “live” in their space and interact with each other whether a human is present or not. However, a human can enter the world, controlling a fourth woggle using a two-button mouse.

    The primary physical repertoire of the woggles is to jump, slide, change body shape, move the eyes, and change eye shape. Two varieties of action have special social significance.

    The first, called “hey,” is to stretch the body upward for a moment while looking at a particular creature. This may be interpreted as a greeting, as simply  acknowledging a creature, or as expressing sympathy. It may also be a request to play “follow the leader,” an agreement to play that game, or a request to finish playing the game. Creatures try to determine the intended meaning in context.

    The second action of special significance is called “in your face.” The creature looks at another while pulling in its chest and expanding sideways. This makes the first creature look larger in the field of view of the second, and it is generally interpreted as a threatening behavior. Threats are quite effective in getting the attention of other creatures, but more subtle interaction and relationships are available using “hey.”

    The user woggle (purple), is treated by the others exactly as they treat each other. Using the mouse, the user can jump, slide, control the eyes, hey, and threaten. Aggressive users will generally provoke fear, anger, sadness, and dislike. Attentive, gentle users will generally comfort, please, play with, and make friends with the other woggles.

    Emotion terms have technical analogs within the minds of the woggles, and thus we are simultaneously speaking technically and subjectively. Our artistic concern is whether the technical capabilities of the woggles clearly convey these subjective internal states, and whether this then creates appropriate emotional and intellectual responses in the human participants.

    The woggle’s world is modeled as a surface defined over a rectangle. The surface is painted, for human viewers, but the woggles see only its shape. In addition, they see the position, body configuration, and actions of other creatures. Each woggle body is controlled by a personality written in Tok, an architecture for the mind that integrates reactivity, goal-directed behavior, and emotion.

    Each Tok mind maintains a tree of top-level goals and the subgoals they engender. The goals and subgoals are used to organize a collection of reactive mechanisms. There is no complete internal model of the world, but there are small, partial models built through selective sensing processes. Goal success, failure, prospective failure, and judgments that other creatures caused these situations give rise to happiness, sadness, fear, gratitude, and anger in varying degrees. Gratitude and anger affect the longer-term relationship of liking or disliking the creature causing the emotion. The emotional state indirectly modulates the creature’s choice of goals and reactions.

    The woggles are a result of collaboration between the Oz project and the Graphics and Animation research group at Carnegie Mellon University. Together, we are 13 faculty, graduate students, and staff, studying artificial intelligence, graphics, robotics, speech, drama, writing, and animation. One of our goals is to provide technology to let artists build dramatic interactive worlds.

    A key feature of interactive story worlds, as in traditional stories, is the presence of rich characters. In interactive media these characters must be autonomous agents, not simply collections of narrative and dialogue. However, unlike traditional Al agents, which are intended to exhibit intelligence, these agents primarily must exhibit desires (goals), emotional reactions, and social relationships. Our approach is to build creatures that combine ideas from cognitive science with traditional hand animation techniques. We are concerned with controlling action, reaction, eyes, inter-character distance, and anticipation, from the creature’s own internal mental state. This is an example of our notion of Al-based arts.

    As this kind of work progresses, and the artistic and technical quality of simulated creatures improves, people interacting with these creatures may begin to feel the bite of philosophical questions that arise in Al research. If Al achieves its grand objectives, the issue of how one judges the presence of thought and feeling in unfamiliar (mechanical) creatures will become an important practical concern.

    This work is intended to help raise these concerns before they arrive in our society in full force. We have attempted to apply known technology under artistic direction to build suitably confusing creatures, and thus perhaps raise these issues in personal, emotionally powerful ways. A sign of success for us is when our visitors sometime ask,”What happens when you turn them off?”

    Production Systems Technology of Pittsburgh very graciously provided use and support of their RAL rulebased programming system. Their assistance is gratefully acknowledged.