“Virtual Kabuki System” by Ebihara
Conference:
Type(s):
Entry Number: 29
Title:
- Virtual Kabuki System
Program Title:
- Digital Bayou
Presenter(s):
Collaborator(s):
Project Affiliation:
- ATR Media Integration and Communications Research Lab
Description:
Facial expressions play important roles in natural human communication. To realize human-like agents, sometimes called avatars, in virtual environments, it is essential to detect facial expressions from an image in real time and reproduce them in the 3D face model. This project demonstrates a real-time facial expression detection and reproduction system and reproduces those expressions in 3D models of Kabuki actors.
The system consists of three main modules: face modeling, facial expression detection, and reproduction. In face modeling, a 3D model is created with a wireframe that approximates the face shape through a set of small triangular patches. The wireframe model maps the color and texture of the face. In this new method of facial reproduction, 3D measurement data for different facial expressions generated by the main facial muscles are exploited to convert shape changes into deformation data of the 3D face model.
In Virtual Kabuki System, not only facial expressions but also body actions are reproduced in a Kabuki actor’s very artistic and exotic 3D human model. Participants are asked to wear a helmet for facial expression detection. To detect body actions, stereo thermograph cameras track significant points of participants’ bodies. According to the detected facial expressions and body actions, the simulated actor performs Kabuki on a large high-definition screen. Participants perceive themselves as Kabuki actors performing traditional Japanese drama.
