“ARCADE: A System for Augmenting Gesture-Based Computer Graphic Presentations” by Stein, Xiao, Tompson, Hendee, Perlin, et al. …

  • ©Murphy Stein, Xiao Xiao, Jonathan Tompson, Charles Hendee, Ken Perlin, and Hiroshi Ishii



    ARCADE: A System for Augmenting Gesture-Based Computer Graphic Presentations



    ARCADE is a system that allows real-time video-based presentations that convey the illusion that presenters are directly manipulating holographic 3D objects with their hands. This illusion is maintained by custom 3D finger tracking from Microsoft Kinect depth-image data, pre-authoring of smart interaction objects that can respond to a presenter’s gestures with high-level behavior, and a suite of rendering effects that convey the visual illusion of a floating-in-air holographic projection. The system is currently used as a research platform for improving finger-based gesture recognition and as a production tool for creating compelling videos for distance education. This Real-Time Live! demonstration features ARCADE’s application for a diverse range of topics, including computer graphics, mathematics, physics, chemistry, biology, and cartography. This project is inspired by the post-production effects typically used in Hollywood films, such as “Minority Report”, “Harry Potter”, and “Star Wars”, but it enables non-professionals to author these experiences and then perform them live.


ACM Digital Library Publication:

Overview Page:


E-Tech Type: