“The Interactive Window” by Paradiso, Leo, Yu and Downie
Conference:
- SIGGRAPH 2002
-
More from SIGGRAPH 2002:
Entry Number: 79
Title:
- The Interactive Window
Presenter(s):
Description:
We have developed a very simple retrofit to a large display surface that enables knocks or taps to be located and characterized (e.g., determining type of hit – metallic tap, knuckle tap, or bash – and intensity) in real time. We do this by analyzing the waveforms captured by 4 piezoelectric transducers (one mounted in each corner of the surface) and a dynamic microphone (mounted anywhere on the glass) in a digital signal processor. Differential timing yields the position, frequency content infers the kind of hit, and peak amplitude reflects the intensity. This technique was first explored in collaboration between Paradiso and Ishii [Ishii et. al. 1999] to make an interactive ping-pong table. Moving to glass display surfaces introduced significant problems, however – knuckle taps are low-frequency impulses that vary considerably hit-to-hit, and the bending waves propagating through the glass are highly dispersive. A heuristically-guided cross-correlation algorithm [Paradiso et al. 2002] was developed to counteract these effects and provide spatial measurements that can resolve knuckle impacts to within σ = 2-4 cm (depending on the material thickness) across a 2-meter sheet of glass. As the requisite hardware is minimal, and everything is mounted on the inside sheet of glass, this is a very simple retrofit to, for example, store window displays, ushering in an entirely new concept of interactive window browsing, where passers-by can interact with information on the store’s products by simply knocking. We have explored this concept in retail, where one of our trackers was installed on the main display window of an American Greetings store near Rockefeller Center in Manhattan for this year’s Christmas-Valentine’s Day season (right figure), and in museums (e.g., left figure, which shows the system running at the Ars Electronica Center in Linz, Austria).
We plan to augment the contact interaction of the knocking system with a noncontact system that detects the presence and activity of participants in front of the screen. Although we have used motion radar systems before for measuring user activity (e.g., [Paradiso et al. 1997]), we are now developing a small, lowpower ranging radar system that can determine the distance to users as they approach the screen. Unlike IR, vision or sonar systems, this technique isn’t sensitive to lighting, clothing, or clutter. It can also sense through opaque, nonconductive material, such as plastic, wood, or wallboard. We plan to place 2-3 radar antennae behind the frosted glass display and outside of the light cone of the projector. This will allow the system to zone the users as they approach, adjusting the audiovisual content accordingly.
The participants’ activity will result in accompanying music and graphics. Free gesture will produce gentle sounds and amorphous cloud-like graphics that become more frenetic with decreasing range. Contact interaction will materialize discrete audiovisual events. In particular, a non-representational, non-photorealistic rendering style will be used that constructs solid waves of color out of lines and points, in which shapes and details repeatedly condense and evaporate. This visualization, which builds its images out of overlapping perturbations of textures, is particularly appropriate for an installation where the primary interaction is the act of striking the display surface.
Other Information:
References
ISHII, H., ET AL., 1999. PingPongPlus: Design of an Athletic-
Tangible Interface for Computer-Supported Cooperative Play.
In Proceedings of the Conference on Human Factors in
Computing Systems (CHI ’99). ACM Press, 1999. 394-401.
PARADISO, J., ET AL., 2002. Passive Acoustic Sensing for Tracking
Knocks Atop Large Interactive Displays. To appear in the
Proceedings of the 2002 IEEE Sensors Conference, June 2002.
PARADISO, J., ET AL., 1997. The Magic Carpet: Physical Sensing
for Immersive Environments. In the Proceedings of the
Conference on Human Factors in Computing Systems (CHI
’97), Extended Abstracts. ACM Press, 1997. 277-278.
