“Augmented Groove: Collaborative Jamming in Augmented Reality” by Poupyrev

Augmented Groove is a musical interface that explores use of augmented reality, 3D interfaces, and physical, tangible interaction for conducting multimedia musical performance. Users can play music together, with or without traditional music instruments, simply by picking and manipulating physical cards on a table. The physical movements of the cards are mapped to changes in musical elements such as timbre, pitch, rhythm, distortion, and reverb. At the same time, users wearing lightweight head-mounted displays can see 3D virtual images attached to the cards. The shapes, color, and dynamics of the virtual images reflect aspects of the music.

In this augmented reality environment, users can see the physical world, virtual objects, and each other, so several users can gather around the mixing table and “jam” together, passing sequences to each other in the same manner in which we pass everyday objects from person to person. In a sense, the music becomes a physical, tangible object, something that can be touched and seen as part of the physical environment.

Musical control in Augmented Groove is deliberately imprecise: performers manipulate short musical sequences, or phrases, rather than individual words. The resulting musical performance is a “computer-supported improvisation” in which the user arranges, mixes, and modifies pre-composed musical phrases on a structural level.