“Seeing, Hearing, and Touching: Putting It All Together” by Fisher, Fels, Maclean, Munzner and Rensink
Conference:
Type:
Entry Number: 07
Title:
- Seeing, Hearing, and Touching: Putting It All Together
Course Organizer(s):
Presenter(s)/Author(s):
Abstract:
Prerequisites
Familiarity with the basics of computer graphics and interactive media, a technical background is not required.
Intended Audience
Interaction designers, artists, and academic and industry researchers who have an interest in multimodal interfaces.
Description
How to design interactive media and applications for emerging computer graphics display technologies. Innovations in large-screen displays enable us to present dynamic, high-resolution graphical scenes, but require designers to predict how those scenes will be parsed by users’ visual systems. Information and data visualization approaches are increasing in importance, but their effectiveness depends on their ability to support visual cognition. Haptic (touch) techniques offer tangibility, but they must be designed for spatial and temporal touch sensitivity as an independent information channel and as support for user interaction (control intimacy). Bottlenecks in sound perception provide their own characteristic design constraints, and producers must determine whether auditory events are perceived as independent channels (for example, system status, speech, music, and background) or an integrated part of a multichannel event (for example, a collision).
The course is divided into five modules: Seeing, Hearing, Touching, Sensory Integration, and Applications/Design. Each module covers relevant aspects of perceptual theory and its application to design and testing of interaction in step-by-step design case studies. Topics include the cognitive science of intersensory processing (vision, hearing, haptics) in scene understanding and interaction, including attention, change blindness, haptics, ventriloquism, and space constancy; enhanced iterative design (Schön’s Reflective Practitioner) for integration of visual display design; haptic devices; and sonified and integrated visual/auditory environments including virtual environments and community/performance spaces.
