“HyperDrum: Interactive Synchronous Drumming in Virtual Reality using Everyday Objects” by Hajika, Gupta, Sasikumar and Pai
Conference:
Experience Type(s):
Title:
- HyperDrum: Interactive Synchronous Drumming in Virtual Reality using Everyday Objects
Presenter(s):
Description:
Hyperscanning is a method to detect if brain wave synchronicity exists between two or more individuals, which is usually due to behavioral or social interactions. It is usually limited to neuroscience studies and very rarely used as an interaction or visual feedback mechanic. In this work, we propose HyperDrum, which is about leveraging this cognitive synchronization to create a collaborative music production experience with immersive visualization in virtual reality. We let the participants wear electroencephalography (EEG) head-mounted displays to create music together using a physical drum. As the melody becomes in synced, we perform hyperscanning to evaluate the degree of synchronicity. The produced music and visualization reflects the synchronicity level while at the same time trains the participants to create music together, enriching the experience and performance. HyperDrum’s main goal is twofold; to blend cognitive neuroscience with creativity in VR, and to encourage connectivity between humans using both art and science.
References:
[1] Thomas D Duane and Thomas Behrendt. 1965. Extrasensory electroencephalographic induction between identical twins. Science 150, 3694 (1965), 367.
[2] Makoto Kumon, ITO Yoshihiro, Toru Nakashima, Tomoko Shimoda, and Mitsuaki Ishitobi. 2007. Sound source classification using support vector machine. IFAC Proceedings Volumes 40, 13 (2007), 465–470.
[3] P Read Montague, Gregory S Berns, Jonathan D Cohen, Samuel M McClure, Giuseppe Pagnoni, Mukesh Dhamala, Michael C Wiest, Igor Karpov, Richard D King, Nathan Apple, 2002. Hyperscanning: simultaneous fMRI during linked social interactions.
[4] Patricia Vanzella, Joana Bisol Balardin, Rogério Akira Furucho, Guilherme Augusto Zimeo Morais, Thenille Braun Janzen, Daniela Sammler, and Joao Ricardo Sato. 2019. fNIRS Responses in Professional Violinists While Playing Duets: Evidence for Distinct Leader and Follower Roles at the Brain Level. Frontiers in psychology 10 (2019), 164.
[5] Kyongsik Yun, Katsumi Watanabe, and Shinsuke Shimojo. 2012. Interpersonal body and neural synchronization as a marker of implicit social interaction. Scientific reports 2(2012), 959.
