“onNote: a musical interface using markerless physical scores” by Kakehi, Yamamoto and Uchiyama

Intuitive music playing using various digital musical instruments with specific tangible interfaces has become one of the ways to enjoy the music experience [Jorda et al. 2007]. When we start studying music, we need to learn how to read musical scores to understand the different instrumental parts and also understand the melody, rhythm, fingering and so on. However, reading scores is often a difficult task for beginners and could create a barrier for playing music. To solve this problem, we purpose a novel musical interface system named “onNote”. In this system, physical markerless musical scores are used as instruments to play music intuitively (see Figure 1). The notes on the score are captured by a camera and are processed by the system which retrieves the music from a score database. In addition, the system can do a real-time recognition of the paper’s position and the rotation. Thus, by physically moving and connecting the musical scores we can play music intuitively.

1. Jorda, S., Geiger, G., Alonso, M., and Kaltenbrunner, M. 2007. reactable: Exploring the synergy between live music performance and tabletop interface. In Proceedings of TEI ’07, ACM Press, 139–146.
2. Nakai, T., Kise, K., and Iwamura, M. 2006. Use of affine invariants in locally likely arrangement hashing for camera-based document image retrieval. In Lecture Notes in Computer Science (7th International Workshop DAS2006), 3872, 541–552.