“Synesthetic color scheme in Fantasia” by Cai, Goto, Shinohara, Nagata, Kurumisawa, et al. …

Synesthesia is a neurological condition in which stimulation of one sensory or cognitive pathway includes automatic and involuntary responses in a second sensory or cognitive pathway. People who report such experiences are known as “synesthetes”. A subset of synesthetes, sound-color synesthetes “see” colors and simple animated shapes in association with music and other sounds. We can find a color-sound synesthete out of two to three thousands. It has been suggested [Stein et al. 1993] that “..every infant is synesthetic, although most gradually grow out of it” p 12 (Stein and Meredith 1993). We performed a set of test based on a standard battery of tests [Eagleman et al. 2007] that include: (1) a pitch test, (2) a chord test, and (3) a musical key test. The test subjects select colors from a color map after hearing sounds or short music generated by various pitches, chords, and keys. We judge the level of the synesthesia based on the battery score formula. Forty-three synesthetes took the tests and we selected only those with the high scores. We were surprised at how the test subjects associated colors that were strongly correlated with the keys, especially with major keys. In both the pitch and chords tests, they choose almost the same colors as in the key test. The key has a tonic and a tonic chord. In both the pitch and chord tests, the chosen color for a certain pitch and chord is almost the same as the color chosen for this key in the key test. One of test subjects results are shown in Fig. 1. The chosen tonics, tonic chords, and their keys, and their associated colors are shown in Fig. 1. In the figure, only the upper half of the circle of fifths is shown. Each column shows the colors associated with the tonic in the upper, the tonic chord in the middle, and the key in the bottom. This strongly suggests that the synesthetes perceive the sound as a structure characterized by the “key”. At the same time, the colors they picked are strongly clustered about several colors for each key. We use the k-means method [Hartigan and Wong 1979]to cluster the selected colors, and obtained representative colors. Then, we list the three to five colors with highest frequencies for tonic, tonic chord, and key from right to left in Fig. 2. The representative colors for the pitch, chord, and key are very much correlated and similar as shown in Fig. 2 (left, only for F major). We compared these representative colors with these in “Fantasia”. Fantasia is a 1940 American animated film produced by Walt Disney. It is the third feature-length film produced by the Walt Disney Company. It is well known various synesthetic effects appear in this film although there is no evidence that the film was made by synesthetic artists. In this talk, we discuss the synesthetic colors are associated with the musical keys and how these colors create special effects. For example, in Fig. 2, the synesthetic “fade-in” effect is used in Chapter 6, the pastoral symphony in Fantasia. We also found a synesthetic “cross-fading” is used before the changing of key in this chapter.

1. Eagleman, D., Kagan, A., Nelson, S., Sagaram, D. and Sarma, A. 2007. A standardized test battery for the study of synesthesia. Journal of neuroscience methods 159, 139–145.
2. Hartigan, J. and Wong, M. 1979. A k-means clustering algorithm. JR Stat. Soc., Ser. C 28, 100–108.
3. Stein, B., Meredith, M. and Wolf, S. 1993. The merging of the senses.