Visualizing Sound: Effects of Pitch Height and Tonality on Luminance Matching

By Naomi F. Maxwell
2013, Vol. 5 No. 10 | pg. 2/2 |

The Present Study

The present study aims to examine a potential relationship between luminance and pitch height by replicating the results of previous studies. It further aims to investigate a potential relationship between luminance and tonality, based on prior research relating emotion with luminance and tonality, respectively. If found statistically significant, these relationships may demonstrate the existence of cross-modal perception among non-synesthete populations, and provide direction for further research with both synesthetes and normal populations.

Hypotheses

Hypothesis 1 – Pitch Height: High-pitched triads will be matched with lighter colors, while low-pitched triads will be matched with darker colors. This hypothesis is based on previous findings related to luminance and pitch height (Collier 2001; Eitan et al. 2012; Haryu & Kajikawa 2012; Hubbard 1996; Marks 1989; Melara 1989; Miller et al. 2003; Walker 2012; Walker et al. 2009; Ward et al. 2006).

Hypothesis 2 – Tonality: Major triads will be matched with lighter colors, while minor triads will be matched with darker colors. This hypothesis is based on previous associations found between emotion and tonality (Suzuki et al. 2008; Kellaris & Kent 1994; Hunter et al. 2010), and emotion and color (Boyatzis & Varghese 1994; Hemphill 1996; Xin 2004).

Hypothesis 3 – Interaction: The effects of pitch height will override tonality for incongruent triads (high-pitched/minor; low-pitched/major). This hypothesis is based on the theory that pitch height is more salient than tonality, and will thus serve as a more effective vehicle for cross-modal mappings with luminance, since prior research with multiple populations has been conducted with regards to pitch height, but not tonality.

Methodology

Participants

Twenty-eight undergraduate students (17 women, 9 men, 2 unreported) at a small liberal arts university in the Pacific Northwest were recruited through the university’s online psycholgy research familiarization system. Students received either research familiarization credit or extra credit points for participating, depending on their course requirements.

Seven subjects responded “Yes” to an item on the study questionnaire which stated, “I see colors when I hear music, even if no colors are physically present.” Although subjects were not tested to verify this response, they were eliminated from the final analysis to ensure that the sample was composed entirely of subjects without sound-to-color synesthesia. An additional two subjects were eliminated from the final analysis because they reported suspicion of the study’s hypotheses on the questionnaire.

Five subjects reported that they had received no musical training in the past, seven subjects reported one to five years of musical training, five subjects reported five or more years of musical training (but not pursuing a music major or minor), and the remaining two subjects did not report their musical training on the study questionnaire.

Materials

Forty triads were recorded by the researcher on an electronic piano, varying in pitch (high or low) and tonality (major or minor), with equal numbers of triads in each category (n = 10: high major; high minor; low major; low minor). Each sound clip lasted approximately five seconds. Each major triad had a minor triad counterpart that shared the same base pitch; thus, 20 unique base pitches were used. Base pitches for low triads ranged from Bb1 (58.270 Hz) to E3 (164.81 Hz), and base pitches for high triads ranged from A4 (440.00 Hz) to B5 (987.77 Hz).

The researcher created an eight-by-eight block color grid as the dependent measure. The grid varied in hue (left to right) and luminance (top to bottom). Although irrelevent to the study, hue was included as a variable of the grid in order to give subjects a wider range of potential responses. Letters (A-H) and numerals (1-8) were assigned to the grid so that subjects could specify an individual block in each of their responses. To control for a possible response bias, the grid was designed with darker hues at the top, and lighter hues at the bottom (see Figure 1). Thus, if subjects associated their responses in spatial terms (e.g., higher-pitched triads may increase the likelihood of choosing a block higher on color grid), they would do so in a manner opposite of Hypothesis 1.

Figure 1

Procedures

Data collection took place over a period of six separate sessions, with four to five participants in each session. After reading and signing a research participation consent form, subjects listened to the 40 triads via headphones while seated in front of a monitor which displayed the color grid. Computer monitor brightness and headphone volumes were homogenized before the start of each session to ensure uniformity. The triads were played in random order, using the same order for each subject. On-screen instructions directed participants to pick the color they considered the “best fit” for a given triad, and to record the corresponding letter and numeral for each on a sheet of paper provided by the researcher. Subjects could replay any triad multiple times, but were not allowed to return to a previous triad once they had recorded a response and moved on to the next triad.

After listening and responding to each triad, on-screen instructions directed the subjects to respond to the questionnaire items on the reverse side of their sheet. These items related to gender demographics, sound-to-color synesthesia reporting, musical training background, understanding of minor versus major chord tonality, and a suspicion check to ensure that participants had not guessed the study’s hypotheses during the experiment. After completing the questionnaire, subjects were orally debriefed on the study’s purpose and hypotheses, and granted credit for participating.

Results

A two by two repeated-measures ANOVA (Pitch Height[High][Low] by Tonality[Major][Minor]) was conducted to examine the main effect of pitch height, the main effect of tonality, and the interaction between pitch height and tonality on luminance matching. The triads were coded into four categories: high major, high minor, low major, and low minor. Mean luminance scores for the four categories were calculated for each subject, and overall luminance means for each category were used in the ANOVA. Luminance scores could range from 1 (darkest) to 8 (lightest).

Pitch Height: the analysis revealed statistically significant main effects for pitch height, F(1, 18) = 51.67, partial eta-squared = .74, p < .001. Mean luminance scores were higher for high-pitched triads (M = 5.26, SD = .24) than low-pitched triads (M = 2.92, SD = .16).

Tonality: the analysis also revealed statistically significant main effects for tonality, F(1,18) = 6.99, partial eta-squared = .28, p < .05. Mean luminance scores were higher for major triads (M = 4.23, SD = .14) than minor triads (M = 3.95, SD = .13).

Interaction: the interaction between pitch height and tonality was statistically non-significant, F(1, 18) = 1.52.

Discussion

Hypotheses and Implications

The findings supported Hypothesis 1, which predicted that high-pitched triads would be matched with lighter colors, and low-pitched triads would be matched with darker colors. These findings are consistent with previous research in which subjects matched luminance with pitch height (Collier 2001; Eitan et al. 2012; Haryu & Kajikawa 2012; Hubbard 1996; Marks 1989; Melara 1989; Miller et al. 2003; Walker 2012; Walker et al. 2009; Ward et al. 2006).

The findings also supported Hypothesis 2, which predicted that major triads would be matched with lighter colors, and minor triads would be matched with darker colors. These findings support the theory that the affective valence of tonality and luminance are comparable, based on respective prior findings on these two topics (tonality: Suzuki et al. 2008; Kellaris & Kent 1994; Hunter et al. 2010, color: Boyatzis & Varghese 1994; Hemphill 1996; Xin 2004).

Hypothesis 3 was unsupported, since no statistically significant interaction was found between pitch height and tonality. Tonality seems to be autonomously related to luminance matching, and does not depend on pitch height.

Study Limitations and Future Research

These results are not necessarily generalizable, since the study sample was limited to the research familiarization pool at a small liberal arts university. The majority of participants were female, and other demographics may be distinct to this sample. Further research should encompass a larger and more diverse sample, to increase statistical and determine whether or not the current findings are generalizable. Also, the sample included no sound-to-color synesthetes. Although prior research on luminance matching with pitch height has been conducted with both synesthetes and non-synesthetes (Ward et al. 2006), no studies exist which compare luminance matching with tonality between the two groups. If tonality functions as an auditory attribute which can be cross-modally mapped by non-synesthetes, future research should investigate if a similar pattern is found among sound-to-color synesthetes.

Potential confounding variavles within the current study design also limit generalizability. Although the triads were played in random order, the same order was used for each participant. Future research should include several different orders, and examine potential differences caused by contrast effects or other such variables.

Other aspects of the study design may have biased participants’ responses, such as the numbers assigned to the color grid for identification purposes. Subjects may have potentially chosen higher numbers in response to higher pitches, which would have contributed to the significant results observed. A future study could control for this potential confound by using a non-hierarchal identification system instead of numbers, or randomly assigning participants to grids with different numbering systems.

Conclusion

In the larger scheme of gaining an understanding of cross-modal perception, future studies should continue to investigate relationships between other visual and auditory attributes, such as chroma, saturation, shape, size, texture (both visual and auditory), loudness, timbre, pitch contour, and rhythm, as well as potential interactions between these variables. Correspondences between luminance and tonality in particular should be explored, using larger populations of both synesthetes and non-synesthetes. Research on auditory-visual correspondences among both groups has the potential to shed light on the underlying neural basis of synesthesia, and the possibility that its idosyncratic manifestations are based on universal and consistent perceptual processes.


References

Boyatzis, C.J., & Varghese, R. (1994). Children’s emotional associations with colors. The Journal of Genetic Psychology, 155, 77-85. doi: 10.1080/00221325.1994.9914760

Collier, W.G., & Hubbard, T.L. (2001). Judgments of happiness, brightness, speed and tempo change of auditory stimuli varying in pitch and tempo. Psychomusicology, 17, 36-55.

Eitan, Z., Ornoy, E., & Granot, R.Y. (2012). Listening in the dark: Congential and early blindness and cross-domain mappings in music. Psychomusicology: Music, Mind, & Brain, 22, 33-45.

Haryu, E., & Kajikawa, S. (2012). Are higher-frequency sounds brighter in color and smaller in size? Auditory–visual correspondences in 10-month-old infants. Infant Behavior & Development, 35, 727-732.

Hemphill, M. (1996). A note on adults’ color-emotion associations. The Journal of Genetic Psychology, 157, 275-280.

Hubbard, T. (2010). Auditory imagery: Empirical findings. Psychological Bulletin, 136, 302- 329.

Hubbard, T. (1996). Synesthesia-like mappings of lightness, pitch, and melodic interval. The American Journal of Psychology, 109, 219-238.

Hunter, P.G., Schellenberg, E.G., & Schimmack, U. (2010). Feelings and perceptions of happiness and sadness induced by music: Similarities, differences, and mixed emotions. Psychology of Aesthetics, Creativity, and the Arts, 4, 47-56.

Kellaris, J.J., & Kent, R.J. (1994). An exploratory investigation of responses elicited by music varying in tempo, tonality, and texture. Journal of Consumer Psychology, 2, 381-401.

Marks, L. (1982). Synesthetic perception and poetic metaphor. Journal of Experimental Psychology, 8, 15-23.

Marks, L. (1989). On cross-modal similarity: the perceptual structure of pitch, loudness, and brightness. Journal of Experimental Psychology, 15, 586-602.

Meier, B.P., Robinson, M.D., Crawford, L.E., & Ahlvers, W.J. (2007). When “light” and “dark” thoughts become light and dark responses: Affect biases brightness judgments. Emotion, 7, 366-376. doi: 10.1037/1528-3542.7.2.366

Melara, R. (1989). Dimensional interaction between color and pitch. Journal of Experimental Psychology: Human Perception and Performance, 15, 69-79.

Miller, C.B., Boteler, E.H., & Larcombe, G.K. (2003). “Seeing things in my head”: A synesthete’s images for music and notes. Perception, 32, 1359-1376.

Suzuki, M., Okamura, N., Kawachi, Y., Tashiro, M., Arao, H., Hoshishiba, T., Gyoba, J., & Yanai, K. (2008). Discrete cortical regions associated with the musical beauty of major and minor chords. Cognitive, Affective, & Behavioral Neuroscience, 8, 126-131. doi: 10.3758/CABN.8.2.126

Walker, P. (2012). Cross-sensory correspondences and cross talk between dimensions of connotative meaning: Visual angularity is hard, high-pitched, and bright. Attention, Perception, & Psychophysics, 74, 1792-1809. doi: 10.3758/s13414-012-0341-9

Walker, P., Bremner, J.G., Mason, U., Spring, J., Mattock, K., Slater, A., & Johnson, S.P. (2009). Preverbal infants’ sensitivity to synaesthetic cross-modality correspondences. Psychological Science, 21, 21-25.

Ward, J., Huckstep, B., & Tsakanikos, E. (2006). Sound-colour synaesthesia: To what extent does it use cross-modal mechanisms common to us all? Cortex, 42, 264-280.

Xin, J.H., Cheng, K.M., Taylor, G., Sato, T., & Hansuebsai, A. (2004). Cross-regional comparison of colour emotions part I: Quantitative analysis. Color Research and Application, 29, 451-457.

Suggested Reading from Inquiries Journal

Adorno’s case is built on the composers’ treatment of melodic subjects, which he views as ‘pure’ in an intellectual and philosophical sense. He contrasts this to the work of the classical and romantic composers, specifically Beethoven, whose work he views as overcommitted to the formal façade unintentionally constructed by Bach: essentially he claims that Beethoven et al rejected Bach’s challenging, philosophically... MORE»
Advertisement
Filmmaker Wes Anderson has developed a distinct aesthetic style that is easily recognizable through his use of striking color palettes. The purpose of this research was to draw conclusions about social stratification, social construction and the role of family as they relate to color palettes in Anderson's films. Relying on the... MORE»
This essay will discuss visual perception as well as examine the relationship between art and hallucination. The paper is divided into three sections. The first section will explore phenomenological aspects of the Ayahuasca hallucinations described in Benny Shanon‟s book Antipodes of the Mind. In the second section, it will... MORE»
John Major was born in Merton, Surrey on March 29, 1943. He was the son of a trapeze artist/self-employed businessman making garden gnomes; he was educated at Cheam Primary and Rutlish Grammar School, leaving at 15. In fact, Major was one of the most upwardly mobile politicians in British history, rising from the dole to Downing Street. He worked as an insurance clerk, trainee accountant, general labourer, overseas banker, and eventually as... MORE»
Submit to Inquiries Journal, Get a Decision in 10-Days

Inquiries Journal provides undergraduate and graduate students around the world a platform for the wide dissemination of academic work over a range of core disciplines.

Representing the work of students from hundreds of institutions around the globe, Inquiries Journal's large database of academic articles is completely free. Learn more | Blog | Submit

Follow SP

Latest in Psychology

2018, Vol. 10 No. 12
Achievement goals refer to the motivational approach of an individual when facing an achievement situation that challenges the person’s sense of competence, such as a university course (Baranik, Stanley, Bynum, & Lance, 2010; Harackiewicz... Read Article »
2018, Vol. 10 No. 09
This study aimed to determine if anxiety and depression in individuals are related to deontological ethical decisions, with particular emphasis on the role of reward responsiveness as an underlying principle mediating any differences. Despite some... Read Article »
2018, Vol. 10 No. 05
Autism Spectrum Disorders (ASD) are among the most common neurodevelopmental disorders. According to the Centers for Disease Control and Prevention (CDC), studies in Asia, Europe, and North America have identified individuals with ASD with an average... Read Article »
2018, Vol. 10 No. 05
This review discusses Mindfulness-Based Art Therapy research to date. A literature review first explores mindfulness and art therapy independently, then investigates the current research on the combination of these two modalities used with clients... Read Article »
2018, Vol. 10 No. 03
Byrnes (1983) found bright colors were associated with positive emotions (e.g., happiness) and dark colors were associated with negative (e.g., sadness) emotions. Emotion is communicated through facial expressions which are a form of nonverbal communication... Read Article »
2018, Vol. 10 No. 02
The current study focuses on the effect of gender-specific (he, she) and gender-neutral (they) pronouns on male and female recognition ability. Crawford and English (1984) showed male pronouns (he) better aid male recall and gender-neutral pronouns... Read Article »
2017, Vol. 9 No. 12
Cooley (1902) introduced the looking glass self as an individual’s self-concept defined, in part, by societal heuristics. Silvia and Phillips (2013) showed self-awareness (SA) was influenced by presenting stimuli that both explicitly increase... Read Article »

What are you looking for?

FROM OUR BLOG

How to Select a Graduate Research Advisor
"Should I Go to Graduate School?"
Writing a Graduate School Personal Statement