Visualizing Sound: Effects of Pitch Height and Tonality on Luminance Matching

By Naomi F. Maxwell
2013, Vol. 5 No. 10 | pg. 1/2 |


Synesthesia, a perceptual condition in which two or more senses intermix, has elicited research on cross-modal perception in general. Previous studies have found similar patterns of matching color to sound among synesthetes and non-synesthetes. This study tested a sample of non-synesthetes for consistent patterns of luminance matching in response to triads varying in pitch height (high versus low) and tonality (major versus minor). A repeated-measures ANOVA revealed significant main effects for both variables, which suggests that non-synesthetic populations are able to make consistent associations between audio and visual attributes in a fashion similar to that of sound-to-color synesthetes. These findings support the possibility that some of the neural mechanisms underlying sound-to-color synesthesia may be shared with normal populations.

Synesthesia is a perceptual condition which challenges the tradition notion of five completely separate senses. Synesthetes experience a blending of the senses; they have additional sensory perceptions in one modality in response to input from another. Although rare, synesthesia can occur between any two or more of the five senses, and research has verified the unusual reports of synesthetes by testing their perceptions and finding them to be unlearned, involuntary, and very consistent over time (Ward, Huckstep, & Tsakanikos 2006).

One form of synesthesia involves the involuntary perception of specific colors in response to sound, especially music. This type of synesthesia, called “colored hearing” or sound-to-color synesthesia, has sparked the interest of researchers who examine emotional correlates of music and color, as well as those interested in cross-modal, particularly audio-visual, associations made by people who lack synesthesia. Comparisons between some attributes of synesthetes’ involuntary color perceptions with audio-visual associations of non-synesthetes has fueled one theory of synesthesia, which claims that this highly unusual condition has roots in normal cross-modal mechanisms common to all human beings (Ward, Huckstep, & Tsakanikos 2006). This study examines potential cross-modal associations made between one visual attribute, luminance, with two different auditory attributes, pitch height and tonality.

Luminance and Cross-Modality

Luminance (the relative lightness or darkness of an object) is relevant to comparisons between synesthesia and normal cross-modal perception because of its associations with overarching emotional and metaphorical concepts. In a 2007 study, Meier, Robinson, Crawford, and Ahlvers found that subjects perceived squares as brighter when paired with positive words, and darker when paired with negative words. This finding suggests that the idea of congruency, or matching the attributes of distinct concepts with each other, can influence individual aspects of perception. Along similar lines, Marks (1982) conducted a study in which participants adjusted the brightness of a light and the loudness of a tone to match given lines of poetry. The subjects adjusted the respective levels of each mode (low, medium, or high brightness or loudness) in almost perfect correlation, proportionate on a line graph. Since subjects in both of these studies seemed to match luminance, a strictly visual attribute by definition, with external, non-visual attributes, we can reasonably conclude that luminance may represent a broader concept, such as intensity, which lends itself to cross-modal mapping.

Hubbard (2010) compared previous empirical findings related to auditory imagery, and suggested that since auditory imagery shares many semantic characteristics with acoustic stimuli, humans’ interpretation of both may be based on shared qualities, such as intensity. Based on this theory, and the findings of Meier et al. and Marks, luminance seems to aptly represent visual intensity, as demonstrated by subjects’ ability to effectively associate it with non-visual concepts.

Pitch Height and Luminance

Research has examined correspondences between pitch height and luminance using several methodologies. In a series of three experiments conducted by Ward, Huckstep, and Tsakanikos in 2006, a group of sound-to-color synesthetes and a non-synesthete control group matched colors with pitches, using a custom color picker on a computer. Although the synesthetes demonstrated much more consistency in their color choices, both groups chose lighter colors for higher pitches, and darker colors for lower pitches. A study by Miller, Boteler, and Larcombe (2003) observed a similar response pattern in a case study of one synesthete, finding that lower-pitched tones elicted larger and darker visualizations than high-pitched tones.

In a 2012 study, Walker found that pairing a light or dark grey background with either low-pitched or high-pitched sounds affected how quickly non-synesthetes correctly completed a word classification task. Congruent combinations (light background, high pitch; dark background, low pitch) enabled faster word classification among the participants. Similar results have been demonstrated in several studies investigating associations of luminance with pitch height among normal populations (Collier 2001; Hubbard 1996; Marks 1989; Melara 1989).

These findings support the idea of a relationship between pitch height and luminance as a function of general cross-modality, which has led researchers to ask whether or not these correspondences are learned. To examine this question, Walker et. al. conducted a study with infants, pairing light or dark animations with various pitches (2009). The infants looked longer at congruent combinations than incongruent combinations, which supports the hypothesis that this type of cross-modality is present very early on, and thus, potentially unlearned. Haryu and Kajikawa conducted a similar study with infants in 2012, pairing greyscale animations of apples (light grey or dark grey) with a standard tone that moved either to a lower pitch or a higher pitch. Consistent with the findings of Walker et al., the infants looked longer at congruent combinations (tone moving up with light grey apple, tone moving down with dark grey apple).

A study by Eitan, Ornoy, and Granot found that congenitally and early blind subjects associated low segments of a Beethoven sonata with “dark,” and high segments with “light,” using a five-point bipolar scale (2012). These findings suggest that blind people make cross-modal correspondences similar to those of sighted people. In contrast to the two studies involving infants, however, this may suggest that such an association is learned.

In a series of experiments varying pitch height, tempo, and pitch contour, Collier found that subjects rated high-pitched tones as not only brighter, but happier and more quickly ascending than low-pitched tones (2001). Collier also found that ascending tones were perceived in similar ways in comparison to descending tones, and discovered interactions between the three musical attributes included in his study. These associations suggest that pitch height is related not only to luminance, but to emotion, and this relationship may be shared with other visual and musical attributes.

Tonality and Luminance

Tonality (for this study’s purposes, major versus minor) is typically characterized in emotional terms borrowed from linguistics. Descriptors for minor key tonality typically include words such as “sad” and “dark,” versus major key tonality descriptors, which often include adjectives like “happy” and “light.” Suzuki et. al. (2008) used positron emission tomography in a study examining the brain areas activated by major and minor chords, and found that distinct regions were associated with the two types. Their findings suggest that the brain processes tonality in accordance with affective valence, e.g., minor chords are processed in an area of the brain which also handles emotions, but is different from the region which processes major chords.

Main effects of tonality on pleasure and surprise were observed in a 1994 study by Kellaris and Kent, in which subjects assigned higher pleasure ratings to major tonality than minor tonality. Participants also rated major music excerpts as happier than minor music excerpts in an experiment by Hunter, Schellenberg, and Schimmack (2010).

Although tonality is likely correlated with emotion, little research has been conducted to examine whether or not descriptors involving luminance, which are commonly assigned to tonality, can extend beyond linguistics and into cross-modal associations where luminance is a real visual concept, not just a metaphor. Previous studies on color and emotion have produced findings very similar to the positive/negative valence observed with regards to tonality (Boyatzis & Varghese 1994; Hemphill 1996), which supports the likelihood of a relationship between color and tonality. Furthermore, a 2004 study by Xin et. al. indicates that luminance is more relevant than some other visual attributes, such as hue, with regards to associations made between color and emotion. If a relationship between luminance and tonality exists, this may support a cognitive explanation for the linguistic metaphors so often used to characterize tonality. These metaphors include the concept of dark and light – a semantic property directly borrowed from luminance.

Suggested Reading from Inquiries Journal

Adorno’s case is built on the composers’ treatment of melodic subjects, which he views as ‘pure’ in an intellectual and philosophical sense. He contrasts this to the work of the classical and romantic composers, specifically Beethoven, whose work he views as overcommitted to the formal façade unintentionally... MORE»
Filmmaker Wes Anderson has developed a distinct aesthetic style that is easily recognizable through his use of striking color palettes. The purpose of this research was to draw conclusions about social stratification, social construction and the role of family as they relate to color palettes in Anderson's films. Relying on the... MORE»
This essay will discuss visual perception as well as examine the relationship between art and hallucination. The paper is divided into three sections. The first section will explore phenomenological aspects of the Ayahuasca hallucinations described in Benny Shanon‟s book Antipodes of the Mind. In the second section, it will... MORE»
John Major was born in Merton, Surrey on March 29, 1943. He was the son of a trapeze artist/self-employed businessman making garden gnomes; he was educated at Cheam Primary and Rutlish Grammar School, leaving at 15. In fact, Major was one of the most upwardly mobile politicians in British history, rising from the dole to Downing Street. He worked as an insurance clerk, trainee accountant, general labourer, overseas banker, and eventually as... MORE»
Submit to Inquiries Journal, Get a Decision in 10-Days

Inquiries Journal provides undergraduate and graduate students around the world a platform for the wide dissemination of academic work over a range of core disciplines.

Representing the work of students from hundreds of institutions around the globe, Inquiries Journal's large database of academic articles is completely free. Learn more | Blog | Submit

Follow IJ

Latest in Psychology

2022, Vol. 14 No. 06
Change blindness is the finding that people often fail to notice substantial changes between different views of a visual scene. The current study investigated the effect of mood states on people’s ability to detect changes, by comparing participants... Read Article »
2022, Vol. 14 No. 05
The prevalent school of thought states that suicidal ideation and suicide planning are not associated with living in households with firearms. Using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health) in the years... Read Article »
2022, Vol. 14 No. 04
Marion Godman makes the argument that Pathological Withdrawal Syndrome (PWS) makes the case for psychiatric disorders as a natural kind. Godman argues that we can classify kinds according to their shared ‘grounding’, but we need not... Read Article »
2022, Vol. 14 No. 03
The study investigated stress, coping strategies, and problem-solving skills among college students. A total of 202 university students completed this study. The purpose of this study was to address gaps in the existing literature regarding stress... Read Article »
2021, Vol. 13 No. 12
Political polarization has been an increasingly salient point of discussion since the 2016 presidential campaign, the election of Donald Trump, and into today. Beyond emphasizing partisan and issue-based divides, scholars have identified emotion... Read Article »
2021, Vol. 13 No. 04
The question of what it means to be a gendered individual has been left unanswered in light of its variants. The feminist movement proceeding the Industrial Revolution propelled philosophical and literary works, such as Simone de Beauvoir’... Read Article »
2021, Vol. 13 No. 03
Positive affect (PA) is active, enthusiastic, and happy engagement in pleasurable activities and negative affect (NA) includes aversiveness, anger, and fear (Watson et al., 1988). Two studies examined linguistic affect presented as emotion words... Read Article »

What are you looking for?


How to Use Regression Analysis Effectively
7 Big Differences Between College and Graduate School
Presentation Tips 101 (Video)