The Influence of Gaze Direction on Approach- vs. Avoidance-Oriented Emotions
When investigating the effect of gaze direction on facial expressions of emotion, previous imaging research indicated that dynamic presentation of stimuli produced higher amygdala responses (Sato, Kochiyama, Uono, & Yoshikawa, 2010). A behavioral study further suggested that approach-oriented emotions are intensified by direct gaze, where as avoidance-oriented emotions are intensified by averted gaze (Adams & Kleck, 2005). We hypothesized that direct gaze would elicit higher amygdala activity for the approach-oriented emotion of anger, where as averted gaze would elicit higher amygdala activity for the avoidance-oriented emotion of fear. Contrast estimates performed for the left and right amygdala supported our hypothesis and also displayed a lateralization effect. Approach-oriented emotions with direct gaze elicited higher responses in the left amygdala, while avoidance-oriented emotions with averted gaze elicited higher responses in the right amygdala.
The integration of facial expression of emotion and direction of gaze is an important aspect of human communication (Adams & Kleck, 2005). Facial expressions provide the outer response to changes within inner emotional states, where as gaze direction is indicative of how attention is directed and suggests behavioral intention. Previous findings implicated the amygdala as the particular region involved in the integration of emotional expression and gaze (Sato, Kochiyama, Uono, & Yoshikawa, 2010).
Sato, Kochiyama, Uono, and Yoshikawa (2010) noted inconsistent results among neural exploration of the amygdala and its role in the processing of facial expressions of emotion as a function of gaze direction. Their study attributed these differences to the type of stimulus presentation used to display the emotion. In their research, Sato et al. (2010) categorized stimulus presentation as either dynamic or static. Dynamic presentation consisted of video clips that showed a neutral expression evolving into an emotional expression of either anger or happiness, while static presentation used successive still image frames to display the change from neutral to emotional. The authors hypothesized that the integration of angry and happy emotional expressions with either direct or averted gaze would elicit higher amygdala activity in the dynamic condition than in the static condition. The results of this study supported their hypothesis and indicated the strength of dynamic presentation. Furthermore, their findings showed that both angry and happy dynamic facial expressions elicited greater amygdala activity in response to direct gaze orientation compared to averted gaze orientation.
Behavioral research by Adams and Kleck (2005) suggested that the integration of emotional expression and gaze direction implies the expressor’s behavioral intent to either approach or avoid. They suggested that the emotions of anger and happiness are categorized as approach-oriented emotions, where as the emotions of fear and sadness are defined as avoidance-oriented emotions (Adams & Kleck, 2005). Furthermore, Adams and Kleck (2005) implicated direct gaze would enhance approach-oriented emotions (anger, happiness) and averted gaze would enhance avoidance-oriented emotions (fear, sadness). Their behavioral findings indicated this effect: direct gaze increased the perceived intensity of approach-oriented expression of anger and joy, where as averted gaze increased the perceived intensity of avoidant-oriented expressions of fear and sadness.
The findings of Sato and colleagues (2010) are not clear in light of Adams and Kleck’s (2005) behavioral study. Sato et al. (2010) found greater amygdala activity for emotions of anger and happiness when combined with direct gaze compared to averted gaze. Anger and happiness, according to Adams and Kleck (2005), are approach-oriented emotions. Therefore, Sato and colleagues’ (2010) study does not investigate the integration of gaze with avoidance-oriented emotions and how this combination influences amygdala activity. Furthermore, Harmon-Jones and Sigelman (2001) explored the effects of both approach-oriented and avoidance-oriented affect and suggested a lateralization of activity. Their EEG study implicated greater left-hemisphere responses to be associated with processing anger, an approach-oriented emotion, and right hemispheric activity to be more connected to fear, an avoidance-oriented emotion.
Based upon the findings of Sato et al. (2010) and Adams and Kleck (2005), we investigated the neural bases of both approach- and avoidance-oriented emotions with regards to gaze direction. We used the dynamic presentation of stimuli adapted from Sato and colleagues (2010) due to the elicitation of higher amygdala activity compared to the static presentation. Our study contained the approach-oriented emotion of anger and the avoidance-oriented emotion of fear because each affect is negative and produced the most salient responses in their respective studies. We hypothesized that the amygdala response to anger stimuli with direct gaze would be higher than that of anger stimuli with averted gaze. Furthermore, we expected the amygdala response to fear stimuli with averted gaze would be higher than that of fear stimuli with direct gaze. Based upon the findings of Harmon-Jones and Sigelman (2001), we also predicted a lateralization of activity within the amygdala, with the left amygdala eliciting higher responses to the approach-oriented emotion of anger and the right amygdala eliciting greater activity for the avoidance-oriented emotion of fear.
Behavioral data analysis
The researchers conducted random effects analyses to determine any significantly activated voxels that displayed interesting effects. Each task-related condition was modeled with a boxcar function, convoluted with a canonical hemodynamic response function. First, we performed planned contrasts for the interaction of anger-versus-fear conditions and direct-versus-averted directions, according to our hypothesis. Further planned contrasts were performed for main effects (main effect of emotion and main effect of gaze direction) as well for simple effects of each condition (anger-direct, anger-averted, fear-direct, and fear-averted). We used an initial threshold of p < 0.001 (uncorrected) to identify significantly activated voxels. For specific contrasts which did not show significant activity with uncorrected threshold, we used a p < 0.05 (corrected) threshold. The researchers carried out these analyses across the whole brain.
To scrutinize the activation patterns of the amygdala, we further conducted region of interest (ROI) analyses. SPM5 (Pickatlas) determined the location of the amygdala using anatomical landmarks. We analyzed the contrast estimate of each condition in both the left and the right amygdala for the interaction. Furthermore, the left and right fusiform gyrus was also located using anatomical landmarks in SPM5 (Pickatlas) and analyzed using contrast estimates for the interaction between emotion and gaze direction.
Main effect of emotion
Main effect of gaze direction
Interaction between emotion and gaze direction
Since there was no statistically significant activation for the interaction between emotion and gaze direction, ROI analyses of the amygdala were performed. The trend of the contrast estimates suggested an interaction between emotion and gaze direction in both left and right amygdala. Within the left amygdala, the anger condition resulted in higher contrast estimates for direct gaze (3.2) compared to averted gaze (1.2). There was little difference between direct gaze (-0.6) and averted gaze (-0.3) within the fear condition (Figure 2). In the right amygdala, the anger condition yielded little difference between direct gaze (2.0) and averted gaze (1.4). However, the fear condition showed less deactivation for averted gaze (-0.6) compared to direct gaze (-3.2) (Figure 2).
An additional planned contrast of the interaction between emotion and gaze direction did not show significant activity within the left and right fusiform gyrus region. However, contrast estimates again suggested an interaction of emotion and gaze direction in the left and right fusifrom gyrus. Within the left fusiform area, the anger condition displayed little difference between direct gaze (5.6) and averted gaze (4.4) direction. On the other hand, the fear condition elicited higher contrast estimates for averted gaze (5.6) compared to direct gaze (1.6) (Figure 3). The right fusiform gyrus showed the opposite trend, in that the anger condition elicited high contrast estimates for direct gaze (3.4) over averted gaze (1.8), while the fear condition showed little difference between the contrast estimates for direct gaze (1.9) and averted gaze (2.4) (Figure 3).
As outlined previously, we hypothesized that the approach-oriented emotion of anger would elicit higher amygdala activity when combined with direct gaze, where as the avoidance-oriented emotion of fear would elicit higher amygdala activity when combined with averted gaze. The trend of contrast estimates supported this hypothesis as well as provided evidence for the lateralization of this activity.
In line with our prediction for approach-oriented emotions, ROI analyses of contrast estimates revealed that anger elicited more amygdala activity when presented with direct gaze. This finding is consistent with the results of Sato et al. (2010), who found that dynamic presentation of angry expressions directed toward the subject also produced greater amygdala activity. As Adams and Kleck (2005) discuss, direct gaze often communicates dominance or aggression. Furthermore, when exhibiting an approach-oriented emotion such as anger, an individual most often directly approaches or confronts their target, which implies the use of direct gaze.
In accordance with our prediction for avoidance-oriented emotions, averted gaze enhanced the processing of fear by eliciting a greater amygdala response. This result is consistent with research by Hadjikhani, Hoge, Snyder, and de Gelder (2008), who found that the combination of fearful faces and averted gaze elicited significant activity within the amygdala. Their study concluded that averted gaze’s enhancement of fearful expressions were indicative of a threat to one’s environment (Hadjikhani et al., 2008).
Taken together, these results suggest the significance of the amygdala in processing facial expressions of emotion and gaze direction. Furthermore, these findings combined with previous research (Sato et al., 2010; Adams & Kleck, 2005; Hadjikhani et al., 2008) implicate the amygdala’s role in processing social signals that express either approach or avoidance behavioral intent. Our study further predicted a lateralization of this approach-avoidant effect within the amygdala, which was also confirmed through contrast estimates. The left amygdala produced higher responses for anger, where as the right amygdala elicited higher responses for fear. This lateralization of activity is consistent with Harmon and Jones’ (2001) EEG study, which suggested that approach-oriented emotions are more associated with the left hemisphere, while avoidance-oriented emotions are more connected to the right hemisphere.
Our results also revealed a main effect of emotion, with significant activity for anger over fear within the right fusiform gyrus, right cuneus, and left precuneus. These results implicate the significant role the fusiform gyrus may have in the processing of facial expressions of emotion, a finding we explored further. After localizing the fusiform gyrus area, we performed contrast estimates for the left and right fusiform. The left fusiform gyrus was more responsive for the integration of fear with averted gaze, where as the right fusiform gyrus was more responsive for the integration of anger with direct gaze. Research has suggested that the fusiform gyrus is modulated by the presentation of emotional faces and scenes with high social complexity (Hadjikhani & de Gelder, 2003). However, Sprengelmeyer, Rausch, Eysel, and Przutek (1998) indicated that fearful expressions elicited higher activity within the right fusiform gyrus, which is inconsistent with the present study. These results in the fusiform area are also opposite from the trend of contrast estimates within the amygdala. Hadjikhani and de Gelder’s (2003) imaging study indicated there is a connection between the amygdala and the fusiform gyrus in the processing of emotional recognition, though they did not specify if there was a similar opposite effect. Therefore although both regions appear to be linked in processing of emotion, the exact connection remains unclear.
The current study contains some limitations. First, although our results demonstrate the gaze-intensifying effect of both approach- and avoidance-oriented emotions, we tested only negative types within each domain. Sato et al. (2010) used both a positive (happiness) and negative (anger) type of approach-oriented emotions. The present study used a negative emotion of each type in order to maintain consistency, however it would be beneficial to explore the integration of gaze direction and emotional expression for positive types of approach- and avoidance-oriented emotions. Happiness has been tested as an approach-oriented emotion in other research (Sato et al., 2010; Adams & Kleck, 2005), however there is limited use of a positive avoidance-oriented emotion. Calmness has been suggested as an example of this type, although there is no known study of its effect when integrated with gaze direction.
Second, the focus of this study was on the manipulation of gaze direction in the perception of facial emotions of expression. Due to this factor, other emotional cues such as a tightened brow or pursed lips were not considered. This raises the question of whether gaze direction is a separate cue that affects processing of emotional expressions or rather part of a system of indicators that includes other features of the face. Further research should explore methods that would clarify the role of gaze direction within the emotional cueing system.
Finally, our dynamic presentation of facial expression displayed the sequence of a neutral face changing to an emotional face. Future studies are needed to explore whether different sequences of changes within emotional expression, such as beginning with an emotional expression and ending with a neutral face, would subsequently affect amygdala activity. Sato and colleagues (2010) also noted this limitation and discussed how sequential presentation could have greatly contributed to the emotion-intensifying effect of gaze.
Although further work is required to gain a more complete understanding, our findings indicate the amygdala’s role in the integration of gaze-direction with facial expressions of both approach- and avoidance-oriented emotions. Furthermore, our study suggests a lateralization of amygdala activity, suggesting approach-oriented emotions are processed within the left amygdala, where as avoidance-oriented are processed as a function of the right amygdala.
Adams, R.B., & Kleck, R.E. (2005). Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5, 3-11.
Hadjikhani, N., & de Gelder, B. (2003). Seeing fearful body expressions activates the fusiform cortex and amygdala. Current Biology, 13, 2201-2205.
Hadjikhani, N., Hoge, R., Snyder, J., & de Gelder, B. (2008). Pointing with the eyes: the role of gaze in communicating danger. Brain and Cognition, 68, 1-8.
Harmon-Jones, E., & Sigelman, J. (2001). State anger and prefrontal brain activity: Evidence that insult-related relative left-prefrontal activation is associated with experienced anger and aggression. Journal of Personality and Social Psychology, 80, 797-803.
Sato, W., Kochiyama, T., Uono, S., & Yoshikawa, S. (2010). Amygdala integrates emotional expression in response to dynamic facial expressions. NeuroImage 50, 1658-1665.
Sprengelmeyer, R., Rausch, M., Eysel, U.T., & Przuntek, H. (1998). Neural structures associated with recognition of facial expressions of basic emotions. The Royal Society, 265, 1927-1931.
*All the brain regions reported here are significant at p < 0.001 with threshold voxel cluster size 9.