Louder Than Words

Humans show remarkable sophistication in how we use language, composing everything from 140-character soundbites to 800-page novels, expressing thoughts and feelings that range from quiet confessions to exuberant declarations of love.

The fact that we are capable of such linguistic feats is one of humankind’s defining features. Yet there is so much that we communicate and express without uttering a single word. Behavioral scientists have long been interested in understanding the various means by which we engage in nonverbal communication and the reasons why we do so. In an Integrative Science Symposium at the inaugural International Convention of Psychological Science in Amsterdam, the Netherlands, an eminent panel of researchers explored the latest science investigating how we transmit powerful signals to one another in the absence of speech.

Honest Signals

To understand the role that nonverbal expressions play in human communication, researchers often have focused on the communication of emotion. Emotions reflect our appraisal of a situation or stimulus — such as whether it’s likely to bring pleasure or danger — and these appraisals spur behaviors that signal how we intend to respond. We can communicate our emotions to others through words, of course, but we also can express our emotions nonverbally, modulating acoustic qualities of the voice and positioning muscles in the face and body, often without awareness.

Both verbal and nonverbal modes of communication allow us to communicate our emotional appraisals, and our intentions to respond, before we act. But what distinguishes the nonverbal signals, says APS Fellow Klaus R. Scherer of the University of Geneva in Switzerland, is their authenticity:

“Since we have developed this beautiful tool, language, it would be perfectly fine to just say how you feel and that’s it. But that is not it,” Scherer noted. “What we need to do is produce an honest signal of emotion and that has to be done without fuzzy categorization and labeling.”

While our verbal expressions may be honest, expressing our emotions through language requires a level of processing and abstraction that necessarily distances the verbal expression from the core emotion. To the extent that our facial and postural expressions are spontaneous and synchronized with the situation or stimulus, they provide more authentic signals of emotion, free from the constraints of language.

That certain emotion expressions seem to be universal speaks to their fundamental communicative function. Seminal research led by APS William James Fellow Paul Ekman has shown that activating facial muscles in specific groupings, or action units, is consistently associated with particular emotions. The simultaneous activation of the zygomaticus major muscle, which runs from the cheekbone to the corner of the mouth, and the orbicularis oculi muscle surrounding the eye, for example, is reliably associated with pleasure and genuine smiles. The combination of a furrowed brow, compressed mouth, and flared nostrils, on the other hand, is associated with anger.

Exquisite Sensitivity

To operate as an effective signal, however, it’s not enough for nonverbal expressions to be produced in a reliable way; they also must be interpreted in a reliable way. In their current work, Scherer and colleagues find a strong link between facial expressions and perceptions of those expressions, indicating that we are exquisitely attuned to read emotional information from faces without any assistance from language.

In one study, participants saw a series of dynamic computer-generated avatars and were tasked with identifying the appraisal that would be going through the head of such a person. Scherer and colleagues programmed the avatars to display specific facial expressions of emotion composed of particular action units. Not only did the participants show strong agreement in the appraisals they identified from the facial expressions, their appraisal ratings also varied in a systematic fashion across different action units.

Findings from a large online study, conducted with a representative sample, showed a similar pattern of results: Participants used the avatar’s facial expression to “read” his or her mind, rating the extent to which the avatar’s appraisal was novel, pleasant, and conducive to goal achievement in a consistent way.

But the face is only one mode by which we express emotion without words. Considerable research has shown that our posture and the positioning of our bodies effectively communicate our emotional states as well. Building on earlier neuroimaging work using electroencephalograms (EEG) and functional magnetic resonance imaging (fMRI), APS Fellow Beatrice de Gelder of Maastricht University in the Netherlands and colleagues are now using transcranial magnetic stimulation (TMS) to better understand the mechanisms that drive how we perceive emotion expression in bodies.

In an experiment, de Gelder and colleagues created images of human bodies and control images of animals that showed either threatening or neutral postures. Participants viewed two images in succession and had to determine whether there was a postural difference between them. The researchers found that applying TMS to an area of the brain in the extrastriate — which is involved in visual perception of human body and body parts — impaired participants’ ability to detect changes in neutral positions but had no observable effect on their perception of bodies displaying fear.

Applying TMS to the posterior superior temporal sulcus (STS) — an area of the brain broadly implicated in social perception — on the other hand, actually improved participants’ ability to detect changes in these so-called fear bodies.

“The body area and early visual cortex do not play a critical role one way or another, but when you stimulate posterior STS and also a parietal area, subjects become better at decoding, at noticing really minor differences in the postures of the
human-fear bodies,” de Gelder explained.

In another study, the researchers used a binocular rivalry paradigm — simultaneously presenting different images to the left and right visual fields — to gauge whether participants would show similar processing in the absence of conscious deliberation. Again, the results showed that stimulating the posterior STS improved participants’ sensitivity to changes in fearful body expressions.

Findings from these and other studies suggest that perceiving the emotion expressions that are displayed via bodies is carried out via the dorsal visual stream and does not require representation and identification of the body as an object before an emotion expression is perceived. In other words, we process emotion expressions in bodies quickly and automatically, via mechanisms that do not require consciousness or verbal labeling.

Power of Pride

Zeroing in on the expansive postural display of pride, researcher Jessica Tracy (University of British Columbia, Canada) has found converging evidence that body expressions, just like facial expressions, can provide robust and reliable signals that influence perceivers’ responses on an automatic, implicit level.

Postural displays of pride — standing tall, with outstretched arms and expanded chest — are perhaps most commonly seen among winning athletes and politicians, but they also emerge when we celebrate everyday victories at work and at home. Cross-cultural studies indicate that pride displays are reliably recognized by people around the world, and Tracy’s work shows that they are more commonly produced by both sighted and blind athletes who have won a match than by those who have lost. Together, these findings suggest that the expression is not a learned response to success, but rather is innate and spontaneously produced.

Tracy and colleagues hypothesize that pride displays are universal because they serve an adaptive function, signaling an individual’s success and, by extension, high status. In one study, for example, the researchers asked student volunteers to write out the digits of pi to the ninth decimal place, an impossible task for most people. Participants would receive extra cash if they got the answer right, and they had the opportunity to see the answer provided by another participant before they responded. The researchers predicted that participants would be most likely to copy the answers of a student who displayed pride as opposed to other emotion expression. And that’s indeed what they found, indicating that pride effectively signals success to others.

Additional findings suggest that pride displays, just like facial expressions and bodily displays of fear, are processed on an automatic, nonverbal level. Tracy and colleagues had participants complete Implicit Association Tests, in which different emotion expressions were paired with words that indicated high or low status. They found that participants were much faster to associate pride with high status and other emotion expressions with low status than they were in making the reverse pairings. When the researchers exported the study to a remote island in Fiji, they saw the same pattern of reaction times.

The link between pride and high status is so powerful, in fact, that the pride display is a valuable signal of status even when competing contextual information is available. In one study, participants viewed a photo of either a homeless-looking man or a professional-looking man whose posture expressed either pride or shame. Their ratings indicated that expressing pride elevated the homeless man’s status:

“Displaying pride can make a homeless man automatically [be] perceived as equal in status to a business man who had a bad day,” Tracy explained.

Vocal Dominance

While face and body expressions are clearly effective as nonverbal signals, accumulating evidence suggests that a particular feature of the voice — pitch — may be an especially robust cue. What we perceive as pitch is measured in terms of fundamental frequency, or the rate at which the vocal folds vibrate. Research shows that, across the world, women tend to vocalize at about twice the fundamental frequency that men do. This sex difference is not just noticeable; it is much greater than that seen in almost all other secondary sex characteristics, including weight, biceps circumference, height, and strength.

From a physiological perspective, this sex difference is no mystery: Increasing testosterone levels during puberty ultimately lead male vocal folds to lengthen and thicken, causing them to vibrate more slowly.

But, as scientist David A. Puts points out, this doesn’t explain why the difference exists in the first place.

“Why did we evolve such a very large sex difference in a mode of communication that both sexes use equally?” asked Puts. “Doesn’t that strike you as bizarre?”

An intuitive answer would be that the sex difference has to do with physical size — men tend to be taller than women and this height difference should result in them having longer vocal cords. But data show that men’s vocal folds are about eight times longer than we would expect given the sex difference in height.

The real explanation, research suggests, may lie in sexual selection. There is considerable evidence that the selection mechanism of mate choice may have played a role in causing these differences: Men tend to prefer women with high-pitched voices and women tend to prefer men with low-pitched voices. The fact that fundamental frequency is correlated with levels of certain hormones suggests that it may serve as a physiological indicator, signaling high fertility in women and robust immune function in men.

But sex differences in fundamental frequency are perhaps even more strongly linked with another mechanism of sexual selection: contest competition. That is, vocal pitch may function as a nonverbal cue that signals dominance to competitors.

In line with this theory, data reveal that people perceive masculine, or low-pitched, voices as belonging to dominant individuals. And research suggests that the relationship also works the other way — men’s perceptions of dominance seem to affect how they modulate the pitch of their voices in a competitive interaction.

Under the guise of a dating game, Puts and colleagues told male participants that they would be competing with a man in another room to have lunch with a woman. The participants’ voices were recorded at the beginning of the study and then again when they directly addressed their “competitor.” The results showed that men who rated themselves as less physically dominant tended to raise their pitch when talking to the competitor, while men who rated themselves as more physically dominant tended to lower their pitch.

These findings demonstrate the robustness of vocal pitch as a nonverbal signal of physical dominance and may help to shed light on nonverbal communication more broadly, says Puts. For example, the findings explain why we tend to lower our pitch when issuing a command and typically raise our pitch when asking a question.

Real-World Consequences

While we may not be explicitly aware of the impact that nonverbal signals such as vocal pitch and body expression have on our day-to-day lives, research suggests that their consequences may be far-reaching.

In one study, Tracy and colleagues asked participants to evaluate a job candidate applying for a position as a bank manager. Some participants saw a strong résumé indicating that the candidate had an honors degree from a top university, good grades, and impressive extracurriculars, while others saw a much more mediocre résumé. The participants then watched a video clip supposedly taken from the candidate’s interview; some saw a video in which the candidate showed a postural display of pride, while others saw a candidate displaying shame. Critically, the candidate followed the same script in both videos.

When participants evaluated the candidate, they seemed to judge her intelligence primarily on the résumé. Those who saw a strong résumé rated the applicant as much smarter than those who saw a mediocre résumé .

However, the résumé didn’t seem to matter in final hiring decisions. Participants were more likely to “hire” the fictitious candidate if she had displayed pride in her interview than if she had displayed shame, regardless of the quality of her résumé.

Similarly, research indicates that there may be significant real-world consequences related to having a high- or low-pitched voice. Puts pointed to correlational studies showing that people perceive individuals with low voices as having greater leadership abilities and being worthy of more respect. People are more likely to “elect” both male and female candidates with low-pitched voices in hypothetical elections, and compensation data from publicly traded companies indicate that male CEOs with lower voices actually may earn more than their higher-pitched peers.

Future Directions

Studies spanning different levels and methods — from investigations into neural mechanisms to explorations of social consequences — continue to show that nonverbal expressions are produced and perceived in robust, reliable, and potentially powerful ways. But there are still many outstanding questions that must be answered to disentangle the mechanisms that drive nonverbal expressions and our responses to them.

Pinpointing the origins of these expressions, and their potential adaptive value, is a particularly thorny issue. As Scherer noted, researchers often make the indirect argument that if we reliably perceive an appraisal from a particular face, then that appraisal must be the one that is being evolutionarily expressed. But this conclusion requires an inductive leap.

“We need more evidence for both parts of the equation, trying to get at it through different means,” Scherer argued.

In addition, emerging research suggests that various individual-level and contextual factors do have an impact on how we perceive and respond to nonverbal expressions. For example, de Gelder and colleagues have found that people show quite substantial differences in brain activity depending on whether they are directed to look at the aggressor or the victim in a scene depicting conflict, and these differences in activation are further correlated with personality factors related to empathy.

In future research, de Gelder argues, “We have to take into account the different functionality of face, body, and voice,” rather than looking for abstract emotion categories, and pay attention to differences related to “emotion content, to visual awareness, to social context, to personality factors, and to ethnic and cultural factors.” œ