Thinking With Gestures

Can gestures speak louder than words? APS President-Elect Susan Goldin-Meadow certainly thinks so. During her William James Fellow Award Address, Goldin-Meadow shared highlights from her seminal research on the power of gesture, beginning with the integral role that gestures play in human learning and cognition.

“You’ve got to be careful how you move your hands, because as you’re moving your hands, you may be changing the way you think,” the University of Chicago psychological scientist warned an audience at the 2015 APS Annual Convention in New York City.

Goldin-Meadow’s research has shown that the gestures we produce when we talk are not merely random movements used for emphasis — instead, these gestures are reciprocally tied to our thoughts and even to our ability to learn. Goldin-Meadow defines gesture as bodily action that represents information and thus has an indirect effect on the world — unlike actions that affect the world directly (e.g., gesturing about opening a jar compared with twisting open the actual jar).

“The gestures we produce provide a window onto our thoughts, and that window often gives a different view from the window that speech gives us,” she explained. “Importantly, gesture not only reflects what we know, but the gestures we produce can change what we know.”

Gesture Reflects Knowledge

Through an influential series of experiments, Goldin-Meadow and colleagues have demonstrated gesture’s integral role in learning. By observing the gestures children made while explaining how they (incorrectly) solved a problem, Goldin-Meadow and colleagues were able to predict a specific child’s ability to successfully learn how to solve the problem.

Using a task originally developed by Jean Piaget, Goldin-Meadow showed children ages 5 to 8 two rows of six checkers carefully lined up so that each checker in one row corresponded with a checker in the paired row. A researcher then spread out the checkers in one row so that there was a wider space between each checker, creating one row that was longer than the other.

Then, Goldin-Meadow asked the children which of the two rows had more checkers. Children didn’t always understand that the two rows, though different lengths, still had the same number of checkers — that is, they didn’t understand conservation of number (i.e., that quantity doesn’t necessarily change when its shape or the container holding it changes).

Goldin-Meadow was able to predict which children would learn conservation based on the kinds of gestures the children produced when they explained why they thought the rows had different numbers of checkers. When asked for an explanation, some children said, “It’s different because you moved them” while using their hands to mimic the spreading gesture the researcher had used. These children were classified as gesture–speech “matchers”; the gestures they made with their hands matched the reasoning they articulated with their words.

Other children, however, gestured in ways that did not match their speech. These children gave the same verbal explanation for why the two rows were different — the researcher spread one row apart to make it longer — but, instead of using a spreading motion, the children pointed at a checker in one row, then at a checker in the other row, then back to the first row, and so on.

“The child is using his gestures to pair up the checkers in one row with the checkers in another row, which is the first step toward one-to-one correspondence,” Goldin-Meadow explained. “When we found kids like this, we thought, maybe they’re in a different cognitive state from the gesture–speech matchers.”

And, in fact, these children, called gesture–speech “mismatchers,” were far more likely to succeed on the task (that is, to recognize that the number of checkers in the two rows was the same even when one row was spread out) after instruction on a conservation-of-numbers task than were the children who matched their gestures and speech.

“What this suggests is that gesture is really an effective tool for instruction — teachers can use it to figure which of their students are ready to learn a particular task,” said Goldin-Meadow. “Gesture conveys substantive information about a speaker’s thoughts. The ideas conveyed in gesture aren’t always expressed in speech, which is where things get interesting.”

In one experiment, researchers taught groups of third and fourth graders a strategy for solving a simple math problem. They were given an equation (e.g., 7 + 8 + 5 = _ + 5) and asked to fill in the blank to balance the equation. These types of equations can be challenging for math novices; they tend to either add up all of the numbers in the whole equation (and get an answer of 25 for this problem) or add up all of the numbers on the left side of the equation (and get an answer of 20).

One group of children received only verbal instruction. Two other groups were taught using matching or mismatching speech and gestures. The relationship between speech and gesture had a profound impact on how well the children learned: Children who received mismatching gestures from an instructor learned more than children who received matching gestures and more than children who received only verbal instructions.

Additional studies showed that gestures can even help learners extend their knowledge to unfamiliar types of problems — which is at the heart of learning.

Gestures That Lead Us Astray

While gesture is clearly a powerful tool for enhancing learning, the information conveyed in gesture also can lead people astray. Children pick up on information in an adult’s gestures even when the information isn’t true — a phenomenon that extends to eyewitness testimony.

“I don’t want to argue that gesture is an unqualified good in terms of learning,” Goldin-Meadow explained. “What I want to argue is that gesture is a powerful tool for learning. It can be used for good, but it can also be used for evil.”

Eyewitness testimony, while sometimes cited as the gold standard for evidence in the legal world, is often unreliable. In a series of experiments, Goldin-Meadow and colleagues demonstrated that gesture may be one way to unduly influence witnesses, particularly children.

In one example, Goldin-Meadow shows a video of a child sitting at a table with a researcher. The child has just watched a musician perform. When the researcher asks the child a perfectly formed open-ended question about the musician — “What else was he wearing?” — the child lists off a series of different types of hats. The musician, however, was not wearing a hat — so how did this open-ended question lead to a series of hat answers?

Looking at the video carefully, you will see that the researcher makes a subtle gesture as she asks the question, pulling the brim of an imaginary hat over her eyes.

“This phenomenon is incredibly powerful, yet completely ignored by the legal world,” says Goldin-Meadow. “So, we don’t know necessarily when interviewers are misleading eyewitnesses.”

Making Meaning From Gestures

Whether used for good, evil, or neutral ends, gestures have power because people extract meaningful information from them — even without realizing it.

This tendency became apparent in another experiment on learning in which children were asked to solve an equation of the same form: 7 + 8 + 5 = _ + 5. All of the children were taught to say the words, “I want to make one side equal to the other side.” Some children were also taught to point with two fingers at the two numbers whose sum was missing from the blank (7 + 8), but another group of children was taught to point at the wrong two numbers (8 + 5). A third group was taught how to solve the equation using only words.

Intuitively, it would seem that the wrong-number condition would be a disaster, but surprisingly, even a misleading gesture led to better learning than no gestures at all. The more correct the child’s gestures were, the better the child’s performance, but even gestures conveying only partially correct information were more helpful for learning than words on their own. The concept of grouping — metaphorically grouping sets of numbers together  — seemed to emerge from the gestures the child was taught to produce, even if the gestures highlighted the wrong two numbers.

“Children start out producing rote movements with no real understanding of what they’re doing, but over time they come to recognize grouping in their own gestures,” Goldin-Meadow explained.

Gesture actively brings action into a speaker’s mental representations, and those mental representations then affect behavior in compelling ways — at times more powerfully than the actions on which the gestures are based, Goldin-Meadow has found.

William James famously said, “Action may not always bring happiness, but there is no happiness without action.” In her address, Goldin-Meadow said she would like to make a friendly amendment to those words: “Gesture may not always bring happiness, but there is no happiness without gesture.”