The Mind Uses Syntax to Interpret Actions

Most people are familiar with the concept that sentences have syntax. A verb, a subject, and an object come together in predictable patterns. But actions have syntax, too; when we watch someone else do something, we assemble their actions to mean something, according to a new study published in Psychological Science, a journal of the Association for Psychological Science. 

“There are oceans and oceans of work on how we understand languages and how we interpret the things other people say,” says Matthew Botvinick of Princeton University, who cowrote the paper with his colleagues Kachina Allen, Steven Ibara, Amy Seymour, and Natalia Cordova. They thought the same principle might be applied to understanding actions. For example, if you see someone buy a ticket, give it to the attendant, and ride on the carousel, you understand that exchanging money for a piece of paper gave him the right to get on the round thing and go around in circles. 

Botvinick and his colleagues focused on action sequences that followed two contrasting kinds of syntax—a linear syntax, in which action A (buying a ticket) leads to action B (giving the ticket to the attendant), which leads to outcome C (riding the carousel), and another syntax in which actions A and B both independently lead to outcome C. They were testing whether the difference in structure affected the way that people read about the actions. 

The experiments were based on studies suggesting that people read a sentence faster if it comes after a sentence with the same grammatical form. But in this case, the scientists varied relationships between actions rather than the order of parts of speech. In one experiment, volunteers read sentences that described three actions. They took one of two forms: either one action leads to the next action, which leads to the outcome, such as “John purchased a carousel ticket, gave it to the attendant, and went for a ride,” or sentences like “John sliced up some tomatoes, rinsed off some lettuce, and tossed together a salad”—in which both of the first two actions lead to the result, without the second depending on the first. 

Indeed, people were able to read a sentence more quickly if it followed a set of actions arranged the same way than if it followed a sentence of the other type. This indicates that readers’ minds had some kind of abstract representation of the ways goals and actions relate, says Botvinick. “It’s the underlying knowledge structure that kind of glues actions together. Otherwise, you could watch somebody do something and say it’s just a random sequence of actions.” 

In the carousel example, a Martian might not understand why John exchanges paper for another piece of paper, why he gives the paper to the other man, why he goes around and around in circles, and what relationship there is between these actions. As humans, we’ve worked all of those things out, and Botvinick thinks he’s a step closer to understanding the process.