Making Thinking Visible

The arts allow us to extend our minds into the world when words would get in the way, said APS President Barbara Tversky (Teachers College, Columbia University and Stanford University) during her Presidential Symposium at the 31st APS Annual Convention in Washington, DC. Tversky invited researchers and experts in the arts to show how our creative pursuits help us forge connections between our internal and external worlds.

Editing Everyday Events

Movies occupy a substantial footprint in our cultural and economic landscape, giving us vivid experiences that we may remember better than most of our real lives — in part because of the way editing allows filmmakers to manipulate event boundaries, said APS Fellow Jeffrey M. Zacks (Washington University in St. Louis).

“The brain itself is a pretty impressive film editor,” Zacks explained. “Commercial editing practices give a sequence of views that cater to our evolved visual routines.”

In the real world, objects appear stable because our visual systems have evolved to construct coherent events that smooth over the discontinuities caused by our eye movements and blinking, among other physiological limitations. In the world of film, an edit that cuts from one shot to the next can cause every point in an image to change, a huge visual disruption human minds had not encountered before film was invented about 125 years ago. Zacks’ research suggests our brains may react to this by filling in the blanks, a phenomenon known as edit blindness, so long as the edits align with the natural event boundaries on screen, such as when a character walks through a doorway or finishes washing their hands.

These edits and events can influence what we remember about a film. In one study, Zacks and colleagues tracked participants’ eye movements as they watched a pair of short films about 2 days in the life of an actor; the days were either exactly the same or included some slight changes. For example, on the first day, the actor might be shown to enter her home by unlocking a doorknob and, on the second day, she might do the same thing or switch to unlocking the deadbolt. When things changed, participants who wrongly predicted that she would unlock the same lock by visually attending to that area were more likely to remember the change than those who did not.

In our day-to-day lives, retrieving past events guides our comprehension of ongoing experiences, influences how they are stored in memory, and helps us to predict the future.

In film, Zacks continued, repeating or contradicting past events can also lead to a sense of nostalgia or surprise, allowing filmmakers to leverage our neural mechanisms to create new experiences on the silver screen.

No Straight Lines

“Art and science, we have something in common: There’s never a final answer. It’s always a process, there’s always something else to ask,” said visual artist Andrea Kantrowitz (State University of New York at New Paltz).

To Kantrowitz, a bad drawing is one that relies, as does bad science, on unwarranted certainty. But good drawing is not simply the ability to reproduce something exactly as it exists in the world or in the mind, she continued. Novices tend to rush toward certainty. Experts use ambiguity and improvisation to prompt discovery, an emergent process that weaves together specific details as part of a coherent whole over time.

Interactions between top-down conscious awareness and bottom-up perceptual processes, between habits and routines, can also lead to the sense that “someone else is in the room” directing the artist’s hand, a form of hyperactive agency detection that can lead artists to feel a drawing is almost drawing itself, said Kantrowitz, cofounder of Thinking through Drawing, an international network linking artists, educators, and researchers across disciplines.

“Drawing makes thinking visible,” she said. “Thoughts not ready or able to be put into words can be put down sometimes in arrangements of marks and lines.”

Similar to the scribble birds that audience members created during her talk, Kantrowitz’s paintings are often intended to evoke an experience similar to cloud watching, inviting viewers to find landscapes or human forms within the extreme close-up view of the torn flesh of an orange or piece of coral.

“Messy lines can actually get you somewhere, somewhere you’re not expecting,” she explained. “They lead you to see, understand, and imagine the world just a little bit differently.”

Thinking Inside Out

Interactions with the outside world can improve creativity by allowing us to think faster, harder, and with more complexity, said cognitive scientist David Kirsh (University of California, San Diego). This process is often aided by tools, such as an artist’s sketchbook, notations, or a musician’s instrument. Research suggests that we can use our bodies to aid in this process as well — for example, “marking” a dance phrase by approximating the choreography rather than practicing it perfectly can enhance learning, enabling dancers to focus on a phrase’s component parts.

“They’re using their bodies to think,” Kirsh said. “This nonneural activity of the body is being harnessed to permit them to drive their thought faster and with greater precision.”

In a study of 10 professional dancers, Kirsh and colleagues found that participants who practiced a new dance phrase by marking it performed with greater accuracy and memory for detail than they did when they practiced the phrase at full resolution. By using their bodies to create a dynamic but imperfect model of the dance phrase they wanted to perform, dancers were better able to visualize what they wanted the phrase to look like, Kirsh explained.

People are able to coordinate what they’re thinking inside with some of the movements and transitions they cause on the outside, Kirsh said. Architects, for example, use computer-aided design programs to model their buildings, allowing them to make alterations while accounting for outcomes they would be unable to visualize on their own.

“I see thought as emergent through the interactions with outside things,” Kirsh said.

A Mind for Music

Artistic inspirations and associations can arrive in a number of forms, from harmless earworms to hallucinations spurred by sensory deprivation or intense trauma, said researcher and composer Jonathan Berger (Stanford University). Many of Berger’s chamber compositions and operas have been inspired by such phenomena — including “Theotokia,” which represents the auditory hallucinations of schizophrenia, and “Memory Slips,” a piano trio based on his mother’s vivid musical hallucinations in the last months of her life.

Through a series of transcriptions, Berger was able to identify his mother’s fragmented tune as “Cruising Down the River (on a Sunday Afternoon),” a 1940s radio hit by Eily Beadell and Nell Tollerton.

That still left one question unanswered, though, Berger said:

“Out of all the music she may have heard throughout her life, how was it that she recalled this song? By that point in her life, my mother didn’t recognize me, she didn’t recall who my brother was, and yet she was enjoying a seemingly random, mediocre popular song.”

After further investigation, Berger realized the song wasn’t so random after all — in fact, it had reached the top of the US music charts the same week his brother had been born in 1949.

“My mother’s sensorially deprived world was an incubator and a breeding ground for her musical hallucination, an auditory analogy [to the visual hallucinations of] Charles Bonnet Syndrome,” Berger concluded.