Confidence Spills Over Across Unrelated Decisions

Research on metacognition, or “thinking about thinking,” has explored important puzzles about how humans monitor and control their thoughts. One of these puzzles is why people’s beliefs don’t match with reality — such as why, for example, people are often overconfident in their performance on perceptual or memory tasks.

New research by Dobromir Rahnev (Georgia Institute of Technology) and his colleagues has identified one possible reason for poor metacognition. Called confidence leak, it’s the finding that when individuals provide their confidence in a decision on one trial, the next confidence rating provided is likely to be related to the previous one, even when what the content of what they’re deciding on changes over time.

In their paper, published in Psychological Science, Rahnev and colleagues report the results of two experiments and a reanalysis of data from two other experiments. The results were used to form a mathematical model describing how confidence changes from one experimental trial to the next. The researchers also identified neural correlates of this surprising pattern.

In one experiment reported in the paper, 27 subjects completed a challenging visual perception task. A number of Xs and Os appeared on the computer screen in red and blue font. First, subjects were asked to decide whether more Xs or Os were displayed on the screen and then rate their confidence in that decision on a scale from 1 to 4. Next, subjects were asked whether there were more red or blue characters, and again they rated their confidence in that decision. The subjects did this 400 times each.

The researchers found evidence of a confidence leak, observing a correlation between the confidence rating on the decision about how many Xs and Os there were and the decision about how many characters of each color there were (r = .23). Almost all of the subjects showed this pattern. And the pattern was even more robust when the researchers took other factors, such as accuracy and reaction time, into account.

This finding stands out because there is no theoretical reason why confidence on the letter task should have been related to confidence on an unrelated task (i.e., distinguishing colors). In fact, the researchers found that actual performance across the two types of decisions was not correlated. That is, although confidence “leaked” from one trial to another, actual performance did not.

To follow up, the researchers replicated and extended the results of the first experiment, and they also analyzed data from two prior studies. The results revealed that subjects who provided more accurate confidence ratings also showed less confidence leak. Additionally, the authors found that the degree to which confidence leaked across a subjects’ trials was related to the gray matter volume in regions of the prefrontal cortex of the subject: the more gray matter they had, the less leak they tended to show.

The authors suggest that future research should investigate whether confidence leak occurs outside of the laboratory, for higher-level cognitive tasks.

“If similar confidence leak is present for cognitive tasks, there will likely be a number of real-world implications related, for example, to how the level of confidence of witnesses in courts, doctors examining mammograms, or drivers deciding whether road conditions are dangerous can be influenced by seemingly irrelevant contexts,” they conclude.

You can download the materials, data, and computer code for this paper on Zenodo. This study received badges for Open Data and Open Materials.

References

Rahnev, D., Koizumi, A., McCurdy, L. Y., D’Esposito, M., & Lau, H. (2015). Confidence leak in perceptual decision making. Psychological Science. DOI: 10.1177/0956797615595037