Truth and Lies, Courts and Spies

Barbara A. Spellman, University of Virginia, began her presentation with a little quiz drawn from ordinary life — “Is the sky blue?” she asked. “Is the earth round? Were there weapons of mass destruction in Iraq? Was O.J. Simpson guilty?”

“I don’t really care about your answers,” she said. “What I really care about is, how do you know?”

According to Spellman, how people judge the credibility and reliability of a source of information may be as important as the quality of the information itself.

“Sources of information matter more — and differently — from how we might expect,” she said.

Research in this area has implications for intelligence analysis and for the courtroom — two areas where decisions are often made based on information from unfamiliar sources. But we also have to rely on such information in everyday life. “You are the passive recipient of a lot of information,” said Spellman, “but you do not know whether all the information sources are trustworthy or reliable. Your job is to take this messy information and make sense of it.”

Spellman, an expert in human reasoning and APS Fellow (as well as a past Secretary of APS), spoke at the APS 19th Annual Convention on the topic, “Truth and Lies, Courts and Spies: Judgments under Source Uncertainty.” Her current research, funded by the CIA’s “Analyst of the Future” program through a grant from the National Science Foundation, was launched soon after 9/11 with the goal of improving intelligence analysis.

What are some of the relevant characteristics by which we judge information sources? One is trustworthiness, specifically the degree to which we might suspect a source of being deceptive. To illustrate, Spellman used the example of car repairs. Suppose your car breaks down and a professional mechanic (“certain but potentially deceptive source”) says it can be fixed for $500, but a friend (“trustworthy but uncertain source”) says the problem can be repaired for $15. On whom do you rely?

In a set of studies that Spellman conducted with her student Lee Hendricks, participants could choose between a 100 percent chance of winning a certain number of gold pieces, and an 80 percent chance of winning more gold pieces. The research finds that participants’ choices differ according to the “type” of uncertainty associated with the 80 percent condition. When told that the source was trying to be deceptive (rather than merely uncertain due to ignorance), they are more reluctant to take the riskier option.

“Being suspicious of possible deception doesn’t make you a better or more vigilant decision maker,” Spellman said. “It just makes you more conservative.”

The research also shows that when people are asked to make decisions slowly rather than quickly they also become more conservative — whether right or wrong. “Going quickly — going with ‘intuition’ — does not make people better decision-makers in this task,” Spellman said. “It just makes them less afraid of taking risks.”

Confidence is another dimension on which an information source can be judged. Previous studies have shown that sources demonstrating high confidence are believed more than those demonstrating low confidence. But, Spellman said, further research being done with her student Liz Tenney (and Robert J. MacCoun at Berkeley) is showing that a kind of calibration is involved when we are assessing sources of information. In one study, a high-confidence source is confident about everything whereas a mixed-confidence source is confident about some things and less confident about others. When both are proven right about some information, but then the high-confidence source is shown to be wrong about something stated with high-confidence and the mixed-confidence source is shown to be wrong about something stated with low-confidence, the usual preference reverses and the mixed-confidence source is seen as more believable overall. This result is based on a perception that the mixed-confidence source is “well-calibrated” and therefore more reliable. Among other things, this means that specific information that allows a reasoner to assess whether a source is well-calibrated supersedes the general belief that high-confidence sources are more accurate.

(Further information about this topic is reported in Tenney, E.R., MacCoun, R.J., Spellman, B.A., & Hastie, R. (2007). Calibration trumps confidence as the basis for witness credibility. Psychological Science, 18, 46-50.)