Economic Field Experiments Complement Understanding of Judgment Bias

Bargaining in an open-air market. Making trades at a sports memorabilia show. Monitoring price changes on a ride-sharing app. Every day, people in real-world marketplaces make choices using decision-making biases similar to those explored by psychological scientists in laboratory settings. And field experiments in such contexts “can serve an invaluable intellectual role alongside traditional laboratory research,” John List proposed in a recent review in Current Directions in Psychological Science. Moreover, “fundamental tenets in economics, such as the law of demand, might have deeper psychological underpinnings than most people believe.” 

List is a professor of economics at the University of Chicago whose insights from field experiments have influenced areas from education to corporate social responsibility to government policy. 

“First, decision-making biases identified in research conducted in psychology labs are alive and well in the marketplace,” he wrote in the Current Directions article. Second, such biases can influence key economic variables such as prices, profits, and taxes. Third, the field-experiment approach provides insights that complement psychological research also observed in lab environments. 

Blending experimentation 

In his review, List divided the history of economic field experiments into three periods. The first, in the 1920s and 1930s, began when Jerzy Neyman and Ronald Fisher conceptualized randomization as an instrument for testing different types of agricultural experimentation.  

The second period, in the mid-20th century, was characterized by government agencies conducting large-scale social experiments with groups of individuals affected by government tax and spending programs. 

The third period, which covers the late 20th century to the present, has produced four general categories of experimentation, List explained. These include controlled experiments in laboratories using university students; artifactual field experiments using relevant populations of interest in controlled environments; framed field experiments using relevant populations of interest in natural markets; and natural field experiments, which are covert.  

In one of his own natural field experiments, List used a within-sellers design to explore the degrees to which different sellers in open-air markets discriminated against women buyers. A key mediator, he wrote, “was sellers’ subjective beliefs about the difference between men’s and women’s willingness to pay for the good.” 

List further explored market-based examples of three behavioral biases previously explored in psychological research.  

The “endowment effect” refers to when people demand much more in compensation to give up a good than they are willing to pay to acquire the same good. In research published in 1990, APS William James Fellow and Nobel Prize winner Daniel Kahneman and colleagues provided evidence of the effect in eight experiments. The researchers argued that this effect was a fundamental characteristic of individual preferences—one that market experience could not eliminate. For corresponding field experiments, List used artifactual, framed, and natural field experiments to examine trading patterns of sports memorabilia at a sports-card show and of collector pins at a Disney market. One finding, he explained, was that “Kahneman et al. seemingly overinterpreted their data.” That is, trading experience mattered. For example, List’s own studies found that “people with vast trading experience behaved much less in accord with endowment theory and much more in line with standard economic theory.” 

The “left-digit bias” refers to when a number’s leftmost digit disproportionately affects choices. Psychologists exploring this bias include Fraser-MacKenzie et al. (2015), who combined loss aversion with the left-digit effect to develop a value function predicting the bias’s key characteristics. In natural field experiments, List and colleagues used data from Lyft, the ride-sharing company (List is its chief economist), involving more than 21 million passengers and 600 million sessions. Their findings revealed a consistent adherence to the law of demand: As prices went down, demand went up. But the researchers also found evidence for the left-digit bias’s outsized contribution to the law: A price drop as small as 1 cent below a dollar value (say, from $8 to $7.99) led to dramatic increases in the demand for rides—as if it had been lowered by 50 cents. “Thus, observed customer choice revealed strong warpings, or nonlinearities, between actual and subjective prices, even in the rich decision-making setting of a rideshare app with highly competitive pricing,” List wrote. 

The third bias List explores, the “omission bias,” is the tendency to judge harmful action as worse than equally harmful inaction. “Looking no further than COVID-19 vaccination decisions will reveal the power of this bias,” he wrote, citing people who attribute their resistance to vaccination to “the potential chance of death from the vaccination, even though the probability of dying from the disease itself is much larger.”  

More recently, List worked with the United Kingdom’s tax authority to approach citizens who had failed to repay overpaid government benefits. Breaking with the government’s traditional approach of treating this inaction as an omission, List’s natural field experiment reframed it as a commission, using this language: “Previously, we treated your lack of response as an oversight. Now, if you do not call [telephone number], we will treat this as an active choice.” Repayment rates roughly doubled, generating more than $1.3 million in estimated tax payments. “This is at odds with the traditional omission-bias literature, which interprets omission as a bias rather than a strategic choice,” List wrote. “An ‘omission strategy’ can be part of how a person optimally deals with such decision environments.” 

List’s review demonstrates how insights from economic field experiments can help psychological scientists think about the external validity of their research. “In the end, all empirical results are externally valid for some settings, and no empirical results will be externally valid for all settings,” he concluded.  

Reference 

List, J. A. (2021). How field experiments in economics can complement psychological research on judgment biases. Current Directions in Psychological Science, 30(5):431–436. https://doi.org/10.1177/09637214211033602