Navigating Regret in Decision-Making

Under the Cortex features William Ryan from UC Berkeley and Stephen Baum from Washington University in St. Louis who recently published an article on this topic in APS’s journal, Psychological Science. 

APS’s Özge Gürcanlı Fischer Baum chats with Ryan and Baum about their new article on how regret impacts risk taking and financial decision making. They also discuss what researchers mean when they talk about logical decisions and how that differs from how it is colloquially used. 

Send us your thoughts and questions at [email protected].

Unedited transcript

[00:00:01.540] – APS’s Özge G. Fischer Baum

Human reasoning is not flawless. Research shows that we are not always logical decision-makers. This is Under the Cortex. I am Özge Gürcanlı Fischer Baum with the Association for Psychological Science. In this episode, Under the Cortex examines how human reasoning can be contextual, discussing the biases that people have in decision making, especially in settings with positive outcomes. I’m joined by William Ryan from University of California, Berkeley, and Stephen Baum from Washington University in San Luis. They have a recent article published in APS’s journal, Psychological Science. William and Stephen, thank you for joining me today. Welcome to Under the Cortex.

[00:00:52.910] – William Ryan

Thank you.

[00:00:53.780] – Stephen Baum

Thank you so much for having us.

[00:00:56.150] – APS’s Özge G. Fischer Baum

William and Stephen, would you introduce yourselves briefly to our listeners?

[00:01:01.490] – William Ryan

Yeah. I’m William. I’m a PhD candidate at UC Berkeley, so I’m five years in my PhD there. My research tends to look at within this broad field of consumer behavior and marketing I mean, how people make decisions under situations of uncertainty. So where there’s risks, where there are things they don’t know, how do they make those sorts of decisions? And do they make them well? Do they make them poorly? And if so, why?

[00:01:27.240] – APS’s Özge G. Fischer Baum

Thank you. What about you, Steven?

[00:01:29.940] – Stephen Baum

I’m Stephen. I’m a postdoctoral researcher at Washington University in St. Louis. I got my PhD in 2023 from UC Berkeley. And I’m very broadly interested in understanding how consumers make judgments and decisions. I’m interested in consumers’ preferences and how researchers measure and understand consumers’ preferences.

[00:01:53.650] – APS’s Özge G. Fischer Baum

Yeah, thank you. Let’s start with people not being logical decision-makers. Why is this something important to study, in your opinion?

[00:02:05.270] – William Ryan

I think what makes this important is because often, I think when we talk with people not being logical decision-makers, in these kinds of situations, tend to mean, especially for consumers, they’re not following the economic logic of the situation. And usually what that means is they’re making decisions which on average, over time, are going to result in worse outcomes for them. The reason they’re a logical decision is because they’re the one which gives you the best outcomes. So in a lot of cases, if people are diverging from that, if they’re making some other type of decision, they’re actually making themselves worse off. So we think it’s important to figure out these cases where people are making these suboptimal illogical decisions and figure out why and if we can improve that. Or at the very least, let them know, Okay, this is a decision you’re making illogically. Maybe you have some other reason for it which we’re not seeing. And this helps us figure that out.

[00:02:57.430] – APS’s Özge G. Fischer Baum

Yeah. Steve, what do you think?

[00:02:59.770] – Stephen Baum

I I agree with everything that William said. I think one thing that I would add is that when people are making suboptimal decisions, that has implications often not just for the welfare of the individual decision maker, but the welfare of other people. I think these are important things to study just because they affect broad, larger groups of people as well.

[00:03:22.370] – APS’s Özge G. Fischer Baum

Yeah, that’s an important point. We live in a society, obviously, and every decision we make will touch someone. Even if I’m buying a house, it is going to affect my family, my family’s finances and all that. Yeah, that’s a great point. Thank you. Let’s talk about the details of your study a little bit. You, in fact, have a series of experiments, and they highlight a new way that people don’t make logical decisions. Can you talk a little bit about the effect you studied and a little bit more details of your experiments?

[00:03:56.870] – Stephen Baum

What we did is we We conducted a variety of studies in which we showed people gambles and lotteries. You would tell people, Hey, you’re entered into a lottery where you have a 20% chance of winning $10. How much are you willing to pay to change that 20% chance to a 30% chance? And participants give us an answer. Then after they give us an answer to that question, they do another question. That other question might ask them something like, Hey, you’re now entered a lottery. You have a 70% chance of winning $10. How much do you value changing that 70% chance to an 80% chance? Each of our participants in our studies completes many trials like that. Trials that, in essence, vary their starting probability of winning the lottery and give them an opportunity to value an improvement that increases the likelihood that they win some a prize.

[00:04:58.040] – William Ryan

I think a good And the way to get the intuition for what we’re talking about is, at least for me, and I think for most people we’ve talked to, people tend to feel the intuitive answer there is they’d rather improve this 80% chance lottery. They’d rather invest in the thing which they’re already likely to succeed in. And that’s the basic effect we find in is this preference for increasing already likely outcomes. And this is going to get a little bit into the weeds, but we think the reason that this is happening is because even though those two situations, the size of the improvement is the exact same, it’s 10 percentage points either way. If you instead think about things in terms of regret, it looks a little bit different. So if you imagine that what people are doing when they’re making this decision is they’re asking themselves not how much will this improvement matter no matter what happens, but instead say, Okay, if I lose, if things don’t work out, how often was it going to be my fault? And then you start to see things look pretty different between those two. So if you’re applying that 10% increase to this 20% to win, 80% chance to lose, but you’re only thinking about what it does to your chances to lose, now you’re going from losing 80% of the time to 70% of the time.

[00:06:09.380] – William Ryan

So it’s lower, but it’s not that much lower. But if you’re thinking about this lottery where you initially have an 80% chance to win, 20% chance to lose, but then we zoom in on, okay, in that 20% of times that I lose, how often would my change have mattered? Now you’re going from a 20% chance of losing to a 10% chance of losing. So you’re fully having your chance to lose. So if you just think about this is like, which of these two choices is going to make it so that if something bad happens, it’s not my fault most of the time. I couldn’t have done anything. It’s going to be better to make that change to the higher chance to win lottery.

[00:06:43.090] – APS’s Özge G. Fischer Baum

People don’t want personal responsibility for the losses. Also, I think they want some type of predictability because they already want to invest in something that already has some likelihood of having a positive outcome.

[00:07:01.270] – William Ryan

Yeah.

[00:07:01.890] – APS’s Özge G. Fischer Baum

Interesting. As I said, you have brilliant series of experiments, but for our listeners, I would like to ask you, how might someone find themselves with this decision making in their everyday life?

[00:07:16.980] – William Ryan

Yeah, absolutely. I think there’s all sorts of cases in which you can run into these kinds of decisions, and they’re generally not going to be as obvious as being entered into a lottery. Unless you’re super into raffles, it’s probably not happening quite like that. But you do have all sorts of And in some cases, you could imagine if you’re a student and you’ve got two different classes with different tests you need to study for. And maybe you have one class where you think you’re already a little bit more likely to pass the test, and another where you think it’s going to be pretty tough to pass the test. So that’s a case where you’ve got two different risky outcomes, whether or not you pass the test with different chances of success, and you need to decide how to invest your time into studying for one versus the other. So I think what our research would suggest is that people might be a little bit disproportionately biased to prefer studying for that test where they already have a pretty good chance of success, and maybe they might be neglecting these ones where things are less likely to work out.

[00:08:09.070] – William Ryan

And I think similar logic applies in all sorts of cases, right? You can think about if you’re in a business and you have different projects or different ad campaigns with different chances of success, maybe you’re tending to neglect the ones that show a little bit more of a long shot. You can think about it in medical decisions when you have different treatments with different chances of success. Maybe people are discounting improvements to less likely to survive patients, for example. So there’s all sorts of cases where you’ve got these different kinds of chances of success, you’ve got these opportunities to change it. And we think it’s possible people are misinvesting their resources in those cases.

[00:08:43.560] – APS’s Özge G. Fischer Baum

What would you say the two main results of your study?

[00:08:49.590] – William Ryan

I think I would say two main results. I think that the first is in that design which Steven laid out, where we’re giving people these cases where they’re entered into lotteries with a certain chance of winning money and different chances of winning. What we end up finding is that when people have these low initial chances of winning, they pay very little for their improvements. But as there’s chances of winning increase, they pay increasingly more for improvement of the exact same size. And this is true even when they’re playing with actual money. So people, even when the outcomes are all real, when it’s actually going to affect their finances, they tend to invest relatively way too little into these initially unlikely to succeed ones and much too much in those very likely to succeed ones. And the pattern in which they do that is the one which is predicted by our regret account. If you look into the math of the regret and you predict what exactly that pattern should look like, it ends up looking very similar to what people, in fact, do. So I think that’s one of our basic findings. And then I think the second finding, one we’re pretty excited about, is the And then we did the final study of the paper, where we looked at how this actually translates to people making choices.

[00:10:07.830] – William Ryan

So we asked people to basically make choices between investing into one of two different risky prospects, and those varied by domains. We asked about a bunch of different types of scenarios. So it could be giving a treatment to one of two critically ill patients, and one has a higher chance of surviving than the other. It could be focusing on meeting one of two bonus goals as an employee. And we gave people a bunch of choices like this, and we didn’t do it to just regular lay people. We actually gave it to both lay people, so just everyday people off the street. We also gave it to two groups of experts who we thought would be a little bit more experienced in some of these types of decisions. And one group was medical professionals, like nurses and doctors. The other group was gig economy workers, so people who drive Uber, DoorDash, stuff like that. And we gave them all these choices, and we set up the choices so that there be one option which would reduce their regret more. So it would be a higher initial chance of success, but the actual size of the improvement they would get was a lot smaller.

[00:11:13.510] – William Ryan

So we might say, for example, Okay, you’re a doctor. You can either take one patient from a 20 to 30 % chance of survival or another from an 80 to 85 % chance of survival. So what’s interesting about that choice is clearly, if you’re just trying to maximize the number of patients who survive long term, you should really want to improve that first patient’s chances by 10 %, because that’s twice the size of improvement as the second one, where it’s only 5 %. But instead, what we see is about half of people, including about half of all doctors, actually don’t do that. And instead, they prefer increasing the chances of survival of the patient who’s already likely to live, even though the size of that improvement is about half as big as the improvement you would give to the less likely to live already patient. Yeah.

[00:12:00.630] – APS’s Özge G. Fischer Baum

It is the experiment after experiment, it was impressive to see that when people are already invested in something, they just go with that decision. They just want to improve what they already have, right?

[00:12:11.540] – William Ryan

Yeah, exactly. To their own detriment or to the detriment of the people they’re making these decisions for in some cases.

[00:12:22.650] – APS’s Özge G. Fischer Baum

I would like to ask you a question about expertise. People often think that experts make better decisions decisions due to their expertise. How did you find experts fair? What are your thoughts about this?

[00:12:36.740] – William Ryan

I think expertise is actually one of the most interesting things going into this because I think initially when we ran this experiment, We thought it could go either way. So on the one hand, experts have a lot of experience with these types of decisions. So you might think they would see how things worked out and eventually update and be like, Oh, I should really be making these optimal decisions over time, or maybe they’ve just been taught to make optimal decisions. On the other hand, I think something which is really interesting about regret and the fact that this bias is coming from regret. This is going to take a second to lay out, but often when we’re thinking about people making these bias decisions or making these decisions which don’t follow logic, we think of them as using quick mental heuristics or shortcuts to make those types of decisions. And that can be what It drives a lot of biases. And in cases where that’s true, then the more important you make the decision, the less likely people are to use those mental shortcuts, and the more likely they are to logic it out instead. So you’d expect when you make things more important or more real for people or higher stakes, they deliberate more and the bias reduces.

[00:13:49.100] – William Ryan

But what’s interesting is that’s actually the opposite for regret, where the higher stakes the decision is, the more regret you expect to experience if you do experience regret. So it can actually drive us in the other direction where this bias is actually more pronounced for more important decisions. So to the extent that experts feel these decisions are, for example, more important, they might actually show our effect more. So what we actually ended up finding was a mix. So when we looked at these gig workers, and we had them making decisions very similar to the types of decisions they have to make often about where to allocate their time to get a certain bonus. We found that they did that about the same as laypeople did. So we didn’t see a difference in of expertise there. But when we looked at a medical sample and we asked a bunch of doctors to make a decision about a patient, what we actually saw was that the medical professionals were almost twice as likely to make the suboptimal regret minimizing choice as were lay people. So they were about twice as likely as lay people were to choose the option where you got a smaller improvement, but it minimized your regret more.

[00:14:55.440] – William Ryan

So it seems like at least their expertise was actually leaving them to show this bias more and make what were arguably more suboptimal choices.

[00:15:03.010] – APS’s Özge G. Fischer Baum

That’s interesting. Do you think it is maybe because they’re in a caring profession and they don’t want to regret what they do to a human being in the end?

[00:15:14.840] – William Ryan

Yeah. So it’s interesting. We’ve had the chance to talk about this research with some medical professionals and present it to a Department of Doctors and ask them what they thought about it. And from talking to them, it seemed like some of them felt that they could feel this pressure while they were making the decisions. It resonated with them that, Yeah, this is something which I tend to do. Maybe I am focusing on these more likely to succeed cases more than others. But yeah, it’s interesting what exactly is driving that.

[00:15:52.000] – APS’s Özge G. Fischer Baum

Thinking about and talking about the real-world settings, based on your research, do you have any suggestions or how people might make better decisions? Anything for our listeners out there?

[00:16:05.470] – William Ryan

I think one thing to think about, which I think can generally be helpful when making decisions, is It’s easy to focus. If you’re focused on just one decision, you’re making a single time, it can be really easy to focus on the regret you might feel and the consequences of that decision. But if you zoom out a little bit and imagine, okay, if I was making this decision like 100 times, what would I do? And I think what that can often do is it can help you step a little bit away from the regret of this one individual instance and think about what the policy for approaching these decisions, which is going to give me the best outcomes overall. And we’ve actually found that when we made people do that, so we gave people the kinds of choices where there’s the regret-inducing choice, and then there’s the more optimal choice. And we had them instead think about what this would mean for the average outcomes if they repeated this process. We actually saw people made much more optimal decisions and were a lot less biased. So I think if you zoom out from this individual choice and think about your choices in general or your choices as you’re making them repeatedly, that can help you feel a little bit less attached to the outcome of any given one of them and make maybe decisions which are a little bit better on average.

[00:17:28.810] – APS’s Özge G. Fischer Baum

Yeah. Okay, Steven, what is your advice?

[00:17:32.150] – Stephen Baum

I think something that follows almost, logically from what we’ve done in the paper, but is something that we don’t really explicitly say a whole lot about in the paper, is the following. We suggest that people are approaching these improvements by asking themselves, Okay, if something bad happens, how much could I have done to prevent it? This is why people value the the 80 to 90 % change more than, say, the 20 to 30 % change. And what that psychology, I think, misses is that in the case where you are going from an 80 % chance to a 90 % chance, your initial chance of winning the lottery is actually very high. Initially, an 80 % chance is pretty good. So if you’re saying to yourself, If something bad happens, how often would it be my fault? That if something bad happens is almost insensitive to the fact that something bad happening is actually pretty unlikely, all things considered. I think it almost logically follows that getting people to focus on and appreciate, initially, what their chances are, almost before a change or before an improvement movements could potentially change the way that this psychology works.

[00:19:06.110] – APS’s Özge G. Fischer Baum

William and Steven, thank you so much for this fantastic conversation. I personally learned a lot.

[00:19:11.810] – Stephen Baum

Thank you so much.

[00:19:13.830] – William Ryan

Thank you. Thank Thank you. It’s great to be on.

[00:19:17.310] – APS’s Özge G. Fischer Baum

This is Özge Gürcanlı Fischer Baum with APS, and I have been speaking to William Ryan from University of California, Berkeley, and Stephen Baum from Washington University in St. Louis. If you want to know more about this research, visit psychologicalscience.org. Would you like to reach us? Send us your thoughts and questions at [email protected].