Super Rare Items Are Most Likely to Be Missed

Various jobs in security, medicine, and other fields require employees to pick out a target item in the midst of lots of distracting information.

To complicate matters, the targets that are are most important to find – say, a weapon or a malignant tumor – are also incredibly rare.

So, how are we at picking out these kinds of ultra-rare targets? Not very good at all, according to new research from Stephen Mitroff and Adam Biggs of Duke University.

The researchers took advantage of anonymous data provided by an app called “Airport Scanner,” developed by Kedlin Co. The app has players examine simulated X-rays of carry-on luggage in search of various types of contraband, ranging from scissors and water bottles to guns and sticks of dynamite.

Out of the 78 illegal items that players were supposed to look out for, 30 were ultra-rare, appearing less than 0.15% of the time. Players correctly spotted these ultra-rare targets only 27% of the time.

Targets that appeared more than 1% of the time, however, were spotted a whopping 92% of the time.

“The searchers seem to be highly sensitive to the individual frequency rates of the targets, adjusting their search focus such that they de-prioritzed those that very rarely appeared,” says Mitroff. “This situation is what occurs in many real-world searches.”

“For example, a regularly mammography exam has an overall prevalence rate of about 0.5 to 1% (~1/100 cases has cancer present), but any given marker of cancer can appear an order of magnitude less often,” Mitroff explains. “That is, cancer may be present 1% of the time, but the presence of various signs of cancer can be ultra-rare.”

This kind of research is typically difficult to carry out due to the huge number of trials required for testing detection rates of such low-frequency items. The Airport Scanner app, which provided data from about 200 million trials, allowed the researchers to conduct a robust analysis of the ultra-rare item effect.

Mitroff says they now have data from roughly 1.5 billion trials:

“We have replicated this particular effect several times now, but are actively examining other questions with this unique data opportunity.”

You can read more about Mitroff’s research in news stories from Wired, Slate, and FOX News.

Mitroff, S.R., & Biggs, A.T. (2013). The ultra-rare-item effect: Visual search for exceedingly rare items is highly susceptible to error. Psychological Science. DOI: 10.1177/0956797613504221