The Verdict Is In: Courtrooms Seldom Overrule Bad Science

In television crime dramas, savvy lawyers are able to overcome improbable odds to win their cases by presenting seemingly iron-clad scientific evidence. In real-world courtrooms, however, the quality of scientific testimony can vary wildly, making it difficult for judges and juries to distinguish between solid research and so-called junk science.

This is true for all scientific disciplines, including psychological science, which plays an important role in assessing such critical pieces of testimony as eyewitness accounts, witness recall, and the psychological features of defendants and litigants.

A new, multiyear study published in Psychological Science in the Public Interest (PSPI), a journal of the Association for Psychological Science (APS), finds that only 40% of the psychological assessment tools used in courts have been favorably rated by experts. Even so, lawyers rarely challenge their conclusions, and when they do, only one third of those challenges are successful.

“Although courts are required to screen out junk science, legal challenges related to psychological-assessment evidence are rare,” said Tess M.S. Neal of Arizona State University, one of the authors of the report. The other authors are Michael J. Saks of Arizona State University, Christopher Slobogin of Vanderbilt University Law School, David Faigman of the University of California Hastings School of Law, and Kurt F. Geisinger of the University of Nebraska-Lincoln.

“Although some psychological assessments used in court have strong scientific validity, many do not. Unfortunately, the courts do not appear to be calibrated to the strength of the psychological-assessment evidence,” said Neal.

The new APS report examines more than 360 psychological assessment tools that have been used in legal cases, along with 372 legal cases from across all state and federal courts in the United States during the calendar years 2016, 2017, and 2018.

These findings are also presented at the 2020 American Association for the Advancement of Science (AAAS) meeting in Seattle.

Psychological scientists provide expert evidence in a variety of court proceedings, ranging from custody disputes to disability claims to criminal cases. In developing their expert evaluation of, for example, a defendant’s competence to stand trial or a parent’s fitness for child custody, they may use tools that measure personality, intelligence, mental health, social functioning, and other psychological features. A number of federal court decisions and rules give judges the latitude to gauge the admissibility of evidence, largely by evaluating its empirical validity and its acceptance within the scientific community.

For their review, Neal and her colleagues gathered results from 22 surveys of psychologists who serve as forensic experts in legal cases. They reviewed the 364 psychological assessment tools that the respondents reported having used in providing expert evidence. They found that nearly all of those tools have been subjected to scientific testing, but only about 67 percent are generally accepted by the psychological community at large. What’s more, only 40% of the tools have generally favorable reviews in handbooks and other sources of information about psychological tests.  

The scientists also found that legal challenges to the admission of assessment evidence are rare, occurring in only about 5% of cases they reviewed. And only a third of those challenges succeeded.

According to the report: “Attorneys rarely challenge psychological expert assessment evidence, and when they do, judges often fail to exercise the scrutiny required by law.”

In an accompanying commentary, David DeMatteo, Sarah Fishel, and Aislinn Tansey, psychology and legal scholars at Drexel University, call for more research on whether trial court judges are functioning as effective gatekeepers for expert testimony. They point to studies indicating that many judges admit evidence from methodologically flawed studies and others that show attorneys and jurors lack the scientific literacy necessary to scrutinize scientific evidence. The Drexel scholars also called on forensic psychologists to ensure they use scientifically sound assessment tools when providing expert evaluations in legal settings.

# # #

APS is the leading international organization dedicated to advancing scientific psychology across disciplinary and geographic borders. Our members provide a richer understanding of the world through their research, teaching, and application of psychological science. We are passionate about supporting psychological scientists in these pursuits, which we do by sharing cutting-edge research across all areas of the field through our journals and conventions; promoting the integration of scientific perspectives within psychological science and with related disciplines; fostering global connections among our members; engaging the public with our research to promote broader understanding and awareness of psychological science; and advocating for increased support for psychological science in the public policy arena.

Published three times per year by the Association for Psychological Science, Psychological Science in the Public Interest (PSPI) is a unique journal featuring comprehensive and compelling reviews of issues that are of direct relevance to the general public. These reviews are written by blue-ribbon teams of specialists representing a range of viewpoints and are intended to assess the current state-of-the-science with regard to the topic. Among other things, PSPI reports have challenged the validity of the Rorschach and other projective tests, have explored opioid addiction, how to keep the aging brain sharp, and have documented problems with the current state of clinical psychology. All PSPI reports are freely available to the public via the APS website. This journal is a member of the Committee on Publication Ethics (COPE).

For a copy of this article, contact [email protected].

For more information about this study, please contact Tess Neal at [email protected].