Researchers Explore Life Beyond p < .05

Several APS Fellows are contributors to a special journal issue focused on the overuse of p values in statistical inference in science.

The 43 articles published in the latest issue of The American Statistician, the flagship journal of the American Statistical Association (ASA), distill a variety of opinions and insights on the use of p values in null hypothesis significance testing.

The collection of papers explores limitations of p values, alternatives to p < .05 as the “gold standard” for determining statistical significance, and recommendations for supplementing and replacing p values. All of the articles are open access.

Contributing authors include APS Fellows Ulf Böckenholt and Jennifer Tackett of Northwestern University, Joachim Krueger (Brown University), Geoff Cumming (La Trobe University, Australia), Jessica Utts (University of California, Irvine), Brian Haig (University of Canterbury, New Zealand), and Eric-Jan Wagenmakers (University of Amsterdam). The special issue follows ASA’s 2016 statement encouraging researchers to be wary of statistical claims based on p values alone.

APS Fellow Geoff Cumming and other psychological scientists have long been advocates for alternatives to null-hypothesis significance testing. These alternatives — dubbed the “new statistics” — include estimation based on effect sizes, confidence intervals, and meta-analysis. In an article in Psychological Science, Cumming outlines eight steps that researchers can follow to pursue research questions using a new-statistics approach.

Transitioning from p values to estimation and other techniques will produce a research literature that reflects a cumulative quantitative discipline, Cumming says. To help researchers adopt these practices, he developed a six-part video tutorial, available exclusively from APS.

Authors are also encouraged to use new-statistics practices when submitting to APS journals, including the flagship journal, Psychological Science. This position represents one of several initiatives aimed at advancing transparent and replicable research. In 2017, APS launched a new journal, Advances in Methods and Practices in Psychological Science, with the mission of bridging and integrating conversations on scientific best practices in various areas of psychological science. APS also offers an online methodology center, a one-stop shop for news and practical insights into the latest developments in research methods and practices.

Researchers seeking a hands-on learning experience also have the opportunity to learn new statistical techniques and methodological approaches through workshops at the APS Annual Convention. The workshop offerings at the 2019 Annual Convention in Washington, DC cover a variety of topics, including the new statistics and Bayesian statistics.

Learn more about the APS Annual Convention program, including workshops, invited talks, cross-cutting theme programs, and other special events at
https://www.psychologicalscience.org/conventions/annual

References

Cumming, G. (2013). The new statistics: Why and how. Psychological Science, 25,7-29. doi:10.1177/0956797613504966

Eich, E. (2013). Business not as usual. Psychological Science, 25, 3-6. doi:10.1177/0956797613512465

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (Eds.). (2019). Statistical inference in the 21st century: A world beyond p < 0.05 [Special issue]. The American Statistician, 72(71).