Update on Transparency and Open Science in APS Journals

Since its founding, APS has been committed to strengthening scientific rigor and reliability. In 2011, APS made open science an explicit organizational priority, convening discussions with top researchers and organizations about mechanisms for increasing the credibility of published results. Eric Eich, then editor in chief of Psychological Science, proposed the idea of Open Practices badges, and in 2014, the journal began awarding them to recognize authors who made available their data (Open Data badge), materials (Open Materials badge), and preregistered study designs and analysis plans (Preregistered badge). 

In the 8 years since the badges were introduced in the journal, author sharing has steadily increased in all three domains: The percentage of articles earning an Open Data badge and Open Materials badge increased 61% and 42%, respectively, and although no articles earned a Preregistered badge in 2014, more than 4 in 10 received one in 2021. In 2014, 20% of articles earned at least one Open Practices badge; in 2021, 83% did.  

The Open Practices badges were adopted by Clinical Psychological Science in 2016, and Advances in Methods and Practices in Psychological Science has been awarding the badges since its inception in 2017. Thus, all APS titles that publish primarily empirical work currently award these badges to recognize and further incentivize open science practices. 

APS is also an original contributor to and signatory of the Transparency and Openness Promotion (TOP) Guidelines, which were created in 2015 to provide journals and organizations with a concrete and actionable strategy for improving research and publishing practices. The TOP Guidelines comprise eight modular standards, each with three levels of increasing stringency: disclosure, requirement, and verification. Psychological Science adopted the guidelines at Level I, at which authors are asked to disclose information regarding their transparency practices (e.g., whether or not they have made their data available to readers).  

The badges program and encouragement of sharing practices with TOP Level I criteria have resulted in widespread adoption of these practices among authors. In the interest of making these practices more standardized, the APS Open and Transparent Practices Committee (OTPC) recommended in 2021 that APS empirical titles adopt TOP Level II criteria, which would require certain open practices, such as sharing preregistration details with reviewers, allowing exemptions only for legal or ethical reasons.  

Psychological Science Editor in Chief Patricia Bauer worked with the OTPC on a plan for 2022 in which the journal will adopt TOP Level II criteria on five of the eight TOP domains. The journal will continue to encourage (but not require) the sharing of data, analysis/code, and materials and collect information on reasons for not sharing these research components, with the aim of developing a more robust list of valid exemptions before the journal eventually adopts requirements for sharing. Bauer laid out the details and rationale for these changes in her February 2022 editorial, “Psychological Science Stepping Up a Level.”  

Clinical Psychological Science Editor in Chief Jennifer Tackett is working with the OTPC to implement similar changes at that journal in the coming months, at which point all three empirical APS titles will have adopted most or all of the TOP Level II criteria for submissions.  

APS journals have always been on the cutting edge of research findings and methodological breakthroughs. With these changes, they will continue to lead the field in the area of open scholarship as well.

Feedback on this article? Email [email protected] or scroll down to comment.