Registered Replication Reports at AMPPS

Instructions for Authors

Registered Replication Reports (RRRs) combine the results of multiple independently conducted direct replications of an important original study, all of which follow a vetted, accurate protocol. AMPPS seeks proposals for such RRR projects.

All RRRs are a unique form of a Registered Report (see: https://www.nature.com/articles/s41562-021-01193-7). For RRRs, AMPPS presently only considers proposals that involved multiple laboratories.

In advance of the submission, we encourage authors to consider the following parameters. We also encourage authors to check-in with the Editor ([email protected]) about these parameters in advance of the submission.

We encourage authors to consider the value of their RRR. Replication value and suitability of a study for an RRR reflects whether:

(a) The study has been influential in the field

(b) The study is methodologically sound and more recent methods have not superseded those in the original study

(c) The interpretation of the result is unambiguous, and the findings either force a reconsideration of an important theory or establish the foundation for a theoretical position

(d) The study has not yet been the subject of published replications, or published replications have yielded inconsistent estimates of the size of the effect, leading to uncertainty about the size of the effect

(e) Theories of or empirical understanding of the effect would benefit from a more precise estimate of the effect’s size

(f) Other labs likely would be able to join the replication effort (the authors are encouraged to identify other laboratories known to be interested in contributing)

(g) The original study and replications of it would be of interest to the broad readership of AMPPS

To address these issues, you may consider working through our RRR Proposal Worksheet, and using this form in consultation with the journal.

RRR submissions are reviewed in a manner that is identical to how the journal handles RR submissions. The review process for a Registered Report occurs before data collection. The review process focuses on whether the planned study is well motivated and well designed. Reviewers will be asked to evaluate whether the study outcome will be theoretically and empirically meaningful regardless of the outcome. They will also be asked to identify possible factors that could undermine the usefulness of the study (e.g., unreliable measures or manipulations, ceiling or floor effects, necessary manipulation checks). In some cases, the review process might suggest necessary pilot work or preliminary testing before the study design can be pre-approved. If accepted, the submission will be provisionally accepted. And, assuming the study meets the prespecified conditions, the eventual article will be published regardless of the study outcome. The post-data manuscript will be reviewed for clarity and for constructive comments about possible additional exploratory analyses, but acceptance will not be contingent on the outcome of the study or any additional tests.

We encourage authors of RRRs to submit their Stage 1 report with a commitment from at least 10 different labs who are willing to join the data collection effort. If the paper is accepted at Stage 1 (an In Principle Acceptance, IPA), we will then encourage authors to make a general recruitment announcement solicit the involvement of other laboratories. The minimal sample size will be determined as part of the Stage-1 review, and that Stage 2 review will not comment until enough independent labs are recruited to achieve the stated sample side.

Once data collection is complete, the authors will analyze the data according to the Stage 1 plan, incorporate the data into the Stage 1 manuscript, and add a brief discussion section. This Stage 2 manuscript is a draft of the final RRR paper. The authors submit this Stage 2 manuscript as a revision to the provisionally accepted Stage 1 manuscript. This Stage 2 manuscript is again sent for review to the original authors, previous reviewers, and possibly additional reviewers. The purpose of this stage of review is to evaluate whether the study adhered to the plan of the Stage 1 manuscript and to ensure that the conclusions are supported by the evidence. Reviewers may suggest additional exploratory analyses. The revision process will continue until the paper can be accepted.