Replication Education

Replications are not only one key component of the scientific method, they are also an effective pedagogical tool.

With this in mind, we recently launched the Collaboration Replications and Education Project (CREP; rhymes with grape) to facilitate student research training while at the same time solidifying psychological research findings. Ideally, replication projects will give students the opportunity to learn how to do research by replicating important findings in psychology, while also immediately contributing to a database of results surrounding what the field thinks is important in explaining the human psyche.

The CREP provides instructors with a straightforward starting point for class projects and independent studies focused on replication. It includes a clear protocol and a large advisory board with experts on both teaching and researching psychology that can support methods educators. Importantly, replication studies let students focus on learning the mechanics of research itself, not whether an outcome is significant.

From the perspective of outside researchers, the CREP has the potential to provide a vast array of data for meta-analyses and other research syntheses. So beyond helping the students learn how to conduct research, it will also help researchers understand theories in psychology. For people whose research is being replicated, we have procedures in place to encourage high-quality replications.

In the first phase of our project, we compiled a list of current studies and invited students and instructors to replicate them as part of research methods courses, independent studies, bachelor theses, and the like. We selected eight recent, highly cited studies that are feasible for bachelor students to replicate from some of psychology’s top journals covering several subdisciplines. We selected these studies based on their importance in psychology, because they explain something about the mind. These studies represent well-cited, but also recent, empirical papers in psychological science.

In the finalization of the project, we encourage the CREP teams to submit their results to a centralized location and write them up for publication. The results can be further analyzed by interested researchers and experts.

Procedures

CREP contributors (students or other researchers) obtain approval from their home institutional review boards and seek approval to make the data publicly available when possible. Upon completion of the replication study, contributors share statistics from the project necessary to complete meta-analyses (e.g., effect sizes). Contributors who complete all the CREP steps can earn small research awards (ranging from $300 to $500) sponsored by Psi Chi and the Center for Open Science.

Any contributor can take the lead in completing an associated research report of the project — i.e., summarizing the total effect across replication attempts, searching for moderators of the effect size (e.g., is it stronger in Europe or the United States), etc. If no student contributor or faculty mentor wishes to write up the results for publication, the CREP members will do so.

The project center is on the Open Science Framework so that contributors can benefit from easy access to helpful resources such as IRB proposal guidance, materials, or power analyses from the different advisory board members.

Research Methods and Capstone instructors are invited to use the studies list in their classes as an option for their students. Psi Chi members at any educational level are also invited to participate. Groups of students may wish to do so as a chapter activity, or single students may wish to do so for a class research project. Students will need to find a faculty sponsor for their projects. Did we also mention that upon completion of a project, students can receive between $300 and $500 as a research award?

Goals

In the short term, we hope to spread the list of studies to be replicated, gather the relevant experimental materials needed to replicate the studies, establish an online hub for CREP using the OSF, and create a system for gathering information about the replication attempts as they are completed.

Over the course of multiple years, we hope to facilitate the research training of psychology students by encouraging replication projects and collecting data about the success (or failures) of the individual projects for use in meta-analyses and other research. In so doing, we now invite researchers and teachers to submit research findings that they feel are of crucial importance to the field, which they can do at www.tinyurl.com/mhyu87n. We hope replications will become a habit of psychology education and research. œ

For information about CREP, visit www.osf.io/wfc6u/.

Current Recommended Studies for Replication

Griskevicius, V., Tybur, J. M., & Van den Bergh, B. (2010). Going green to be seen: Status, reputation, and conspicuous conservation. Journal of Personality and Social Psychology, 98, 392–404. Study 1

Elliot, A. J., Niesta Kayser, D., Greitemeyer, T., Lichtenfeld, S., Gramzow, R. H., Maier, M. A., & Liu, H. (2010). Red, rank, and romance in women viewing men. Journal of Experimental Psychology: General, 139, 399. Study 3

Back, M. D., Stopfer, J. M., Vazire, S., Gaddis, S., Schmukle, S. C., Egloff, B., & Gosling, S. D. (2010). Facebook profiles reflect actual personality, not self-idealization. Psychological Science, 21, 372–374. Study 1

Diener, E., Ng, W., Harter, J., & Arora, R. (2010). Wealth and happiness across the world: Material prosperity predicts life evaluation, whereas psychosocial prosperity predicts positive feeling. Journal of Personality and Social Psychology, 99, 52. Study 1

Mazar, N., & Zhong, C. B. (2010). Do green products make us better people? Psychological Science, 21, 494–498. Study 2

Weinstein, N., & Ryan, R. M. (2010). When helping helps: Autonomous motivation for prosocial behavior and its influence on well-being for the helper and recipient. Journal of Personality and Social Psychology, 98, 222. Study 4

Kool, W., McGuire, J. T., Rosen, Z. B., & Botvinick, M. M. (2010). Decision making and the avoidance of cognitive demand. Journal of Experimental Psychology. General, 139, 665. Study 1

Kiefer, M., & Martens, U. (2010). Attentional sensitization of unconscious cognition: Task sets modulate subsequent masked semantic priming. Journal of Experimental Psychology: General, 139, 464. Study 1

Recommended Reading

Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F., Geller, J., Giner-Sorolla, R. S., … van’t Veer, A. (2014). The Replication Recipe: What makes for a convincing replication? Journal of Experimental Social Psychology. Available at SSRN: http://ssrn.com/abstract=2283856 or http://dx.doi.org/10.2139/ssrn.2283856