Back Page: Collaborating With a Crowd

Raphael Silberzahn

University of Sussex researcher Raphael Silberzahn describes how an early setback led him to develop an innovative crowd-sourced research project. This “many analysts” project reveals how research teams can draw different conclusions from the same data set as a result of the choices they make in conducting their analyses.

Before transitioning to academia, you had a career as a business consultant. What led you to pursue graduate studies in behavioral science?

After my undergraduate studies in international business administration I couldn’t wait to try myself out in the real business world, to participate in the action rather than merely learn about it. During my work, however, I noticed that I enjoyed deep reflections about my observations, and that I was quite curious to understand why our global managers and our employees around the world behaved the way they did. I was fortunate to be able to do so at the University of Cambridge.

You coauthored a collaborative commentary that essentially overturned your previously published findings. How did this experience plant the seed for the “many analysts” project?

Science has an aura of perfection, of lasting truth, to it, and I was in the process of learning to become a scientist. During my doctoral studies, I published a paper in Psychological Science about names and career outcomes, which attracted the interest of media and fellow scientists. The findings, however, turned out to be wrong. It was hard to admit so. It turned out to be a test for me and my coauthor Eric Uhlmann, whether we were indeed scientists more interested in truth or in maintaining our own aura of perfection. This paper became the foundation for our interest in crowdsourcing research.

That research involved coordinating many research teams over many months. Why undertake such a large project?

We were delighted about the interest in our project, which combined the work of 65 authors and 29 teams. My background in IT helped me set up processes that allowed for organized contributions from many participants. We worked well as a team and most importantly everyone involved was very responsive and willing to take on tasks.

During the project I was laid off and went through unemployment (going to the unemployment registration was hard), through a separation, through uncertainty about where I’d live and work, and through questions about whether I could remain present in the lives of my children. There were also times when the project got stuck, when we didn’t know how to find statistical experts to evaluate the suitability of different responses (we found experts among the authors), when an author rightly noted that two variables didn’t capture what was intended and shouldn’t be used (each team reran their analysis without the variables).

In the end, we finished our work and got the project published in Advances in Methods and Practices in Psychological Science 4 years after its start. I feel proud about persisting and about the understanding and kind words from coauthors when I was unable to focus on the project.

How has that experience informed your own line of research?

It was a humbling experience. I now publish fewer papers but I aim to get others involved more and get people with neutral eyes to reexamine the data and the conclusions. I feel we live in a world thriving on the newest headline, in a race for surprised eyeballs and short-lived facts. Science benefits from having safe spaces prior to publication to shape and refine novel ideas and double check evidence.