Entwining Teaching and Research: Creating a Collaborative Review Paper

Two of the many shared goals professors and graduate students have are (a) taking and teaching courses that are integrated with our research and (b) contributing to the field through publications. At the University of Virginia, we recently implemented a model that satisfied both of those goals. A seminar on a laboratory’s main research interests was offered to graduate students, and the group from this course produced a major review paper published in Psychological Bulletin (Lillard et al., 2013a). In this article, we describe the process from both a professor’s and a student’s perspective and focus on what was learned and the model’s possible replication.

Laying Out the Project: The Professor’s Perspective

Every paper begins with an idea. I saw a need for a review paper addressing the issue of whether and how pretend play helps development. The claim is often made, but at least some of the evidence for it is clearly problematic. I wanted to dig more deeply into the literature, so initially I taught an undergraduate seminar. In addition to the usual format — where I picked works for discussion and we all read and commented on them — the undergraduates each also located, read, and added one new experimental article on the benefits of pretend play to a Google Documents database each week. In addition to finding many new articles, the undergraduate seminar made clear the need to explore how pretend play influences cognitive development, language, narrative, theory of mind, social skills, and executive function.

The following semester, I approached six graduate students and offered them an expanded course on whether pretend play helps development, with each student taking responsibility for the topic closest to his or her interests. Our goal was specified at the outset: a Psychological Bulletin paper. An early assignment was to read and discuss an article in which the current editor, Stephen Hinshaw, at the University of California, Berkeley, outlined what he sought for the journal. In addition, I covered philosophical and theoretical background issues in the initial four weeks of class, including articles presenting the strong view that pretend play has a major impact on children’s development.

The students then presented their topics by putting the assigned reading in context (locating other literature through extensive and disciplined electronic searches) and leading a discussion of the important issues. Why would we expect pretend play might (or might not) influence this development? How did it fit with the theoretical viewpoints we had discussed? Did the evidence support the conclusions drawn?

We struggled with several issues. For example, in the beginning it was not entirely clear whether our domain was restricted to pretend play or whether it might include all forms of play. We knew the latter was too big, but the issue of how to refine the domain never went away entirely. We detailed this in the manuscript, but we did so with much deeper consideration than if I had written this article alone without the many thorough discussions with a great group of graduate students. We also took up issues that did not ultimately go in the paper, such as play fighting and mathematics. When articles dealt with more than one outcome, we had to decide how to best present them in the paper.

At the end of the semester, with all of us possessing much richer knowledge about the topic, each student submitted a paper addressing the question of whether pretend play has an impact on his or her assigned topic. Although final for the course, these papers were preliminary for our opus. For the next 12 months, drafts went back and forth — first between myself and each student regarding their specific paper, but then all collated into one manuscript with my introduction and final sections and edited (many times over) to bring one voice. I read or reread all the source articles since I was responsible for the integrity of the entire manuscript. Our reading kept unearthing additional articles, leading to some massive organizational changes, and gradually we reached a point where we thought it was ready.

As submission time neared, authorship order had to be determined. Typically, I would specify this at the start of a collaborative project, but the course format rendered that option awkward as students were on equal footing in the course. Student authorship order was based on quantitative and qualitative contributions from the course’s inception and onward. The quantitative criteria were the number of pages of their original class paper, the number of references in that original paper, and the number of pages and references in his or her section of the final draft of the paper. The qualitative criteria were how much they contributed to the ideas and structure — based on the seminar discussions and exercises through the submitted product — and how much they contributed in terms of feedback on the whole paper, including one another’s sections. Although parts of the ordering were clear, it became difficult to decide whom to put before whom. Happily, the students told me they felt that the second author was clear and that any positions after that were insufficiently different to matter; thus, I used pages as the primary criterion for the others. An alternative approach to this would be to have the class agree at the beginning on a determining principle by which to order authors, but this could change behaviors to fit that principle and lead to competitiveness during the process.

We were ready to submit in early December, but the manuscript was 30 percent over the established page limit; so we conferred on how and where to cut. After a round of ravenous editing — another good collaborative experience — we sent it back again in mid-December. Reviews were returned in late February. We all sat down together to discuss them and how we could respond. This led to more good discussion about theoretical issues of import. The experience of working on this paper together throughout the semester made the process deeper than the usual response to reviews. Everyone had a sense of everyone else’s topic and was clearly the expert on his or her own. We resubmitted the manuscript in early April, and received a provisional acceptance in late May.

The benefits of this project for the students extended further, in that we had the experience of responding to commentaries with a published reply (Lillard et al., 2013b). Creating a constructive reply to several disparate commentaries is a useful exercise. Although this is a rare opportunity with journal articles, one could simulate it in a course by asking students to write commentaries on one another’s course papers and then write replies.

Another benefit from this course was that, given our main conclusion that the pretend play field was in dire need of better research, I wrote a grant proposal out of the project and shared the process of writing that proposal and responding to reviews on it with the students. This was well beyond the scope of the course, but it showed how one continues to use the ideas generated by a paper to move one’s laboratory along. Deeper involvement of the students in this aspect of the process would have been great, but they were working on their own teaching and research at that time. Discussing this process in the course beforehand would have been fruitful.

Further helping the students, invitations for chapters and journal articles arrived in response to our article. Our goal of producing an article from the course had expanded not just to two articles (the main article plus the Reply to Commentaries), but to several articles (with different students taking on different subsequent pieces).

In addition, both the journal editor and the University of Virginia wanted to do a press release, and the students contributed ideas to and observed construction of it and saw how one responds to media outlets that pick up the story. In the future, I would also have students attend the press release interview and listen in on the phone interviews.

The key ingredients making this a successful project were an idea, a group of motivated graduate students with strong research and writing skills, and a topic that lent itself to division into sections that corresponded to the number of students involved. Having a good topic is key to any article, but most of us can think of a good review paper or two we’d like to write. I can’t say that we’ll again be fortunate enough to meet Psychological Bulletin’s criteria, but the model worked well to accomplish our mutual goals.

A Student’s Lessons Learned

Reflecting on my graduate student career, I recently realized that I had started mentally categorizing experiences as subcategories of “Lessons Learned.” One subcategory concerns lessons pertaining to writing a comprehensive review. Over the last two years, this subcategory accumulated content from the project just described. Here, I explore two aspects of this project from the student perspective: article preparation and responding to commentaries.

Review articles represent the gold standard of scholarly contributions, demanding effective synthesis of a large body of literature and a novel thesis that informs future research. Undertaking this as a graduate student seemed unwieldy — perhaps even impossible. The collaboration allayed many concerns because each student drafted just a section of the paper, making the task attainable! When I sat down to write, I did not grapple with pretend play’s impact, or lack thereof, on six forms of cognitive development — I had to write about just one: the one closest to my interests. This niche-based technique not only made the task enjoyable, it also promoted thorough and deep examination. I first traced scholarly contributions in my area, noting dynamics in methods, measures, and findings across time, then merged them and reflected on their implications. However, this did not happen in a vacuum: It was generated through the seminar. Presenting in the seminar provided an opportunity to test my synthesis of the literature. Coauthors could, and often did, identify gaps in reasoning and previously overlooked strengths and weaknesses. As an audience member, class presentations permitted analysis of how others were tackling their similar task with a different topic. Observing the growth of respective sections into a cohesive product was very rewarding.

Among the many important lessons learned, one of the most notable was learning how to respond to commentaries. Rarely in graduate school have I faced such a metacognitive exercise. Much discussion went into not only choosing which points to respond to and which to leave alone, but also finding the most constructive ways to approach those to which we did respond. The commentary process was scholarly progress at its best: a group of passionate researchers debating about issues that most interested them in a high-impact, wide-ranging journal.

In sum, this experience was extremely enriching. I have no doubt the lessons filed away from this experience will serve me well throughout my career.

References

Hinshaw, S. (2009). Editorial. Psychological Bulletin, 135, 511–515.

Lillard, A. S., Lerner, M. D., Hopkins, E. J., Dore, R. A., Smith, E. D., & Palmquist, C. M. (2013a). The impact of pretend play on children’s development: A review of the evidence. Psychological Bulletin, 139, 1–34. doi: 10.1037/a0029321

Lillard, A. S., Hopkins, E. J., Dore, R. A., Palmquist, C. M., Lerner, M. D., & Smith, E. D. (2013b). Concepts and theories, methods, and reasons: Why do the children (pretend) play? Reply to Weisberg, Hirsh-Pasek, and Golinkoff (2013); Bergen (2013); and Walker and Gopnik (2013). Psychological Bulletin, 139, 49–52. doi: 10.1037/a0030521