I was the PSPI Canary

I had the singular privilege of being the team leader for the “juried analysis” published in the inaugural issue of APS’s Psychological Science in the Public Interest (PSPI). Robyn Dawes, John Monahan and I co-authored the report “Psychological Science Can Improve Diagnostic Decisions,” which was published in May 2000. In the spirit of giving psychology away, we set out to show that statistical prediction rules can improve the accuracy, and statistical decision rules can improve the efficacy, of yes-no decisions in many diagnostic fields – including, for example, the prediction of violent behavior, detection of breast cancer and advanced prostate cancer, weather prediction, and detection of cracks in airplane wings. Several fields make routine use of the statistical aids, but in other arenas of critical importance to personal and public health and safety, they are unknown.

I agreed to write the first PSPI report because I was ready for the opportunity to reach a different audience for a line of work I had published before only in medical journals. So I was willing, as PSPI Editorial Board member Carol Tavris characterized it, to be the canary that PSPI sent down into the mine. Now that I have successfully come up for air, I’d like to describe the experience for fellow APS Members. Robert Bjork asked me to put my thoughts in this column, the better to inspire other psychological scientists to participate in the PSPI process.

PSPI Co-Editors Steve Ceci and Bob Bjork were very effective in setting the tone and in working through the myriad details in starting a unique journal. They kindly kept me abreast of developments and solicited my perspective. Without doubt, their greatest contribution was encouragement to round out the writing team with Robyn Dawes and John Monahan.

Among his many contributions, Robyn provided history and sophistication concerning the relevant statistical procedures. His good eye and high standards also gave us the great comfort that what reached final copy would be right and true. In drafting my own sections of the report I found it liberating and reassuring that any mistakes or infelicities of language would not get past Robyn.

John Monahan, in addition to his general contributions and a centerpiece on violence, wrote about such topics as weather prediction, school admissions, and wine-tasting – somehow supplying finished text within a few days of the time a subject was broached. His enthusiasm for the effort and our goals also helped to spur on and invigorate Robyn and me.

Our e-mail traffic, with copies to Bob Bjork, who served as project editor, worked so well that the first PSPI issue came out a year ahead of schedule. Bob applied well-honed skills to help us communicate more effectively to a new kind of audience. He also enlisted reviewers Klaus Fiedler, Elliot Hirshman, and John Wixted, who improved our report significantly.

From APS headquarters, Alan Kraut and Sarah Brookhart deftly guided the entire operation. The editor at Blackwell Publishers for the PSPI series, Roberta Spinosa-Millman, is highly competent and congenial, and helped me undo some faulty copy editing supplied by the printers. She designed the journal, including its handsome cover, on one try.

For some of us, the crowning achievement of PSPI and APS leadership is the journal’s partnership with the Scientific American. John Rennie, editor-in-chief of Scientific American, worked with PSPI and APS to develop a collaboration in which PSPI reports are rewritten for Scientific American’s broad audience. Rennie’s editorial in the October 2000 issue focuses on “Better Decisions through Science,” the rewrite of our report, and on the partnership with PSPI. There he characterized the report as demonstrating that “a fusion of math and psychology can improve decision making” and “strongly recommend[ed] this article to politicians, managers, physicians, educators and anyone else routinely making tough choices.” Of the planned series of Scientific American articles based on PSPI reports, he wrote “Our hope is that these articles will inform political and social discussions to good effect.”

Scientific American’s assistant managing editor Ricki Rusting took responsibility for converting the PSPI version of our report into an article suitable for the magazine. Authors of PSPI reports should be warned that the process of working with a Scientific American editor (Rusting herself, if they are lucky) is an unusual experience.

One day, Ricki called me to say that she and the report authors would share a massive undertaking – all to take place, without fail, within the next two weeks. She would have many questions, so the authors should not stray from their e-mail. She understood every point in the article, oversaw the production of tutorial graphics on signal detection theory and receiver operating characteristics (more elegant than I have managed to produce in many tries), and was assured by her checkers that the citations supported the article’s assertions.

Ricki’s final call was to mention that she had room for six more words and to inquire which of two sentences would better be expanded.

So the canary emerges with a song. The PSPI enterprise is off and running, with hopes intact to affect how the public views the relevance of our science. The second report in the PSPI series was published in November 2000: “The Scientific Status of Projective Techniques,” by Scott O. Lilienfeld, James M. Wood, and Howard N. Garb. This report is undergoing the Scientific American treatment and has already received prominent coverage in The New York Times (February 20, 2001). Other reports are in the pipeline that should also produce considerable public interest and stimulate debate.

APS members who are fortunate enough to be involved in the PSPI process – as authors, reviewers, consultants, researchers who propose topics for consideration to the Editorial Board, or individuals who bring the published PSPI reports to the attention of others who could use the information – can be confident that they are contributing to an important endeavor for psychological science.