The Limits of Our Archival Stance (cont.): We Fail to Contribute to Policy Debates

The story so far: I have suggested in previous columns that psychology, perhaps uniquely among the sciences, has adopted the perspective that our task is to discover the basic mechanisms of human memory, emotion, thought and action, and to demonstrate by experiments that we have gotten it right. We demand that the generalizations that find their way into our journals be empirically supported, preferably based on experimentation that allows for clear causal inference. We must not allow anything else to be archived, cited, depended on.

Much has been gained by taking this stance, but costs have been incurred as well. That is, there are consequences that limit the advancement of our own science and our utility to society. Last month I suggested we do not have many outlets for insights that would be published in the form of essays suggesting connections between phenomena that we deal with in separate specializations (“speculative essays” in our somewhat dismissive terms). Certainly many psychologists have more in the way of interesting ideas than can fit into their painstakingly causal research agendas; we rob our collective progress because we do not let those insights out in public.

Other aspects of our paradigm also inflict limits. We believe that, again perhaps unique among the sciences, we are discovering truths of human functioning that transcend culture and context. We are universalistic in our scientific aspirations. For this reason, we found it appropriate, among other things, to study learning in rats and pigeons, and memory with nonsense syllables. Experimentalists were not the only psychologists to take this approach; my own tribe of social psychologists experimented with English-speaking, Anglo- culture respondents and cheerfully generalized the results to all people in all cultures.

Unfortunately, a nasty thing happened on our way to our universal generalizations: culture and context turned out to have a much more fundamental effect on our generalizations than we expected. This has come home to social and cognitive psychologists in the discoveries of cultural psychologists. Apparently, citizens of some cultures do not commit what we call the “fundamental attribution error,” which involves imputing volition to actors such that even actions that they seem compelled to take (rather than actions that are freely chosen) give us some clues about their underlying dispositions. Nor do citizens of other cultures always use the “either-or” logic that characterizes much of our thinking. Evidence accumulates that even brain development is influenced by demands generated by cultural circumstances. Expert reasoning often turns out to be different in kind from the reasoning displayed by novices.

One way of putting this is that there is more plasticity in human functioning that we anticipated. Consequently, it is harder to generalize from “basic processes” to mature, skilled human functioning than we had imagined. It is often the case that “we can’t get there from here,” if “there” is the understanding of human thought and action processes in situ, and “here” is the results of our standard laboratory studies.1

We actually have groups of psychologists on whom we count to make the bridges between basic generalizations and the workings of these basic processes in various contexts. We called them “applied psychologists.” But of course, “applying” was a process to which we gave less prestige than “discovering.” Would you want your daughter to grow up to be an applied researcher, if she could be a pure researcher instead? Hardly.

So we haven’t paid enough attention to what the low-prestige applied researchers were discovering; in our paradigm, the information flow has been unidirectional, from the basic and pure researchers to the applied researchers, not in the other direction.

But think for a minute about the revolution in modern statistical theory that emerged in the late 1940s. During the Second World War, many smart psychologists were drawn to work on personnel selection or signal detection problems. After the war, returning to their academic settings, they had the prestige to push the thinking they developed while working in applied settings into the center of our thinking about measurement issues.

Or think of the more recent flourishing of the field of judgment and decision making. Again, it is due to the insights gained by psychological scientists studying actual people making consequential decisions, and they were at least partially drawn to study this by urgent requests from governments that needed some very practical answers. We have many such examples of important discoveries made in applied settings, examples that call into question our preference for studying context-independent “basic human functioning.”

An even more serious charge I will lay against our archival stance is that it has worked to hamper us in making contributions in the national policy-setting forum. First, I ought to mark one considerable exception to this generalization. Led by Don Campbell, Julian Stanley and Tom Cook, we made one methodologically brilliant innovation for policy research, that of “program evaluation.” We began by pointing out that any intervention program (perhaps an educational innovation or a mental health treatment program) was going to be of sufficient complexity and sufficient distance from the basic scientific findings on which it was based, as to be unsure of working successfully in the real world settings for which it had been intended. Thus we created the large scale experiment, in which the new treatment was introduced into some settings, and in other settings, religiously shielded from contamination by the innovation, the older treatment was continued.2 The treatment was the program, and we were brilliantly clear on the various threats to the internal or the external validity of the demonstration of the utility of the innovation. This has really been a major contribution to social innovation, but notice one ironic thing about it: It flows from our typical archival skeptical stance and our demand for clean causal demonstrations. We entered the policy arena by exporting our central methodological innovation of the double blind experiment!

This stellar example aside, we have often left it to scientists in other disciplines to make clear the applications of our general discoveries on policy-relevant concerns. Behavioral economics rejects the rational choice assumptions of standard economics and replaces them with generalizations about human decision making drawn from experimental research – with prospect theory being an obvious and frequent source. Many economists are aware of the flaws in the rational choice assumptions, and are quite excited by the alternative theory. A leading experimental economist, Matt Rabin, who has recently won the Clark prize in economics, comments that what he does is turn the discoveries of psychologists into the more abstract formulations that economists prefer to work with. As this indicates, it is often experimental economists rather than psychologists who move the research into policy-relevant areas.

A good deal of psychological research shows that people have a hyperbolic discounting function, such that future events are given much less than their due weight in decision-making. But only belatedly have psychologists noticed the implications of this for people’s under-enrollment in voluntary retirement savings plans, for instance, or for teenagers’ decisions to take up smoking, where the costs are likely to be incurred in what seems to them to be old age. More broadly, this has implications for the ongoing debate between allowing people free selection of the short-sighted or future-neglecting choices that they would make when unconstrained, versus a paternalistic intervention into the structure of the choice alternatives they faced now, leaning on them to better provide for the future.

Academics know this. Even after recent drops in the stock market, most of us in academia will experience a reasonably financially comfortable retirement. Why? Because our universities were not so foolish as to give us an unconstrained choice between putting money aside for retirement versus taking it all in the moment. In a world of hyperbolic discounting, “paternalism” has a more attractive character as a social policy.

What we psychologists have learned about human thinking and acting can be made to have policy implications. We have not done a very good job of extracting those implications, and of doing the studies that would demonstrate not the conceptual validity of our findings, but the pragmatic utility of their applications. I’m pleased to note that increasingly, we are turning our attention to these problems. The APS series, Psychological Science in the Public Interest, summarizes psychological findings in policy-relevant ways. At our New Orleans convention, in June 2002, we will have a symposium on claiming a place for psychology in the policy arena.

In a future column, I will examine what characteristics our contributions to policy debates need to have in order to be taken seriously.

1Heaven knows we are not slow. When we saw the need, we figured out how to explore contextualized questions with experimental methods. So we now have ingeniously deployed our experimental approach to, for instance, study chess masters, diagnostic radiologists, citizens of other cultures.

2Medical researchers, concerned with the efficacy of innovative treatments, have also independently contributed to the development of methodologically sophisticated “treatment evaluation” research.