Teaching Scientific Methodology

Although we construct and justify scientific knowledge on the basis of experimental evidence, the way we do this is much more interesting, and much more problematic, than science textbooks suggest. The suggestion of these textbooks that to adopt a scientific method is to adopt a simple routine fails to do justice to the sophisticated skills which scientists use when they experiment and when they reason from evidence.

-Gower, 1997, p. 11

In 1960, F. J. McGuigan published a groundbreaking methodology text that set the mold for practically all subsequent methodology texts in psychology. Prior to McGuigan’s book, methodology texts tended to deal almost exclusively with substantive topics of experimental psychology, for example, perception, learning, and so on. In contrast, McGuigan organized his book around methodological issues in research design and statistics, emphasizing:

The point of departure for this book is the relatively new conception of experimental psychology in terms of methodology, a conception which represents the bringing together of three somewhat distinct aspects of science: experimental methodology, statistics, and philosophy of science. (p. iii)

McGuigan’s approach to research methodology, which was highly innovative in 1960, has become the standard today. Contemporary methodology texts excel in providing what might be called the nuts and bolts of research: experimental design, proper control procedures, statistics, and so forth.

However, one of the three cornerstones of methodology emphasized by McGuigan, the philosophy of science, has tended to be neglected in current texts, and this omission has resulted in a failure to provide students with an adequate understanding of contemporary science. As reflected in the opening quote from Gower’s (1997) book, Scientific Method: An Historical and Philosophical Introduction, this is a general problem that extends beyond psychology.

In point of fact, methodological texts typically provide a view of science that was initially proposed not by scientists but by philosophers of science. Consider a few examples. Up until about 1850, the accepted view of good scientific procedure was to employ induction, a method initially proposed by the philosopher Francis Bacon. Around 1850, the philosopher William Whewell proposed hypothesis testing as a more adequate scientific method than induction. In the mid 1900s, the falsification principle enunciated by the philosopher Karl Popper became popular in science and is often cited in current methodology texts. The impression most methodology texts convey is that science is to be identified with hypothesis testing and falsification of hypotheses, two views that originated as much in philosophic as in scientific practice.

By ignoring the general context in which specific scientific methodologies were, and are, initiated, methodology texts and our teaching fail to adequately provide students with a sufficient methodological background to understand science as it is actually practiced. Moreover, the texts and instructors do little to provide students with the ability to critically evaluate the numerous methodological claims offered by a variety of psychologists and other social scientists who reject science as it is conventionally and, in our view, properly understood. These unconventional approaches, which are in many respects unscientific, go under such names as social constructionism and contextualism and are various expressions of relativism and postmodernism. Some of these unconventional approaches recommend replacing conventional science with such interpretive procedures as narrative, hermeneutics, and dramaturgy (see Capaldi & Proctor, 1999).

We conclude, then, that there are at least two reasons to acquaint students with modern, up-to-date philosophy of science. It will make them better scientists, and it will prepare them to better evaluate the all-to-common antiscientific claims they will encounter in psychology and in the social sciences generally.

TIPS FOR IMPROVING TEACHING ABOUT SCIENCE

The teaching tips that follow are intended to improve the teaching of science by remedying various deficiencies in the description of science that appear in available methodology and related psychology texts. Because methodology is often covered in content courses, the tips that follow apply to them as well as to the methodology course itself. The tips were gleaned from two sources: science as it is currently practiced and contemporary philosophy of science.

EMPHASIZE THAT CONTEMPORARY PHILOSOPHY OF SCIENCE TAKES A NATURALISTIC APPROACH

Naturalism is the view that all statements are to be evaluated as they are in science, empirically. The naturalistic approach to the philosophy of science was first popularized in psychology by Thomas Kuhn (1962), who suggested that science could be best understood by examining the historical record, that is, how scientists actually behaved in practice. This approach has been refined and developed by a number of outstanding philosophers of science, including Ronald Giere, Larry Laudan, Deborah Mayo, and Dudley Shapere, among others. The naturalistic approach suggests that a methodological principle should be evaluated on how useful it is in practice in giving rise to appropriate empirical and theoretical propositions. Prior to naturalism, methodological propositions tended to be evaluated almost exclusively on the basis of their logical/intuitive characteristics (called foundationism), rather than their empirical adequacy.

Recently, significant findings have been produced by psychologists investigating how science is actually practiced by working scientists in laboratory settings (see, e.g., Klahr & Simon, 2001). Various investigators have utilized such techniques as (a) examining the notebooks of practicing scientists, (b) observing the behavior of scientists in the laboratory, and (c) examining scientists’ approaches to problems as a function of how experienced they are (e.g., whether they are novices to the field or senior scientists), and (d) examining the behavior of nonscientists when provided with scientific problems in laboratory settings. As one example, Dunbar (1999) has performed extensive cognitive analyses of laboratory meetings of scientists and has identified key components of scientific thinking that are important in generating new models, modifying old models, and solving difficult problems. These components include analogical reasoning, attention to unexpected findings, experimental design, and distributed reasoning among the group of scientists.

DEMONSTRATE THE INADEQUACY OF THE FOUNDATIONAL APPROACH

Consider two widely accepted methodological principles that teachers might employ to demonstrate that methodological principles that seem intuitively reasonable may be false empirically. One example, widely discussed in a variety of philosophy of science sources, is Popper’s (1959) proposition that researchers should employ rigid tests of their hypotheses with the intention of doing their best to falsify and thus reject them. Contrary to Popper, however, a variety of philosophy of science sources indicate that (a) in practice scientists do not have the primary goal of attempting to falsify their hypotheses, (b) science would be poorer if scientists routinely rejected falsified hypotheses without further effort to rescue them, and, in any case, (c) all theories are already falsified in one or another respect (e.g., Chalmers, 1999; Kuhn, 1970). Thus, one can illustrate to students that Popper’s popular methodological canon, which he admitted is based solely on logic and/or intuition, is incompatible with scientific practice. Popper provides an extreme example of a philosopher of science who emphasizes how science should be practiced as opposed to the view of naturalism, which attempts to determine how science is actually practiced.

As a second example of the inadequacies of the foundational approach, Kuhn (1962) demonstrated that the widely accepted methodological principle that successor theories should explain all that displaced theories explain, plus a variety of new phenomena, has never been observed in practice. Kuhn concluded on the basis of historical evidence that if that methodological principle were observed, no new theory would have ever been introduced in science. This certainly applies in psychology to such movements as behaviorism that, when introduced, were not able to explain many of the phenomena which their rival theories successfully dealt with. Historical evidence on this point is available in several sources, including Kuhn (1962), Donovan, Laudan, and Laudan (1992), and any standard text on the history of psychology.

DEMONSTRATE HOW EVEN INADEQUATE METHODOLOGICAL PRINCIPLES MAY BE EMPLOYED IF BETTER THAN AVAILABLE ALTERNATIVES

We cite two of many possible relevant examples. The first is the contemporary debate over the merits of statistical null hypothesis significance testing. This approach, still widely used, has been roundly criticized by a number of statistical methodologists because it is has certain logical problems. However, Krueger (2001) suggests that it is proper to employ null hypothesis testing, despite its logical problems, because it is useful in practice.

As a second example, Kuhn (1962) has shown that scientists are reluctant to abandon a paradigm that is problematic until it is replaced by a better one. In other words, the point that needs to be conveyed is that methodological and theoretical decisions in science are based on pragmatic considerations. Most notably, one attempts to select the best of the available alternatives. This approach, which itself is a methodological principle, applies to both empirical and theoretical propositions.

To be more specific, it appears that scientists, regardless of field, are reluctant to abandon a theory, and generally will not do so, on the basis of criticism of the theory itself, even if that criticism is accepted as valid. What seems to be required in addition is the availability of a better alternative. Kuhn (1962) put this succinctly, stating that scientists never abandon a paradigm, whatever its problems, until a better one is available. This generalization applies to decisions at any level of science, and the important point to convey to students is that criticism alone is not sufficient to nullify theories and methods.

DISCUSS PROBLEMS ASSOCIATED WITH HYPOTHESIS TESTING

The considerable virtues of hypothesis testing, which are emphasized in methodology texts, hardly need emphasizing here. What needs to be recognized is that hypothesis testing, as valuable as it otherwise is, has several limitations. Students should be informed that it is exceedingly easy to falsify hypotheses in the beginning stages of a research program. They should also become acquainted with the well-known Duhem-Quine thesis, that failure to confirm a hypothesis may be due to a number of factors other than deficiencies in the hypothesis (see Chalmers, 1999). Among the problems are: the apparatus may have malfunctioned, the deduction from the hypothesis may be faulty, and a minor change in some auxiliary assumption might bring the hypothesis into line with experimental results.

An example of rejecting a theory because of a faulty deduction from its premises is available from the psychology of color vision. Ewald Hering’s opponent process theory of color vision, proposed in the late 19th century, was not held in high regard for much of the 20th century because the opponent processes it postulated were incorrectly identified with the photopigments, which showed no evidence of the hypothesized properties. Subsequent evidence revealed that these properties were exhibited by neurons in the visual system.

INFORM STUDENTS OF THE ADVANTAGES AND THE DISADVANTAGES OF INTRODUCING AUXILIARY HYPOTHESES

Introducing auxiliary hypotheses is almost universally portrayed as a flawed strategy, one that seeks in a more or less self-serving manner to avoid disconfirmation of a pet hypothesis. Yet, as Laudan (1996) emphasizes, the introduction of an auxiliary hypothesis that both rescues a theory and increases its explanatory and predictive power is a difficult task, but one that is very often extremely valuable scientifically. Examples of auxiliary hypotheses from physics having both of these characteristics are discussed at some length by Chalmers (1999) and Lakatos (1970).

Within psychology, two examples of auxiliary hypotheses, which not only rescued the theory but improved it, are as follows. The Rescorla-Wagner model, which was originally suggested to deal with classical conditioning, was subsequently extended to apply to human and animal causal learning. Van Hamme and Wasserman (1994) introduced auxiliary hypotheses that allowed that model to be applied to a broader range of causal phenomena, without changing the fundamental character of the model. In the area of attention, Treisman (1960) revised Broadbent’s filter theory from one in which unattended messages were filtered completely prior to identification to one in which unattended messages were only attenuated. This modification allowed the theory to accommodate evidence that meaning of unattended messages sometimes influences performance, without modifying the basic nature of the theory.

DISCUSS ALTERNATIVES TO HYPOTHESIS TESTING

Limitations of space allow only a cursory description of each of these methods (but see Proctor & Capaldi, 2001a, 2001b, for more detailed treatments). The methods given short shrift in current methodology texts include:

• Induction – Generating general statements from particular instances.
• Promise – Advocating a theory based on the perceived likelihood of its solving significant problems better than its competitors.
• Importance – Advocating a theory based on the likelihood that it will solve important problems of practical, philosophical, and theoretical significance.
• Explanatory theory – Developing a theory that attempts to explain existing phenomena but does not make novel predictions at its inception.

As for induction, Newton’s theorizing, at least overtly, eschewed hypothesis testing and emphasized induction, induction being the generally accepted procedure in science until about 1850, at which time hypothesis testing was introduced as an accepted scientific procedure (see, e.g., Laudan, 1996). Regarding promise, Watson (1913) indicated that the evidence for behaviorism was no stronger than for structuralism, but he recommended behaviorism on the basis of its greater promise. This recommendation obviously was accepted within the U.S., as behaviorism quickly became the dominant movement in this country. The tendency to accept theories on the basis of their promise exists not only in psychology but in science generally. For example, Kuhn (1962) showed that scientists from a variety of areas often accept theories on the basis of their promise. Greene (1999) emphasizes importance in his recent interesting book on string theory, in which he gives numerous instances of theories in physics that when originally suggested seemed far-fetched and improbable, but continued to engage scientists because of their potential importance. This activity ultimately proved scientifically rewarding in several instances in the case of development of atomism from Democritus to Dalton. As regards explanatory theories, there were many such theories in science that were originally proposed on the basis of their explanatory capacity that initially did not entail novel predictions but turned out to be highly useful predictive devices. Two notable examples include plate tectonics and Galileo’s acceptance of Copernican theory (Laudan, 1996).

CONVEY THE IDEA TO STUDENTS THAT ACCEPTED METHODOLOGICAL PRINCIPLES MAY BE MODIFIED OR SUPPLANTED BY NEWER METHODOLOGICAL PRINCIPLES

Because methodological statements are empirical statements, it should be expected that they are in a constant state of evolution. After all, science itself, as a clearly recognized activity, is relatively recent. As shown above, some methods important to science at one point in time, for example, induction, are seen as less important at other times. As another example, Popper’s falsification principle, as he proposed it, was once heralded in the philosophy of science as a major important innovation, but it is now seen as flawed and in need of substantial modification. As a final dramatic example, it is not inconceivable that the role of hypothesis testing, considered to be of utmost importance today, may decline in influence in the future due to the discovery of a rival, more adequate methodological procedure.

PROVIDE CONCRETE EXAMPLES OF HOW SCIENTISTS HAVE BEHAVED IN A VARIETY OF CIRCUMSTANCES

It seems useful to provide students with specific examples (case histories and the like) of how particular scientists went about solving significant empirical and theoretical problems. That approach would seem to be as useful with respect to particular subject matter problems as to methodological principles. Fortunately, the various procedures that have been employed to examine science as an empirical activity provide a rich source of material that students can be encouraged to consult. We discussed much of this material previously in the section on taking a naturalistic approach to science. We would especially recommend perusing the excellent case histories provided in Scrutinizing Science: Empirical Studies of Scientific Change, edited by Donovan et al. (1992). For examples from the broader history of science, Kuhn (1962) provides excellent material from a variety of sciences, including psychology.

DEMONSTRATE THE IMPORTANCE OF RESEARCH TRADITIONS IN SCIENCE

In dealing with methodological issues, students should be apprised of the importance of research traditions in psychology and science, such as behaviorism, cognitive psychology, and psychometrics. Research traditions define the kind of entities contained in the world (e.g., stimuli, responses, mental images, mechanisms to process information), and they specify what sort of questions may or may not be asked. As shown by Kuhn, Lakatos, and Laudan, methodological decisions may be seriously affected by the research tradition in which they are embedded. An extended discussion of the role that research traditions play in physics and psychology is to be found in Gholson and Barker (1985).

CONCLUSION

Successful scientific, and thus appropriate psychological, practice depends to some extent on having a proper appreciation of major issues in contemporary science. Of these issues, few are as critical as those involving methodology. Yet, psychology students’ appreciation of methodology does not seem to be as well served by current texts as it might be. In this spirit, the above teaching tips are offered in the hope that following them will improve the science education of psychology students.

REFERENCES & RECOMMENDED READINGS

Capaldi, E. J., & Proctor, R. W. (1999). Contextualism in psychological research? A critical review. Thousand Oaks, CA: Sage.

Chalmers, A. F. (1999). What is this thing called science? (3rd ed.). Indianapolis, IN: Hackett Press.

Donovan, A., Laudan, L., & Laudan, R. (Eds.) (1992). Scrutinizing science: Empirical studies of scientific change. Baltimore: Johns Hopkins University Press.

Dunbar, K. (1999). How scientists build models invivo science as a window on the scientific mind. In L. Magnani, N. J. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 85- 99). New York: Kluwer.

Gholson, B., & Barker, P. (1985). Kuhn, Lakatos, and Laudan: Applications in the history of physics and psychology. American Psychologist, 40, 755-769.

Gower, B. (1997). Scientific method: An historical and philosophical introduction. New York: Routledge.

Greene, B. (1999). The elegant universe. New York: Norton.

Klahr, D., & Simon, H. A. (2001). What have psychologists (and others) discovered about the process of scientific discovery? Current Directions in Psychological Science, 10, 75-79.

Krueger, J. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist, 56, 16-26.

Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press. (revised edition published in 1970)

Kuhn, T. S. (1970). Logic of discovery or psychology of research? In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 1-23). New York: Cambridge University Press.

Lakatos, I. (1970). Falsification and the methodology of scientific research programmes. In. I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 91-196). New York: Cambridge University Press.

Laudan, L. (1996). Beyond positivism and relativism: Theory, method, and evidence. Boulder, CO: Westview Press.

McGuigan, F. J. (1960). Experimental psychology: A methodological approach. Englewood Cliffs, NJ: Prentice-Hall.

Popper, K. R. (1959). The logic of scientific discovery. New York: Basic Books.

Proctor, R. W., & Capaldi, E. J. (2001a). Empirical evaluation and justification of methodologies in psychological science. Psychological Bulletin, 127, 759-772.

Proctor, R. W., & Capaldi, E. J. (2001b). Improving the science education of psychology students: Better teaching of methodology. Teaching of Psychology, 28, 173-181.

Treisman, A. (1960). Contextual cues in selective listening. Quarterly Journal of Experimental Psychology, 12, 242-248.

Van Hamme, L. J., & Wasserman, E. A. (1994). Cue competition in causality judgments: The role of nonpresentation of compound stimulus elements. Learning and Motivation, 25, 127-151.

Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158-177.