Sample Chapter - Voices of Experience

Successful Empirical Scholarship of Teaching and Student Learning1

Baron Perlman & Lee I. McCann
Yale University
University of Wisconsin Oshkosh

Until relatively recently, pedagogical scholarship was limited to a small subset of educational researchers from colleges of education and a small number of faculty in the different disciplines. Most teaching journals rarely published empirical work, just anecdotes and descriptions of what worked, and was exciting, in the classroom. Over time, a set of accepted “truths,” based primarily on experience rather than empirical data, found their way into the canon of good teaching. This body of “pedagogical wisdom” is neither to be dismissed nor ignored, and to this day it may provide the most insight for the most people about good teaching. It can be found in books such as McKeachie’s Teaching Tips (2002) or anthologies such as Lessons Learned (Perlman, McCann, & McFadden, 1999, 2004).

The clarion call for a change in the academy came in 1990, when Ernest Boyer’s Scholarship Reconsidered: Priorities of the Professoriate called for a redirection of faculty energy into other, more nontraditional types of scholarship — most specifically, scholarship on teaching and learning (SoTL). Boyer’s writing provided the basis for the subsequent drumroll of support, publications, and activities by the Carnegie Foundation and the American Association for Higher Education, and more importantly for the avalanche of faculty attention to SoTL in the subsequent 15 years (Diamond & Adam, 1995; Halpern et al., 1998; Mathie et al., 2004; Richlin, 2001).

Boyer (1990) saw the challenge as being to redefine scholarship to include an emphasis on teaching and student learning, and to end the separation between the educational researcher and the teacher. He broadened the definition of traditional scholarship to include a SoTL that could be considered acceptable research for personnel purposes, and called for more and different ways that faculty could become scholars teaching — something the traditional academy had previously had difficulty accepting. Teaching was to be integrated into, not separated from, the community of scholars (Shulman, 2004). For example, Angelo and Cross (1993) presented a wide variety of ways in which faculty can poll students to assess what the students are learning.

A SoTL was to help faculty take professional responsibility for their teaching (Shulman, 2004) in the face of increased outside attention to higher education by legislators and the public. Faculty needed to demonstrate accountability, and explain and assess their work, before others made them do so. A SoTL also would increase discussion of teaching among faculty, administrators, and others, thereby lessening the isolation of faculty in their teaching, opening classrooms to fresh ideas, and making teaching “community property” (Hutchings, 1993; Shulman, 2004). The SoTL’s greatest success may be in encouraging more talk about teaching (Halpern et al., 1998: Mathie et al., 2004).

To define an acceptable scholarship, Hammack (1997) used a public outcome as the dividing line between self-inspection by the excellent teacher and a SoTL. In other words, a SoTL was more than excellent teaching and more than scholarly teaching (Hutchings & Shulman, 1999; Richlin, 2001). Accountable, professional faculty must have their SoTL assessed and publicly communicated. These processes are reflected in the literature on peer review (e.g., Chism 1999, Keig & Waggoner, 1994), and teaching portfolios (e.g., Hutchings, 1993, 1998; Seldin, 2004), mainstays of the SoTL. Portfolios are often lengthy and always documented, and thus are, to some degree, public.

Problems
There are three major problems with the scholarly nature of the current SoTL. First, many authors describe SoTL (e.g., Cross & Steadman, 1996) in terms of individual classroom research, in which faculty dissect, observe, attend to, get feedback about, and then institute changes in their own teaching. Such an approach can be invaluable in improving teaching and student learning. Much of it, however, violates many of the tenets of the empirical approach to knowing.

For example, there are limits to the reliability and validity of intuition and authority, and teachers may better remember those cases that fit their value system. They may recall a teaching technique that worked well and disregard others that were equally valuable or negative, or they may not recall other uses of this same approach that were less successful. Teachers may too readily accept their own personal judgment or a single episode or story about their own or someone else’s experience in teaching, failing to question its validity or generalizability. They also may justify and rationalize what it is they do when they teach in order to protect themselves from the failures that all teachers experience, or may draw erroneous conclusions about cause and effect.

Scientifically acceptable scholarship follows certain guidelines, regardless of specific methodology. Sampling, generalizability of the data, bias, and the like are critical dimensions that should be considered in any SoTL, empirical or not, if we are to develop a worthwhile canon. Our experience is that faculty who do empirical SoTL do not forfeit the opportunity to also develop valuable perspectives on their teaching. For example, we are currently interested in how lower level courses differ from those at the upper levels. Although we can find nothing in the literature on this question, our colleagues affirm that there are differences, although each has a different idea on the matter. We have begun thinking more critically about our own courses and are learning new criteria and dimensions to apply to their design and to our own teaching of them as we develop a questionnaire for students to complete.

Second, proponents of SoTL (e.g., Boyer, 1990; Glassick, Huber, & Maeroff, 1997) hoped that a SoTL would make teaching more central in personnel decisions. Yet in talking with faculty we conclude that many find it difficult or impossible to have their SoTL respected for evaluative purposes (renewal, tenure, promotion, or pay increases). In part, this struggle may exist because many of those working to fit under the SoTL umbrella never quite dispel the feeling that behavior traditionally associated with the gentleman scholar is a satisfactory component of the SoTL (i.e., to read and know is sufficient as “acceptable” scholarship). This view, and the results it produces, often provides a basis for those who define scholarship traditionally to argue that SoTL is not “real” scholarship. We believe the more empirical approach that we describe below is harder to dismiss.

Third, scholarship must be public and disseminated, something that does not often happen with most SoTL except within one’s home college or university, even if written documentation is provided on the Web (see Seldin, 2004, on placing teaching portfolios on the Web). Such departmental or institutional sharing is a valuable starting point for greater discussion of teaching and improved pedagogy, but may not meet the traditional scholarly criteria of “public-domain work,” such as jury review.

Empirical work has the greatest potential to meet all the criteria of a strong SoTL and be more broadly accepted as “real” research, especially in faculty personnel decisions. Fortunately, empirical inquiry is well suited to psychologists, as it is a methodology with which most of us are well acquainted.

An Empirical SoTL
An empirical SoTL is a way of knowing that is different from the qualitative or anecdotal work so often reported in the SoTL literature, but it never excludes practical application. Whether using experimental, quasi-experimental, correlational, observational, survey, or case-study methodology, it values reliability and validity, and uses quantitative evidence for argument, deduction, and induction. It avoids overgeneralization from one’s findings, emphasizes empirical “truth” as opposed to conjecture or conclusions based on limited or individual observations, and involves an approach that we often think of as “science” or “scientific.”

Whenever possible, an empirical SoTL uses quantitative analysis to describe teaching and student learning and to summarize findings, allows others to confirm conclusions and speculation, and points the way for further work. Private insights into publicly available material are insufficient. For example, when we studied how faculty deal with students who miss exams (Perlman, McCann, Dettlaff, & Palladino, 2003), we developed a ranking of the most common to least common methods and mean faculty opinions on how well each worked.

Principles of Empirical Scholarship of Teaching and Student Learning
What is successful empirical SoTL? Based on talking with others, reading the literature, and our experience with empirical SoTL, we have some suggestions.

Why Collect Data?
All good teachers try to learn how well they are teaching, are concerned with students learning important course material, and make constant adjustments in their teaching. An empirical SoTL assists in these processes and improvements through the gathering and interpretation of numerical data. These data can provide a context for teachers’ decisions, shed light on teaching effectiveness, and help teachers assist their students with their learning.

Getting Ideas
Collect data, rather than merely expressing your opinion (no matter how inspired that opinion may seem). Be alert to topics where everyone “knows” a procedure or particular approach is correct but there are no data to support the accepted opinion. Think about what you do in a typical semester (e.g., different types of final exams and why faculty use one or the other, and how much, how, and when students study for them). What are faculty’s goals in giving comprehensive exams? (Faculty usually argue such exams are a good thing.) How do students view them?

Talk with colleagues. When you hear yourself arguing passionately with a colleague or a colleague arguing passionately with someone else about teaching, pay close attention ¾ you may identify an idea for investigation. Will a little data shed light on the topic? For example, we often lament low levels of student performance in our courses, but what causes such lackluster achievement? If the college degree has mainly become a societal requirement, perhaps many students are being rational if they only want a C average, the minimum to earn the degree. We are in the process of learning what goals students have for grades in our courses and how that is related to the amount of work they do.

Talking with colleagues about teaching is often enlightening. A good example of the value of talking with others is our interest in student attendance. Why do students attend and not attend class? How would one gather valid and reliable data? They may be making good decisions about missing classes if they have a sick child, important assignments due in another course, find class a waste of time, and so forth. Would they have to be paged daily or reminded to e-mail their class attendance and reasons for missing each day? Would such research only be reliable and valid if students had to use an identification card that they swiped in a computer-terminal slot in order to gain class access? In the process of thinking about student attendance we talked with a colleague at a workshop we facilitated. Her proposed solution was simple and elegant. She will teach two sections of the same class each semester, identical except for the students and time of day. She will require student attendance in one section and not in the other, using an AB–BA design, with attendance required for only the morning section one semester and for only the afternoon section the next. The research is quasi-experimental in that she will not randomly assign students to sections. She will look at grades on course assignments (which will be identical in each section) and final grades, and ask students to complete a questionnaire about how attendance being required (or not) affected whether they came or not, the amount of work they did, and what they learned in class.

Talk with students. Students often are surprised and pleased to be able to talk about their education and course experiences with their teachers, and their insights can be powerful and useful. Often the best or most motivated students belong to honors societies and student clubs. Talk with such groups about their student experiences. They have much to say.

Be a student yourself. If you truly want to empathize with your students and experience what they are going through in your courses, attend a presentation in an area in which you know little or nothing and try to take notes. Your appreciation for the students’ role may widen and increase, and all sorts of questions may come to mind.

Go to teaching conferences. Teaching conferences provide an opportunity to focus on the importance and process of pedagogy. It is almost impossible to leave without several ideas on which you could collect data. We talk with colleagues repeatedly at such venues and sometimes it is comforting to learn that some problems seem to beg for good solution. For example, we still have not determined how to make the opening class period of our courses one that communicates information in a way that students retain. After all, students typically have several such class periods that day and their agenda is simply to find out how much work the course requires and, for some students, how well they think they will do. Aside from putting the course syllabus on brightly colored paper or giving a quiz on its content, we do not have any good ideas and we may gather data on what our colleagues do.

Review the literature.Before you undertake a SoTL study, review the relevant literature. Weimer (1993) called this strategy informed practice, placing one’s SoTL within the context and history of what has gone before. In psychology, informed practice means consulting such data-based journals as Teaching of Psychology and Psychology Learning and Teaching. Read psychology’s national taskforces and their reports. Consult the American Psychological Association’s (APA) or the American Psychological Society’s (APS) offices devoted to undergraduate education and teaching.Many books on teaching provide discussion and relevant information on what you want to study. Perlman (in press) provides a list of such books.

This strategy has the additional benefit of assisting you in identifying outlets for your scholarship. The annual meetings of the APA and APS both have teaching-related venues. The National Institute on the Teaching of Psychology held each January in St. Petersburg Beach, Florida provides poster sessions, as do regional teaching conferences. Two Web sites contain a variety of forums you can consider as outlets: the Office of Teaching Resources in Psychology (http://www.lemoyne.edu/OTRP/index.html) and the Society for the Teaching of Psychology (httw://www.lemoyne.edu/teachpsych/div/divindex.html).

Keep a list of research ideas. When you think of a possible research idea write it down and jot down a few lines describing what the idea means. You may find common elements among several of your ideas and be able to blend what seemed like separate studies into a unified project.

Before You Begin: Goal Setting and Adequate Preparation
In Scholarship Assessed, Glassick, Huber, and Maeroff (1997) proposed criteria and procedures for assessing various types of scholarship, including SoTL. We describe them below. One usually thinks of assessment occurring after work has been completed and shared with peers for informal peer review, or as it is being considered for public-domain presentation or publication through a rigorous jury review. Our experience is that these standards also can help as you plan your research.

Have clear goals. Good scholars think clearly and ask good questions. For example, does your proposed SoTL project “show an examination of the assumptions in the teaching of the field … or provide a framework for analysis of the issues, theoretical or otherwise?” (Chism, 1999, p. 104). If you read the results of the exact study you propose to carry out, what would you or your colleagues learn? If everything goes as planned, will your work achieve your goals, add consequentially to the field, and open additional areas for further exploration? Will the results help you or others to become more effective teachers or your students better learners?

Have realistic and achievable objectives. For some of the national surveys and catalog research we have done, the work required time we did not have. However, we accomplished the research in a timely manner by using student assistants. If you want to study the impact of something you are doing when you teach, can you “get at it” in ways that shed light on the issue? A good way to learn if your goals are clear is to try to state clearly the basic purposes of your work. If you cannot do that, you need to think more about what it is that you want to study and communicate to others.

Adequate preparation. All SoTL requires the necessary, practical research skills. Join the Society for the Teaching of Psychology (APA’s Division 2) and learn what services and programs it supports. Martial the resources necessary to move the project forward ¾ paper and printing costs, postage, and envelopes, or financial support (see below).

Appropriate methods. Careful consideration of control groups, sample size, bias, and the like gives research integrity and validity. Always ask colleagues and students to review your methodology prior to data collection, and modify your procedures based on their comments or in response to changing circumstances. It often is helpful to imagine the different ways your results might turn out, and then to consider whether your methodology will provide a strong basis for explaining these outcomes.

Effective presentation and reflective critique. Results must be effectively communicated. Use public-domain jury-review forums when possible; follow APA style and organize your work effectively when presenting it. Use the comments you receive from reviewers to improve the quality of the next manuscript draft or your future work.

Practical Methodological Advice
Pilot test.We always learn a great deal when we pilot test before sampling students, faculty, or both. Questions that seemed clear sometimes need to be reworded. We often get feedback like “what about _____?” and “add more variables to the study.” You need not pilot test on your own students if you are studying something in one of the courses you are teaching, but can use students in your department’s honor society or in other courses. Our experience is that colleagues are happy to fill out a questionnaire and provide (often painful, but gleefully given) feedback on what is unclear or missing. We do not ask this favor too often, however. It also is helpful to explain to them why you are doing the study.

Collaborate with others.When it comes to empirical SoTL, two heads often are better than one. Collaborators can provide more critical thinking and thus better ideas and help when difficult ethical or methodological decisions must be made. Consider working with someone outside your discipline. Their way of viewing problems and going about gathering data may be completely different from yours; the chances increase for a multi-methods project, or collecting data from a broader sample. If you are somewhat uncomfortable with empirical methods, work with someone well versed in them.

Obtain as broad and representative a sample as you can. If you want your research findings to generalize, the broader the sample the better. It is easy to get a good sample of student participants when doing survey research, and there are hundreds of them in various courses in your department. Unless you are studying only majors in a department that has few, you will have no problems obtaining a large sample. At other times, to obtain a sufficient sample you may need to sample majors over two or three years, or have colleagues assist you with using majors at their institutions.

Sampling faculty about their beliefs and teaching practices also is interesting, but it may be difficult to get a good-sized or diverse sample. We have tried to broaden and enlarge faculty samples at conferences, but our experience is that the bystander effect is powerful: Everyone assumes someone else will fill out the questionnaire you have left at the registration table or included in the notebook of conference materials. To get a good sample, one needs to cultivate relationships with faculty at other institutions and ask if they will distribute your questionnaire to their colleagues. When studying curricular issues, our experience is that we get at least a 40-percent response rate from chairs of departments nationally in our discipline, which is sufficient for reliable and valid results.

When gathering data from students, be overly explicit. Define your terms and spell out the criteria to be used. If you do not, it may be impossible to interpret your findings or you may have to start over with a new sample. For example, in doing a course portfolio, one of the authors had a group of students assess different texts. He did not specify criteria, but left them open ¾ a big mistake. He wanted an evaluation of clarity of writing, clearness and boldness of examples, and depth of discussion. The students rated the texts on amounts of summary information, bold print, and pictures, and never mentioned content.

Student and faculty data on the same question are usually fascinating. If you can gather both faculty and student attitudes, beliefs, and perceptions on the same topic (e.g., what makes for a good text?) and compare them, you may have more robust and interesting data. You also may be able to make statistical comparisons between the groups.

Data must be accessible. The goal of empirical SoTL is that the final outcome or results of the work be public domain. This goal has a major impact on how such work is carried out. The rigorous criteria of empirical work and peer review must be met; it often necessitates more work than if the goal is simply to know the answer to a teaching problem. In brief, empirical scholarship is not a process in and of itself, but a process directed toward a public sharing of outcomes. Others must be able to replicate your work. Replication is an important SoTL issue, as all experienced teachers know that students respond differently, sometimes from semester to semester, to the same teaching style. It is essential to know what was done and how it was done, so subsequent studies can be accurately compared to your work.

Variables and the sample must be operationally defined. Systematic study means more than merely proceeding with forethought. For example, studying a small group of faculty or students in a single class or department may suggest powerful insights and solutions to teaching problems. However, if colleagues are to assess the validity of the findings and arguments, the sources of the data must be clearly defined. Small-sample or case-study scholarship may thus be useful as pilot studies for developing methods and hypotheses, but not adequate for public-domain work. Ideally, individual observation of our teaching and student learning takes place in classes or institutions that mirror many others in which our colleagues work. However, since we know that there are important distinctions among institutions and their faculty and students, the limitations of our samples must be acknowledged.

Those who use an empirical approach to the SoTL are concerned when demographics and methodology — which give context to findings and enable replication or comparison — are not reported. The operations or procedures we use define what was done, and limit our conclusions to the situations or types of cases studied.

Confounds must be avoided, or at least acknowledged.A SoTL requires recognition that correlation does not demonstrate causation. The third-variable problem —such as unusually motivated teachers or particular types of institutions and students accounting for outcomes instead of the teaching method under study — can be problematic.

Involve students. Involve students both as coauthors, and in refining and pilot testing your research when its focus will be students. SoTL data sets usually involve descriptive or simple parametric or nonparametric statistics. They can teach students a great deal about nonexperimental research. Invite students into the research from its onset, sharing the question you are trying answer and seeking their ideas. When studying teaching strategies and techniques in a course you are teaching, however, be careful that students’ knowledge of your research goals, and the fact their behavior is under observation or study, does not bias the outcome.

Consider ethical problems. Always overtly consider any ethical problems your research may engender. It is easy to obsess too much about ethical problems, but close attention to them is important. Researchers are responsible for the welfare of their participants and those who engage in a SoTL must protect students, colleagues, and other research participants when doing their work. Whether your goal is to describe, infer relationships, or investigate causality, you must do so in an ethical manner.

Seek fiscal support. Look for fiscal support for your work. Doing so helps you become aware of who in the discipline supports such work, it may get you “free” pre-readings of your proposal, and it will improve the quality of your work by forcing you to think critically about it and formally present it to others. The Society for the Teaching of Psychology awards grants each year. Do not consider only monetary support. Time is a faculty’s most precious resource, and the release of time to do your work should be sought when possible. (We know it is paradoxical that dedicated teachers might teach less in order to do scholarship so they can teach better.) If your SoTL relates to student learning (e.g., general education issues) or behavior (e.g., retention in your institution) perhaps a dean or other administrator will provide funding.

Outcomes
Make your scholarship public. Plan to make your empirical SoTL public at a workshop at your college or university, or in a poster, symposium, presented paper, or a journal article. Consider newsletters and electronic forums, and learn which ones are the most prestigious and respected. If nothing else, such goals may set deadlines for you and keep you working on the project. The feedback you get from making your work part of the public domain improves future SoTL.

Exercise caution in drawing conclusions. Galguera (2002) correctly suggested the possibility of research bias and students giving socially desirable responses when faculty study students in classes they are teaching. Do not over-generalize from your results.

Work backwards. If you want the outcome of your work to be a journal publication, the rigor and nature of the research may differ from a non-jury-reviewed poster or from scholarship done just for yourself or for personnel purposes (e.g., a course portfolio). We have seen many interesting posters at national forums on the teaching of psychology that, because of sample size and methodology, would not be accepted as a journal article.

One study leads to the next. Typically your study will produce unanswered questions, and one study leads to the next. Findings are often counterintuitive or raise other questions. In talking to people about your results, you will probably end up with more ideas than you have time to pursue. If you have too many ideas, or some do not interest you, give them away to colleagues.

Personal Requirements for the SoTL Process
Practitioners of SoTL need to be patient. Good SoTL takes just as much time and critical thinking as any other form of scholarship. Gathering data can take several semesters. You also must be able to tolerate ambiguity and trust your feelings. If what you are studying is interesting and exciting, persevere. If it becomes tedious and boring, think seriously about whether you want to continue. Finally, you must value what you do. You are contributing to your discipline, student learning, faculty development and professionalism, and, one hopes, to your local community and larger society. If you do not value your SoTL, who will?

Implications for Teachers and Students
SoTL often seems to have been defined in practice as faculty studying teaching and student learning in their own courses, mandating a “personal involvement.” Empirical social science methodologies can contribute to more meaningful and deeper SoTL. Successfully practicing an empirical SoTL has several important implications for faculty and their students.

The Disconnect Between Research and Practice Is Lessened
Although research in the past may have been disconnected from the practice of teaching, there is no reason this separation must continue. Empirical SoTL can inform and help improve teaching, and the disconnect between the two seems to be lessening. For example, Chin (1999) reported that more evaluation research is appearing in the journal Teaching Sociology, and Weimer wrote (personal communication, March 16, 2004) that social-science research is more “prized and celebrated in pedagogical periodicals” than in the past.

Empirical scholarship of teaching can build knowledge and improve practice. For example, we recently completed a national study of how psychology undergraduate students practice psychological science (Perlman & McCann, 2005). Teaching psychology as a science has been agreed upon and encouraged by the discipline for the last 50 years, and faculty ranked thinking of and teaching psychology as a science first in importance as an expectation for the undergraduate psychology major (McGovern & Hawks, 1986). Although it is generally agreed that actually doing science is the best way to learn it, no one had ever described how often or in what ways our students learn it. We believe these findings have broad and important implications for how faculty teach psychology across the country, and we hope they will improve pedagogical practice.

Methodological Pluralism Increases Good Teaching and Student Learning
Methodological pluralism is an important component of a strong SoTL (Morreale, Applegate, Wulff, & Sprague, 2002; Wankat, Felder, Smith, & Oreovicz, 2002): “faculty will have to learn and borrow from a wider array of fields and put a larger repertoire of methods behind the scholarship of teaching” (Hutchings & Shulman, 1999, p. 14). Huber and Morreale (2002) supported the value of breadth of methods and, more specifically, empirical social-science methods. We agree with them, but to acknowledge pluralism means just that. Social-science methods are not the only way to understand the nuances and day-to-day complexities of teaching and learning. These subjects are, by definition, not totally amenable to the experimental or survey methods.

Empirical Work Does Not Eliminate a Sense of Wonder and Excitement
 In the SoTL writings, there is emphasis on “the wonderful sense of introspection, reflection, and community [among SoTL practitioners]” (Bilimoria & Fukami, 2002, p. 134) and a concern somehow that these will be destroyed if a SoTL becomes more rigorous in its standards. From personal experience, we know there can be as much excitement and wonder in empirical work as in the creation of an individual teaching or course portfolio. When we replicated four previous studies going back over 60 years (Perlman & McCann, 1999) that looked at the most frequently listed psychology courses in institutional course catalogs nationally and finalized the table comparing current to past data, we experienced a sense of history, continuity, and contribution that is difficult to put into words.

Empirical Scholarship of Teaching Complements a More Personal, QualitativeSoTL
Seitz (2002) provided a good example of a situation in which an empirical SoTL might complement a qualitative approach. In talking of narrative reporting he stated:

Not only does it oversimplify the relationship between pedagogy and student achievement, but it also tends to efface differences among students in any given course, some of whom may well respond positively to a particular teacher or method at the same time that others remain detached, skeptical, and seemingly unaffected by the same approach the teacher’s narrative celebrates. It is these latter, often unmentioned students whom I find myself wondering about whenever I read yet another tale of pedagogical success. (p. 65)

These unmentioned students can be heeded if we gather empirical data to complement qualitative methodology.

Empirical SoTL Yields Insight and Knowledge Although Teaching Is Complex
 Just because complexity exists is not sufficient cause to give up empirical ways of investigating and knowing the effects of our teaching on learning. Such work often builds on the practical, thoughtful problem solving and narratives other colleagues have shared publicly. Because of the methods they use, colleagues who use an empirical approach can be valuable members of the teaching community.

Empirical SoTL May Be More Likely To Be Publicly Known
Everyone who writes about a SoTL agrees that scholarship of teaching must be shared with the teaching community. “The work of the professor becomes consequential only as it is understood by others” (Boyer, 1990, p. 23). We favor carrying such discussion beyond department or institutional groups. The dissemination, when possible, should be to as wide an audience as possible. An empirical approach may provide the greatest opportunity to participate in such public forums.

The more empirical SoTL is carried out and disseminated publicly, the more it benefits all teachers, even those not interested in doing such scholarship themselves. As consumers of such scholarship, they can improve their teaching based on findings important to them and their students. For example, a conference presentation (with data) on the teaching of Introductory Psychology prompted us to change how we teach the course.

Empirical SoTL Is Most Likely To Be Accepted as Scholarship
Publishing in pedagogical journals and writing about teaching are sometimes dismissed as not being legitimate scholarship (Weimer, 1993). We have talked with faculty members who have encountered such attitudes from department chairs, deans, and colleagues. Given the nature of much SoTL, these attitudes may be understandable or even at times deserved. However, the empirical approach is well known and has a long history as accepted scholarship, and may be perceived as more rigorous and systematic than personal anecdote, the “wisdom of practice” or “tips ‘n quips” (Calder, Cutler & Kelly, 2002, p. 49). The repeatedly stated hopes of SoTL authors are that such research will find greater favor over time. We think an empirical component increases the chances of this acceptance.

Although many disciplinary pedagogical journals were not empirical (Weimer, 1993) they may be moving in this direction (Weimer, personal communication, March 16, 2004). Weimer (1993) concluded that most of the writings in pedagogical journals generalizes to other disciplines, with the potential to increase cross-disciplinary discussions. Thus, much of such work in other disciplines could be useful in our own teaching and scholarship. Perhaps we need a journal that publishes the best of empirical SoTL, choosing the best works from a wide variety of disciplines and publishing them in one place.

Conclusion
An empirical SoTL answers questions about (a) what works in the classroom; (b) what is happening in the classroom, discipline, or curricula; and (c) how conceptual frameworks from subdisciplinary areas can be applied to teaching and student learning. It is a strong addition and supplement to other SoTL, is most likely to earn respect as scholarship in one’s college or university, and provides the useful bonus of possible public dissemination in jury reviewed forums and journals.

1We thank the Society for the Teaching of Psychology and the University of Wisconsin Faculty Development Program for support for this work.


References

Angelo. T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for
 college teachers (2nd ed.). San Francisco: Jossey-Bass.


Bilimoria, D., & Fukami, C. (2002). The scholarship of teaching and learning in the management sciences: Disciplinary style and content. In M. T. Huber & S. P. Morreale. (Eds.). Disciplinary styles in the scholarship of teaching and learning: Exploring common ground (pp. 125-142). Washington, DC: American Association for Higher Education.


Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton,
NJ: The Carnegie Foundation for the Advancement of Teaching.


Calder, L., Cutler, W. W. III., & Kelly, T. M. (2002). History lessons: Historians and the
scholarship of teaching and learning. In M. T. Huber & S. P. Morreale (Eds.). Disciplinary styles in the scholarship of teaching and learning: Exploring common ground (pp. 45-67). Washington, DC: American Association for Higher Education.


Chin, J. (1999). Is there a scholarship of teaching in Teaching Sociology? A look at
 papers from 1997-1999. Unpublished manuscript.


Chism, N. V. N. (1999). Peer review of teaching. Bolton, MA: Anker.


Cross, K. P., & Steadman, M. H. (1996). Classroom research: Implementing the
            scholarship of teaching. San Francisco: Jossey-Bass.


Diamond, R. M., & Adam, B. E. (1995). Recognizing faculty work: Reward systems for the year 2000. San Francisco: Jossey-Bass.


Galguera, T. (2002). Too close for comfort and/or validity? In P. Hutchings (Ed.). Ethics
of inquiry: Issues in the scholarship of teaching and learning (pp. 55-64). Menlo Park, CA: Carnegie Publications.


Glassick, C. E., Huber, M. T., & Maeroff, G. I. (1997). Scholarship assessed: Evaluation
of the professoriate. San Francisco: Jossey-Bass.


Halpern, D. F., Smothergill, D. W., Allen, M., Baker, S., Baum, C., Best, D., et al.,
(1998). Scholarship in psychology: A paradigm for the twenty-first century. American Psychologist, 53, 1292-1297.


Hammack, F. M. (1997). Ethical issues in teacher research. Teachers College record, 99,
247-265.


Huber, M. T., & Morreale, S. P. (Eds.). (2002). Disciplinary styles in the scholarship of
teaching and learning: Exploring common ground. Washington, DC: American Association for Higher Education.


Hutchings, P. (1993). Making teaching community property: A menu for peer
collaboration and peer review. Washington, DC: American Association for Higher Education.


Hutchings, P. (1998). The course portfolio: How faculty can examine their teaching to
advance practice and improve student learning. Washington, DC: American Association for Higher Education.


Hutchings, P., & Shulman, L. S. (1999). Change, 31(5), 10-15.


Keig, L., & Waggoner, M. D. (1994). Collaborative peer review: The role of faculty in
improving college teaching. (ASHE-ERIC Higher Education Report No. 2.) Washington, DC: The George Washington University School of Education and Human Development.


Mathie, V. A., Carlson, J. F., Johnson, D. E., Buskist, W., Davis, S. F., & Smith, R. A.
(2004). Expanding the boundaries of scholarship in psychology through teaching, research, service, and administration. Teaching of Psychology, 31, 233-241.


McGovern, T. V., & Hawks, B. K. (1986). The varieties of undergraduate experience.
            Teaching of Psychology, 13, 174-181.


McKeachie, W. J. (2002). McKeachie’s teaching tips (11th ed.). Boston: Houghton
            Mifflin.


Morreale, S. P., Applegate, J. L., Wulff, D. H., & Sprague, J. (2002). The scholarship of
teaching and learning in communication studies, and communication scholarship in the process of teaching and learning. In M. T. Huber & S. P. Morreale (Eds.), Disciplinary styles in the scholarship of teaching and learning: Exploring common ground (pp. 107-123). Washington, DC: American Association for Higher Education.


Perlman, B. (in press). Books to enhance your teaching and academic life. In W. Buskist, V.
W. Hevern, & B. C. Beins (Eds.), Preparing the new psychology professoriate: Helping graduate students become competent teachers. Syracuse, NY: Society for the Teaching of Psychology (Download from http://teachpsych.lemoyne.edu).

Perlman, B., & McCann, L. I. (2005). Undergraduate research experiences in

            psychology: A national study of courses and curricula. Teaching of Psychology,

            32, 5-14..

Perlman, B., & McCann, L. I. (1999). The most frequently listed courses in the
undergraduate psychology curriculum. Teaching of Psychology, 26, 177-182.


Perlman, B., McCann, L. I., Dettlaff, D. M., & Palladino, J. J. (2003). Teacher evaluations
            of make-up exam procedures. Psychology Teaching and Learning, 3, 36-39.


Perlman, B., McCann, L. I., & McFadden, S. L. (Eds.). (1999). Lessons learned:
Practical advice for the teaching of psychology. Washington, DC: American Psychological Society.


Perlman, B., McCann, L. I., & McFadden, S. L. (Eds.). (2004). Lessons learned:
Practical advice for the teaching of psychology (Vol. 2). Washington, DC: American Psychological Society.


Richlin, L. (2001). Scholarly teaching and the scholarship of teaching. In C. Kreber
(Ed.), The scholarship of teaching: New directions for teaching and learning, no. 86 (pp. 57-68). San Francisco: Jossey-Bass.


Seitz, J. E. (2002). From private to public classrooms: “Inadequate” student texts in the
scholarship of teaching and learning. In P. Hutchings (Ed.), Ethics of inquiry: Issues in the scholarship of teaching and learning (p. 65-73). Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching.


Shulman, L. (2004). Teaching as community property: Essays on higher education. San Francisco: Jossey-Bass


Seldin, P. (2004). The teaching portfolio: A practical guide to improved performance and
 promotion/tenure decisions. Bolton, MA: Anker.


Wankat, P. C., Felder, R. M., Smith, K. A., & Oreovicz, F. S. (2002). The scholarship of
 teaching and learning in engineering. In M. T. Huber & S. P. Morreale (Eds.), Disciplinary styles in the scholarship of teaching and learning: Exploring common ground (pp. 217-237). Washington, DC: American Association for Higher Education.


Weimer, M. (1993). The disciplinary journals on pedagogy. Change, 25(6), 44-51.