A Self-Correcting Approach to Multiple Choice Tests

When you were a student, did you have any of the following experiences?

You dreaded taking multiple-choice tests.

You thought of the right answer after you had handed in the test.

You never went over the exam questions in much detail when it was returned.

As an instructor, have you had any of these experiences?

Students complained they were not good at taking multiple- choice tests.

The average class grade on an exam was much lower than anticipated.

Students did not understand fully material they just studied and it was relevant to upcoming class work.

You are not alone. The bright, engaging, and committed students I teach repeatedly declare that they hate multiple-choice tests and suspect that instructors design them just to trick students. I had the same attitudes as a student. Nevertheless, during my 15 years of teaching I have used multiple-choice tests extensively despite their “dubious” reputation. However, by appreciating students’ concerns, considering the goals of my tests, and thinking more broadly about the test-taking experience, I believe I have learned some valuable lessons about how to enhance the learning potential of my multiple-choice tests. Certainly, designing good questions is an important first step. Equally, if not more importantly, it is vital to consider what you wish to accomplish with your tests and how your testing format can help achieve your goals. I view tests as an opportunity for learning. Thus, I have developed a multiple-choice testing format fueled by observations that learning is facilitated when students are given opportunities to practice for exams (Balch, 1998), when they are encouraged to thoroughly consider alternative answer options (McClain, 1983), and when they are given immediate feedback about their performance (Friedman, 2002).

I use standard multiple-choice tests in my classes. However, instead of coming to class, taking the exam, and waiting until it is graded to see how they did, students engage in what I call self-correcting. Students come to class prepared as they would be for any other multiple-choice exam, take the exam, and then they take it home and review each question to assess whether their answer was indeed the best one. Students can use class notes, readings, and even discuss the questions with their classmates (indeed such collaboration is encouraged). As they do so, they can change their answers. Students return exams during the next class period and the self-corrected version determines their final grade, as follows. For each correct answer (no change) students receive full credit. For each corrected answer (wrong to right), students receive half-credit. Incorrect answers — originally wrong and unchanged, or changed to wrong — receive no credit.

I have observed that students do not just perform better than with a more traditional testing format, but they also learn better. Why? Students no longer see tests as teaching weapons but as learning tools. Certainly students are still concerned about their performance but they feel more in control of the experience and its outcome. As well, they have the opportunity to reflect on the material and correct misunderstandings, making learning subsequent material more efficient. Students also learn how to take multiple-choice tests more effectively and, because tests now require using a variety of skills, they appeal to students with diverse learning styles.

Frequently Asked Questions and Answers
Surely this self-correcting approach invites questions. Here are some frequently asked questions along with my responses.

Can’t students just change their answers to all the right ones and not show any corrections?

No. Students know that I have a copy of their original answers so I can track any changes they make.

Couldn’t students simply take answers from other students without participating in any constructive discussion? Doesn’t this set the stage for more extreme cheating schemes?

Sure. Although students are forewarned that their minds and grades will benefit the most by approaching the test as advised, some will attempt a bit of “social loafing”. Some may even contemplate ways to “beat the system”. However, I am open with students about the advantages and disadvantages of their test-taking attitudes and strategies, and I hope our discussions provide an opportunity for another level of learning. It is my opinion that cheating behaviors reflect situational and interpersonal demands of the classroom context and are not simply the mark of corrupt or lazy students. Thus, if students are resorting to serious and systematic cheating, then there likely is a bigger issue of concern in the classroom that warrants attention. Moreover, invariably a case arises when a student took a friend’s word about an answer that turned out to be wrong, which in turn gave the student pause about taking advice blindly in the future. This is a good lesson to learn.

Don’t students study less?

I often have students take a second class with me and ask them to describe their experience with my exams and give advice to new students. Consistently, their advice is to study as best they can for the exams because they are challenging and they need to be prepared. Some students report studying less than usual for the first exam but then realize that the boost that comes from self-correcting is built on starting off on solid ground. Thus, they report putting in more effort for subsequent tests.

Aren’t your grades inflated?

No. The means of my exams are a bit higher than in previous classes but not so much that they would be considered inflated. Moreover, there is always a good range of scores. Also, exams generally count for no more than 60 percent of the final grade and so grades become refined and distributed through other class requirements. In comparison to other courses, I often award a lower percentage of As (yet I have very high course evaluations with approving comments about the testing format). Thus, the format does not appear to inflate grades. It does seem to improve morale and attitudes toward tests.

Why not just give practice tests?

I think the reflective experience and the opportunity to personally correct errors that this format affords is an important element of the learning process.

Are all of the exams self-correcting?

I typically give three exams. The first two are self-correcting. The final exam is not and it is cumulative.

Why not have the final be self-correcting?

This testing format is directed toward helping students refine their working knowledge and learn how to approach a test to assess that knowledge. I think the cumulative final helps to further these goals by giving students an opportunity to “walk on their own.” Besides, getting a corrected version back before final grades must be assigned would be a problem. In addition, the final cumulative exam can serve as an evaluation tool.

What are the downsides of the format?

There are two time-related challenges. I give students back their exams after they have been officially corrected. They are encouraged to use them to study for their cumulative final. This means that prior exams are in circulation and new students may use them. Although I have not seen direct evidence of this, I initially needed to spend extra time generating new questions. However, I now have a large pool of questions from which I can generate exams with ease.

I have teaching assistants who correct exams in classes that average about 60 students, and it takes about 2 hours to complete the scoring. Instructors with larger classes without assistants may find a manual correcting process difficult, especially if it is used in multiple classes. Perhaps an innovative instructor will discover a technological solution.

Does this format work better for some students than others?

This is an interesting question about which I wish I had empirical data. Research suggests that “A” students work through question options more than “C” and “F” students (McClain, 1983), and so I would predict that students who may otherwise perform poorly would benefit the most. However, I often find “A” students are very anxious and report that this format relieves some tension so they have more of an opportunity to “digest” rather than to “regurgitate” information. Thus, I would like to believe this format offers something of benefit to all students.

If you have read to this point, you are hopefully now interested in seeing how the process operates in more detail. Described below are a few suggestions for writing multiple-choice questions, how to implement this format, and some advice about how to approach this testing experience.

Writing Good Questions
There are a number of excellent resources for more detailed information about how to construct good multiple-choices tests (e.g., Clegg & Cashin, 1986; Renner & Renner, 1999; Sechrest, Kihlstrom, & Bootzin, 1999). I have found the following suggestions especially useful.

• Write questions that make use of a variety of thinking skills. The taxonomy suggested by Bloom (1956) and presented in Table 1 is a useful guide.
• Write questions that reach for more complex thought processes as often as possible. Include words that point to the kind of thinking you expect the question to utilize.
• Present the problem in the simplest form with precision and clarity, and the options in the simplest form possible. Do not repeat material unnecessarily.
• Do not include unnecessary descriptive material in questions and answers. It does not contribute to interest or complexity, and it just adds confusion and reading time.
• Use options such as all of the above or none of the above with caution. Avoid implausible or absurd distracters in the options. They make questions easier to answer simply because the option can be easily rejected. However, an amusing or entertaining option used sparingly can break tension and remind students about the constructive goal of the test.
• Avoid giving irrelevant clues to the answers such as length of clue (longest is often the correct option) or inconsistencies in grammar. Avoid the use of specific determiners (all, never, and always suggest incorrect options whereas usually, sometimes, typically, and maybe suggest correct options).
• If a question contains controversial material, cite the author whose opinion is stated. For example, “according to class lecture…” or “as stated by Rosenthal…” This citation helps reduce subjective interpretations.

Implementing the Self-Correcting Format
Students should be given an exam with instructions and questions similar to those below with a separate numbered answer sheet. Be sure names are recorded on all materials.

Sample Instructions
Welcome. Keep in mind that exams in our class are treated as opportunities for learning. To this end, complete this exam by circling the best answer to each question on your exam. Do not skip any questions. Also, record your answers on the separate sheet that has been provided and hand it in before you leave. When you are done, take your exam home and review each question using your notes, text, and discussions with classmates (be sure to consider each answer option thoroughly). If you wish to change an answer, indicate it in the space provided on the exam. Use this space only for changed responses. Return your corrected exam at the next class meeting. Late exams lose the self-correcting option.

Sample Question Format
A researcher interested in the impact of story telling on the development of social norms asks a fifth grade teacher to read stories to his class involving cooperation. Another teacher reads stories involving competition. The researcher then codes the amount of sharing children from each class displayed during their lunch periods. In order to avoid experimenter bias in the coding of behaviors, what should be done?

1. The children from each class should wear different colored tee shirts.
2. The children should be observed over the course of several lunch periods.
3. A researcher unfamiliar with the manipulation should record the behaviors.
4. The researcher should read the stories to the children in each class.

Corrected answer: ____________

Advice for the Instructors
Listed below are points to keep in mind when using this format in your class.

Take a Few Trial Runs
At the same time, take seriously the need to think the format out fully to avoid confusion and frustration. It might be useful to pilot test the format on a quiz rather than a lengthy exam.

Expect Students to be More Willing to Voice Their Reactions to the Test
When I enter the classroom on the day exams are due, students are always chattering about the test. I purposefully acknowledge the energy – even tension – in the room to remind students that the test was designed to promote this activity. Some will ask to discuss certain questions. I ask them to make note of the questions and after exams are handed in I address ones about which several students have concerns. Each instructor will need to decide how to gauge these discussions, bearing in mind the importance of modeling constructive dialogue. I keep discussions short and focused. Time for additional discussion is offered after class or via e-mail. Students also sometimes include written comments on their exams. These comments have been invaluable in helping me craft better questions.

Describe Your Exam Philosophy in Your Syllabus
Explain your testing philosophy to students on the first day of class and how it is reflected in your exam format. The notion that tests can be opportunities for learning will be a novel perspective and you can use it to generate discussion and interaction. You might begin by asking who does and does not like multiple-choice exams, and why they do or do not. Drawing on their responses, explain why you selected this new format. Students do not need to know all of the procedural details at this point, a preview should be sufficient. A short time before the first test go over the testing procedures and how students should prepare for the exam.

Do not Make Tests and Questions More Difficult than Usual
Keep in mind the goal of your test as a learning tool as you decide what material to include.

Learn to Write Good Multiple-Choice Tests
If your tests are learning tools, they should represent the material to be learned and ask questions that encourage students to use a range of thinking skills.

Sometimes Students will Change an Answer from Correct to Incorrect and They Will Want to Know What Happens if They Do This
This is typically treated as a wrong answer and no credit is given. It has been my experience that this happens for one of three reasons:

1. The question was “bad” and several students got it wrong in this manner. If this appears to be the case you can drop the question or accept the answers that make sense with hindsight.
2. Students simply listened to their friends and changed their answers without further scrutiny. In this case, students need to be reminded how to approach reviewing their answers and relying on more reliable sources.
3. A student reads too much into the question. Sometimes students get questions wrong for the right reasons because the instructor failed to realize how a question might be perceived or interpreted by someone with a different level of knowledge. These instances can be dealt with on a caseby- case basis and the instructor’s assessment of the student’s knowledge of the topic and question at hand. This is a benefit because you get to know your students better as learners.

Students Should be Told in Advance How these Details would be Handled
Given the opportunity this format provides for students to obtain a better grade through self-reflection and revision, typical complaints about the wording of questions, trick questions, lack of information, and the like are often significantly reduced. Moreover, when I do encounter debates about questions (which will happen), I remind students that this type of involvement was something the test intended to inspire. Having students more actively in charge of their learning experience is uplifting for them as well as the instructor.

Look at Students’ Exams and Answer Sheets Before They Leave Class
A few students will fail to record their answers on both the exam and the answer sheet. It is important that all materials are completed.

Return Exams so Students Can Go Over Corrections
It is important that students have the opportunity to review their exams after they have been formally corrected. I give exams back with incorrect and partial credit answers marked as well as an answer sheet with correct answers to the entire exam. If you give a cumulative final it will be helpful for students to use previous exams as a study tool. Suggest to students that they mentally rewrite questions so that they point to different answers as a way to test themselves. Also point out that it is not sufficient for students to merely learn what the right answer was. They need to know why they were wrong so that when they see a different question on the same topic they will be able to answer it correctly.

Advice for Students
For optimum effectiveness, it is important to inform students about how to approach and manage the testing format. Emphasize these points.

Tests are Another Opportunity for Learning. Take Full Advantage of Them.
Tests can help you identify your strengths and weaknesses in the learning context. Try to reframe your perspective on taking tests by approaching this new format with a serious and open attitude. Use this opportunity to take a closer look at how you handle tests — what you study, how you organize time, how you answer questions — and what skills you can polish for future use.

Study as You Would for any Multiple-Choice Test
Do not put too little effort into preparing for the test because you have an opportunity to review and revise your answers.

Do Not Answer a Question Hastily with the Expectation that You can Change it Later
If you want a good grade you need to get full credit for as many questions as you can. This means trying your best to work through the question accurately the first time.

Consensus Does Not Insure Accuracy
If you are working with classmates and they say a particular answer is correct, but it is different from your answer, be sure they have satisfactorily justified their choice and that you understand and accept fully the reason.

Conclusion
You may think of issues or concerns I have overlooked in designing the testing format I have described. If so, I hope you will share your thoughts with me. I also hope you will share with me the experience of developing other new and novel ways to enhance the potential for learning in your classes.

References

• Balch, W. R. (1998). Practice versus review exams and final exam performance. Teaching of Psychology, 25, 181-185.
• Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. New York: Longmans, Green.
• Clegg, V. L., & Cashin, W. E. (1986). Improving multiple-choice tests. Idea paper no. 16. Kansas State University: Center for Faculty Evaluation and Development.
• Friedman, H. (2002). Immediate feedback, No return test procedure for introductory courses. In R. Griggs (Ed.). Handbook for teaching introductory psychology (p. 132). NewYork: Lawrence Erlbaum Associates.
• McClain, L. (1983). Behavior during examinations: A comparison of A, C, and F students. Teaching of Psychology, 10, 69-71.
• Renner, C. H., & Renner, M. J. (1999). How to create a good exam. In B. Perlman, L. McCann, & S. McFadden (Eds.). Lessons learned: Practical advice for the teaching of psychology (Vol. 1) (pp. 43-47). Washington, DC: American Psychological Society.
• Sechrest, L., Kihlstrom, J. F., & Bootzin, R. R. (1999). How to develop multiple-choice exams (1999). In B. Perlman, L. McCann, & S. McFadden (Eds.). Lessons learned: Practical advice for the teaching of psychology (Vol. 1) (pp. 49-56). Washington, DC: American Psychological Society.