October 2005
Volume 18, Number 10
A Self-Correcting Approach to Multiple Choice Tests
When you were a student, did you have any of the following experiences?
You dreaded taking multiple-choice tests.
You thought of the right answer after you had handed in the test.
You never went over the exam questions in much detail when it was returned.
As an instructor, have you had any of these experiences?
Students complained they were not good at taking multiple- choice tests.
The average class grade on an exam was much lower than anticipated.
Students did not understand fully material they just studied and it was relevant to upcoming class work.
You are not alone. The bright, engaging, and committed students I teach repeatedly declare that they hate multiple-choice tests and suspect that instructors design them just to trick students. I had the same attitudes as a student. Nevertheless, during my 15 years of teaching I have used multiple-choice tests extensively despite their "dubious" reputation. However, by appreciating students' concerns, considering the goals of my tests, and thinking more broadly about the test-taking experience, I believe I have learned some valuable lessons about how to enhance the learning potential of my multiple-choice tests. Certainly, designing good questions is an important first step. Equally, if not more importantly, it is vital to consider what you wish to accomplish with your tests and how your testing format can help achieve your goals. I view tests as an opportunity for learning. Thus, I have developed a multiple-choice testing format fueled by observations that learning is facilitated when students are given opportunities to practice for exams (Balch, 1998), when they are encouraged to thoroughly consider alternative answer options (McClain, 1983), and when they are given immediate feedback about their performance (Friedman, 2002).
I use standard multiple-choice tests in my classes. However, instead of coming to class, taking the exam, and waiting until it is graded to see how they did, students engage in what I call self-correcting. Students come to class prepared as they would be for any other multiple-choice exam, take the exam, and then they take it home and review each question to assess whether their answer was indeed the best one. Students can use class notes, readings, and even discuss the questions with their classmates (indeed such collaboration is encouraged). As they do so, they can change their answers. Students return exams during the next class period and the self-corrected version determines their final grade, as follows. For each correct answer (no change) students receive full credit. For each corrected answer (wrong to right), students receive half-credit. Incorrect answers — originally wrong and unchanged, or changed to wrong — receive no credit.
I have observed that students do not just perform better than with a more traditional testing format, but they also learn better. Why? Students no longer see tests as teaching weapons but as learning tools. Certainly students are still concerned about their performance but they feel more in control of the experience and its outcome. As well, they have the opportunity to reflect on the material and correct misunderstandings, making learning subsequent material more efficient. Students also learn how to take multiple-choice tests more effectively and, because tests now require using a variety of skills, they appeal to students with diverse learning styles.
Frequently Asked Questions and Answers
Surely this self-correcting approach invites questions.
Here are some frequently asked questions along with my
responses.
Can't students just change their answers to all the right ones and not show any corrections?
No. Students know that I have a copy of their original answers so I can track any changes they make.
Couldn't students simply take answers from other students without participating in any constructive discussion? Doesn't this set the stage for more extreme cheating schemes?
Sure. Although students are forewarned that their minds and grades will benefit the most by approaching the test as advised, some will attempt a bit of "social loafing". Some may even contemplate ways to "beat the system". However, I am open with students about the advantages and disadvantages of their test-taking attitudes and strategies, and I hope our discussions provide an opportunity for another level of learning. It is my opinion that cheating behaviors reflect situational and interpersonal demands of the classroom context and are not simply the mark of corrupt or lazy students. Thus, if students are resorting to serious and systematic cheating, then there likely is a bigger issue of concern in the classroom that warrants attention. Moreover, invariably a case arises when a student took a friend's word about an answer that turned out to be wrong, which in turn gave the student pause about taking advice blindly in the future. This is a good lesson to learn.
Don't students study less?
I often have students take a second class with me and ask them to describe their experience with my exams and give advice to new students. Consistently, their advice is to study as best they can for the exams because they are challenging and they need to be prepared. Some students report studying less than usual for the first exam but then realize that the boost that comes from self-correcting is built on starting off on solid ground. Thus, they report putting in more effort for subsequent tests.
Aren't your grades inflated?
No. The means of my exams are a bit higher than in previous classes but not so much that they would be considered inflated. Moreover, there is always a good range of scores. Also, exams generally count for no more than 60 percent of the final grade and so grades become refined and distributed through other class requirements. In comparison to other courses, I often award a lower percentage of As (yet I have very high course evaluations with approving comments about the testing format). Thus, the format does not appear to inflate grades. It does seem to improve morale and attitudes toward tests.
Why not just give practice tests?
I think the reflective experience and the opportunity to personally correct errors that this format affords is an important element of the learning process.
Are all of the exams self-correcting?
I typically give three exams. The first two are self-correcting. The final exam is not and it is cumulative.
Why not have the final be self-correcting?
This testing format is directed toward helping students refine their working knowledge and learn how to approach a test to assess that knowledge. I think the cumulative final helps to further these goals by giving students an opportunity to "walk on their own." Besides, getting a corrected version back before final grades must be assigned would be a problem. In addition, the final cumulative exam can serve as an evaluation tool.
What are the downsides of the format?
There are two time-related challenges. I give students back their exams after they have been officially corrected. They are encouraged to use them to study for their cumulative final. This means that prior exams are in circulation and new students may use them. Although I have not seen direct evidence of this, I initially needed to spend extra time generating new questions. However, I now have a large pool of questions from which I can generate exams with ease.
I have teaching assistants who correct exams in classes that average about 60 students, and it takes about 2 hours to complete the scoring. Instructors with larger classes without assistants may find a manual correcting process difficult, especially if it is used in multiple classes. Perhaps an innovative instructor will discover a technological solution.
Does this format work better for some students than others?
This is an interesting question about which I wish I had empirical data. Research suggests that "A" students work through question options more than "C" and "F" students (McClain, 1983), and so I would predict that students who may otherwise perform poorly would benefit the most. However, I often find "A" students are very anxious and report that this format relieves some tension so they have more of an opportunity to "digest" rather than to "regurgitate" information. Thus, I would like to believe this format offers something of benefit to all students.
If you have read to this point, you are hopefully now interested in seeing how the process operates in more detail. Described below are a few suggestions for writing multiple-choice questions, how to implement this format, and some advice about how to approach this testing experience.
Writing Good Questions
There are a number of excellent resources for more
detailed information about how to construct good multiple-choices
tests (e.g., Clegg & Cashin, 1986; Renner & Renner,
1999; Sechrest, Kihlstrom, & Bootzin, 1999). I have found
the following suggestions especially useful.
| Tapping a Range of Thinking Skills A Useful Taxonomy Suggested by Bloom (1956) |
||
| Thought Process (Simple to Complex) |
Learning Activity | Words to Use in Questions |
| Knowledge | Remembering facts, terms, concepts, definititions, principles | Define, list, state, identify, label, name, who? when? where? what? |
| Comprehension | Explaining or interpreting the meaning of material | Explain, predict, interpret, infer, summarize, is an example of, can account for |
| Application | Using a concept of principle to solve a problem | Apply, solve, show, make use of, modify, demonstrate |
| Analysis | Breaking material down into its component parts to see interrelationships and hierarchy of ideas | Differentiate, compare and contrast, distinguish __ from __, how does __ relate to __, why does __ act this way? |
| Synthesis | Producing something new or original from component parts | Design, construct, develop, formulate, imagine, create, change |
| Evaluation | Making a judgement based on established set of criteria | Appraise, evaluate, justify, judge, which would be better? |
- Write questions that make use of a variety of thinking skills. The taxonomy suggested by Bloom (1956) and presented in Table 1 is a useful guide.
- Write questions that reach for more complex thought processes as often as possible. Include words that point to the kind of thinking you expect the question to utilize.
- Present the problem in the simplest form with precision and clarity, and the options in the simplest form possible. Do not repeat material unnecessarily.
- Do not include unnecessary descriptive material in questions and answers. It does not contribute to interest or complexity, and it just adds confusion and reading time.
- Use options such as all of the above or none of the above with caution. Avoid implausible or absurd distracters in the options. They make questions easier to answer simply because the option can be easily rejected. However, an amusing or entertaining option used sparingly can break tension and remind students about the constructive goal of the test.
- Avoid giving irrelevant clues to the answers such as length of clue (longest is often the correct option) or inconsistencies in grammar. Avoid the use of specific determiners (all, never, and always suggest incorrect options whereas usually, sometimes, typically, and maybe suggest correct options).
- If a question contains controversial material, cite the author whose opinion is stated. For example, "according to class lecture..." or "as stated by Rosenthal..." This citation helps reduce subjective interpretations.
Implementing the Self-Correcting Format
Students should be given an exam with instructions and
questions similar to those below with a separate numbered
answer sheet. Be sure names are recorded on all materials.
Sample Instructions
Welcome. Keep in mind that exams in our class are treated
as opportunities for learning. To this end, complete this exam
by circling the best answer to each question on your exam.
Do not skip any questions. Also, record your answers on the
separate sheet that has been provided and hand it in before
you leave. When you are done, take your exam home and
review each question using your notes, text, and discussions
with classmates (be sure to consider each answer option
thoroughly). If you wish to change an answer, indicate it
in the space provided on the exam. Use this space only for
changed responses. Return your corrected exam at the next
class meeting. Late exams lose the self-correcting option.
Sample Question Format
A researcher interested in the impact of story telling on
the development of social norms asks a fifth grade teacher
to read stories to his class involving cooperation. Another
teacher reads stories involving competition. The researcher
then codes the amount of sharing children from each class
displayed during their lunch periods. In order to avoid experimenter
bias in the coding of behaviors, what should be
done?
- The children from each class should wear different colored tee shirts.
- The children should be observed over the course of several lunch periods.
- A researcher unfamiliar with the manipulation should record the behaviors.
- The researcher should read the stories to the children in each class.
Corrected answer: ____________
Advice for the Instructors
Listed below are points to keep in mind when using this
format in your class.
Take a Few Trial Runs
At the same time, take seriously the need to think the
format out fully to avoid confusion and frustration. It might
be useful to pilot test the format on a quiz rather than a
lengthy exam.
Expect Students to be More Willing to Voice Their
Reactions to the Test
When I enter the classroom on the day exams are due,
students are always chattering about the test. I purposefully
acknowledge the energy – even tension – in the room to
remind students that the test was designed to promote this
activity. Some will ask to discuss certain questions. I ask them
to make note of the questions and after exams are handed in
I address ones about which several students have concerns.
Each instructor will need to decide how to gauge these
discussions, bearing in mind the importance of modeling
constructive dialogue. I keep discussions short and focused.
Time for additional discussion is offered after class or via
e-mail. Students also sometimes include written comments
on their exams. These comments have been invaluable in
helping me craft better questions.
Describe Your Exam Philosophy in Your Syllabus
Explain your testing philosophy to students on the first
day of class and how it is reflected in your exam format. The
notion that tests can be opportunities for learning will be a
novel perspective and you can use it to generate discussion
and interaction. You might begin by asking who does and
does not like multiple-choice exams, and why they do or do
not. Drawing on their responses, explain why you selected
this new format. Students do not need to know all of the procedural
details at this point, a preview should be sufficient. A
short time before the first test go over the testing procedures
and how students should prepare for the exam.
Do not Make Tests and Questions More Difficult than
Usual
Keep in mind the goal of your test as a learning tool as
you decide what material to include.
Learn to Write Good Multiple-Choice Tests
If your tests are learning tools, they should represent
the material to be learned and ask questions that encourage
students to use a range of thinking skills.
Sometimes Students will Change an Answer from Correct
to Incorrect and They Will Want to Know What Happens
if They Do This
This is typically treated as a wrong answer and no credit
is given. It has been my experience that this happens for one
of three reasons:
- The question was "bad" and several students got it wrong in this manner. If this appears to be the case you can drop the question or accept the answers that make sense with hindsight.
- Students simply listened to their friends and changed their answers without further scrutiny. In this case, students need to be reminded how to approach reviewing their answers and relying on more reliable sources.
- A student reads too much into the question. Sometimes students get questions wrong for the right reasons because the instructor failed to realize how a question might be perceived or interpreted by someone with a different level of knowledge. These instances can be dealt with on a caseby- case basis and the instructor's assessment of the student's knowledge of the topic and question at hand. This is a benefit because you get to know your students better as learners.
Students Should be Told in Advance How these Details
would be Handled
Given the opportunity this format provides for students
to obtain a better grade through self-reflection and revision,
typical complaints about the wording of questions, trick questions,
lack of information, and the like are often significantly
reduced. Moreover, when I do encounter debates about questions
(which will happen), I remind students that this type
of involvement was something the test intended to inspire.
Having students more actively in charge of their learning
experience is uplifting for them as well as the instructor.
Look at Students' Exams and Answer Sheets Before They
Leave Class
A few students will fail to record their answers on both
the exam and the answer sheet. It is important that all materials
are completed.
Return Exams so Students Can Go Over Corrections
It is important that students have the opportunity to review
their exams after they have been formally corrected.
I give exams back with incorrect and partial credit answers
marked as well as an answer sheet with correct answers to the
entire exam. If you give a cumulative final it will be helpful
for students to use previous exams as a study tool. Suggest
to students that they mentally rewrite questions so that they
point to different answers as a way to test themselves. Also
point out that it is not sufficient for students to merely learn
what the right answer was. They need to know why they were
wrong so that when they see a different question on the same
topic they will be able to answer it correctly.
Advice for Students
For optimum effectiveness, it is important to inform students
about how to approach and manage the testing format.
Emphasize these points.
Tests are Another Opportunity for Learning. Take Full
Advantage of Them.
Tests can help you identify your strengths and weaknesses
in the learning context. Try to reframe your perspective on
taking tests by approaching this new format with a serious
and open attitude. Use this opportunity to take a closer look
at how you handle tests — what you study, how you organize
time, how you answer questions — and what skills you can
polish for future use.
Study as You Would for any Multiple-Choice Test
Do not put too little effort into preparing for the test
because you have an opportunity to review and revise your
answers.
Do Not Answer a Question Hastily with the Expectation
that You can Change it Later
If you want a good grade you need to get full credit for
as many questions as you can. This means trying your best
to work through the question accurately the first time.
Consensus Does Not Insure Accuracy
If you are working with classmates and they say a particular
answer is correct, but it is different from your answer,
be sure they have satisfactorily justified their choice and that
you understand and accept fully the reason.
Conclusion
You may think of issues or concerns I have overlooked in
designing the testing format I have described. If so, I hope you
will share your thoughts with me. I also hope you will share
with me the experience of developing other new and novel
ways to enhance the potential for learning in your classes.
References
- Balch, W. R. (1998). Practice versus review exams and final exam performance. Teaching of Psychology, 25, 181-185.
- Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. New York: Longmans, Green.
- Clegg, V. L., & Cashin, W. E. (1986). Improving multiple-choice tests. Idea paper no. 16. Kansas State University: Center for Faculty Evaluation and Development.
- Friedman, H. (2002). Immediate feedback, No return test procedure for introductory courses. In R. Griggs (Ed.). Handbook for teaching introductory psychology (p. 132). NewYork: Lawrence Erlbaum Associates.
- McClain, L. (1983). Behavior during examinations: A comparison of A, C, and F students. Teaching of Psychology, 10, 69-71.
- Renner, C. H., & Renner, M. J. (1999). How to create a good exam. In B. Perlman, L. McCann, & S. McFadden (Eds.). Lessons learned: Practical advice for the teaching of psychology (Vol. 1) (pp. 43-47). Washington, DC: American Psychological Society.
- Sechrest, L., Kihlstrom, J. F., & Bootzin, R. R. (1999). How to develop multiple-choice exams (1999). In B. Perlman, L. McCann, & S. McFadden (Eds.). Lessons learned: Practical advice for the teaching of psychology (Vol. 1) (pp. 49-56). Washington, DC: American Psychological Society.
