The AP Psychology Reading: Life’s a Beach for Blair-Broeker

The AP Reading Process The exams are stored in folders that hold 25 each and in boxes that hold about 10 folders. This is only a portion of the total. Readers receive rubric training for Question 2 on the exam. Over a half a day is spent on training. After training, readers are assigned to a table and supervised by a table leader. Each reader is also assigned a partner.

Imagine getting paid to spend a week in Daytona Beach with a couple hundred psychology-teaching colleagues. You’ll stay at the Adam’s Mark Hotel, right on the beach, and spend your days interacting with high school teachers and college professors who share your passion for psychology and teaching. Every hour or two you’ll be fed a meal or a snack, and if you’re feeling adventurous at dinnertime there are dozens of restaurants within an easy walk. Some evenings involve professional activities, including teaching exchanges, video screenings, and a talk about Inez Beverly Prosser by Ludy Benjamin, a noted historian of psychology. Other evenings are social, with a trip to see the Daytona Cubs (where members of your group are selected to throw the ceremonial first pitches) and parties featuring skits so clever you’ll laugh hard enough to cry. Did I mention that you’d get paid for all this?

There is a catch, of course. The reason you’re being paid is to bring your professional expertise to the task of reading the essays written by high school students hoping to receive college credit for the Advanced Placement Psychology courses they have been enrolled in. Seventy-two thousand students took the AP Psychology Exam this year, and each of them wrote two essays (called “free responses” to differentiate them from formally constructed English essays). That’s almost 150,000 essays that deserve the careful attention of you and your colleagues.

The College Board’s Advanced Placement Program has been in existence for almost half a century. AP Exams are developed, administered, and scored by Educational Testing Service, or ETS, for 31 courses. Psychology is one of the newer kids on the block, having been added in 1992. That year, about 4,000 students took the psychology exam, and a team of about 20 readers was sufficient to get the essays scored. It has been gratifying to see the rapid growth in popularity in AP psychology, especially since the quality of student performance has not dropped off as the numbers have increased.

Exams for the various AP disciplines are read in a variety of locations around the United States – not everyone gets to spend time on Daytona Beach! In fact, psychology readers spent time at Clemson University, the University of Maryland, and Trinity University in San Antonio before landing in Florida a few years back. ETS is required to solve a logistical puzzle each year when it comes to finding locations to fit the size of each subject’s reading session in an efficient manner. Most reading sessions occur on university campuses and subjects may get shuffled to different places each year as the number of test takers changes.

It is a lot easier to describe the events surrounding the reading session than to describe the essay reading process itself. One might think that sitting at a table going through answers to the same question for eight hours a day is little better than hell on earth. It is, in some ways, an exhausting task, but it is also fascinating to see how hundreds and hundreds of students respond to the same question. You quickly learn that some communicate their knowledge much more efficiently and effectively than others, and the insights gained from the reading is especially useful to high school teachers who will be preparing students to take subsequent tests. It’s much, much more than that, though.

High school teachers are often isolated. Unlike our college-level cousins, we usually don’t have colleagues in our building who teach psychology. Internet discussion groups help tremendously, but there is nothing like face-to-face contact with other psychology teachers in an environment of respect and equality to recharge our enthusiasm for the profession. Every year I meet new colleagues and find myself wondering what they’re like in their own classrooms. A dream of mine is to find a grant somewhere that would allow me to spend a year watching these folks work their magic in their own classrooms. I bet what I learn would make a pretty good book.

It’s also impressive to read the work of well-prepared students. Every day at the reading, I score superbly crafted essays that convey deep conceptual understanding of core psychological concepts. It is truly humbling to see high school students after their first psychology course write essays as good as I could muster even if I wasn’t in a timed, high pressure test environment. As you can imagine, not all of the student work is of this quality. Some of it is at least as awful as the good work is good (occasionally made more tolerable by the unintentional twists of the English language engineered by a struggling young author). My overall impression, though, is that the majority of these students are handling the college-level expectations well. I take pride in the inference that this is so because most of my high school colleagues are doing a good job of teaching the college-level AP course.

ETS stresses the importance of reliable, valid testing because the College Board would not be able to convince colleges to give college credit for exam performance otherwise. As psychology teachers, I think the readers in our discipline have a great appreciation for the need to reliably score essays. A team of teachers, known as “table leaders” in ETS parlance, arrives two days early to hammer out a concise, specific scoring rubric with the help of a “question leader” and ETS testing experts. The debates that this process produces are among the most stimulating professional conversations I have ever experienced. New layers of meaning and insight are invariably discovered as one considers the nuanced meanings of what often appear to be, at first blush, straightforward questions. The rubric is tested against hundreds of actual student essays during this period, and samples are selected to help train the readers on their first morning of work.

Readers are assigned to a table supervised by a table leader. Each reader is also assigned a partner, with care taken to pair new readers with experienced readers. New readers are brought on board every year to make sure the group doesn’t grow stagnant. The table leader is responsible for ongoing reliability checks during the reading process. This is accomplished by selecting essays randomly from each reader’s completed stack and rereading them, blindly, to see if the judgment of the reader and the table leader match. If they don’t, a conversation occurs to determine the cause of the discrepancy. In the vast majority of cases, the problem is caused by ambiguity in the student’s answer that can be interpreted more than one way. The overall accuracy of the scoring is remarkable, a tribute to the determination of everyone involved to give each and every student a fair shake when his or her answer booklet is opened. My students often have an image of readers as scowling meanies with red pens poised to rob them of hard-earned credit. Nothing could be farther from the truth.

It truly is difficult to describe the atmosphere that surrounds the reading, but most of us who participate look forward to it as the most interesting and rewarding event on our professional calendars. If you are interested in joining us, check out the AP Central Web site at http://apcentral.collegeboard.com/. It contains general information about the AP Program, and, if you complete the free registration, additional information about AP courses and exams. The application to become a reader can be found by following the “Professional Development” link on the left side of the page. Check it out. Consider joining us. Find out for yourself about the curious pilgrimage that draws us away from our homes and families each June.