Science Goes to School: Grant Program Seeks Scientific Foundation for Nation’s Schools

He was hooked by a 5-year-old girl. While at State University of New York-Stony Brook, Robert Siegler conducted a standard experiment, but with low expectations that he could duplicate Jean Piaget’s findings. He poured water from one glass into a taller, thinner glass and asked the young girl if the amount of water had changed. She said yes, there was now more water.

He repeated the procedure and asked his advisor, Robert Liebert, the same question. Liebert answered correctly, of course, and explained why pouring didn’t change the amount.

Siegler again asked the little girl, but she stuck by her answer, so Liebert repeated his response “in a lower voice and about 10 decibels louder.” Again Siegler asked the girl. “To my astonishment, she repeated her original answer and reasoning. I went back to Liebert once more, but even then she clung to her belief. That’s what hooked me: any intellectual belief that could motivate a small child to defy a large, imposing adult was worthy of serious investigation.”

The US government now appears hooked as well and is putting money where its curiosity is by funding studies that seek to build a scientific foundation for the nation’s education system of the future. Today, Siegler is one of three APS Fellows among eight scientists (all but one a psychologist) who are undertaking three-year research projects in the first wave of the U.S. Department of Education’s Cognition and Student Learning grant program. The other two are APS Past President Robert A. Bjork at the University of California, Los Angeles, and Harold Pashler at the University of California, San Diego.

The program is a serendipitous intersection of political will and scientific readiness. As Bjork points out, “The opportunity to upgrade education comes from extensive progress in the last several decades on understanding the cognitive processes that underlie learning. Basic research on learning and memory now provides a foundation for improving educational practices, potentially in revolutionary ways.”

Their projects in brief (see below for expanded descriptions):

• At Carnegie Mellon University, Siegler and his colleagues are testing kindergarten students to determine if difficulties in learning to solve science problems stem from misconceptions about the difference between “living” and “nonliving,” and second-, third- and fourth-graders to see if difficulty with math problems stem from fundamental misconceptions about whole number relationships. The researchers base their hypotheses on prior findings that, for example, “most US fourth through sixth graders draw a false analogy between decimal fractions and whole numbers and conclude that decimals with more numbers always are larger than ones with fewer numbers.”
• At the UCLA, Bjork is teaming with Marcia Linn, an educational psychologist and researcher at the University of California-Berkeley, to see if infusing her Web-based science curriculum for middle and high school students with what Bjork calls “desirable difficulties” will result in longer retention of what is learned. His “difficulties” include spacing rather than massing study sessions; interleaving practice on separate topics; varying how material is presented; reducing feedback, and using tests as learning events. Earlier studies that show the benefits of such strategies “not only raise concerns about prevailing educational practices,” Bjork says, “but also suggest unintuitive ways to enhance instruction.”
• Pashler’s team, based at the UC-San Diego, is also working on learning retention both with college students in a laboratory and a more diverse population on the Internet. They are measuring how much time lapse between study sessions produces the best results; measuring how much “over-learning” can most efficiently maximize what is remembered later, and exploring novel ways to predict and minimize forgetting. They hope to determine whether such strategies that work for simple rote learning can also work for other learning tasks.

One thing all three have in common is gratification at the new emphasis on evidence-based approaches to improving American education.

“The vast majority of educational research hasn’t even involved measures of learning outcomes, much less random assignment of treatments,” says Pashler. “Imagine if medical research had been done without clinical trials. The New England Journal of Medicine would be printing surveys about the kinds of leeches doctors prefer. There is a real bipartisan push now to promote evidence-based approaches in education, medicine, and social policy generally.”

Siegler says he is “extremely heartened by the Bush administration’s support for scientifically sound research on children’s learning. I think the funding climate is more favorable for developmental psychologists interested in learning than it’s been in a very long time.

“In the past, many developmental psychologists felt, with good reason, that the vast majority of Department of Education funding wasn’t intended for them. The current administration has expressed, both in words and deeds, a heightened interest in scientifically sound research on children’s learning. The field is indeed ripe for increased efforts to apply developmental psychology techniques to educationally relevant issues.”

One factor fueling the new effort to bring the science of learning into America’s classrooms is the shift in developmental psychology from “thinking” back to “learning.” As Siegler explained in his grant proposal:

“At one time, learning was the central topic in developmental psychology. However, with the rise of Piaget’s theory – the emphasis shifted from learning to thinking. This shift helped create a vibrant field of cognitive development, but it also had a cost. It led developmentalists to focus on differences among children of different ages rather than on processes that lead to cognitive growth. The situation is changing, though. Research on learning is now informed by a much richer understanding of what children bring with them to the learning process.”

“Psychological research is driven in part by fads,” Siegler says, “but the enduring issues always return. I always believed that developmental psychology, which by its very nature is concerned with change, would inevitably refocus its attention on how change occurs.”

Last year Congress created within the Department of Education an Institute of Education Sciences, replacing the Office of Educational Research and Improvement that had formerly been responsible for education research and statistics and had launched the CASL program, now being administered by the new Institute.

Assistant Education Secretary Grover J. “Russ” Whitehurst, formerly a professor of psychology and pediatrics and psychology department chair at Siegler’s alma mater, SUNY-Stony Brook, was appointed its first director.

He described CASL’s purpose to scientists at a February 2002 meeting, but could just as well have been describing the future Institute’s mission: “The field [of education] still operates largely on the basis of folk wisdom. It does that more or less effectively. We have, after all, as a human species, been educating ourselves for thousands of years. “[T]hat process succeeds, as long as the tasks to be learned are simple, and as long as we are willing to tolerate an elitist system in which only those who are most cognitively able and have the most cultural support are expected to learn complicated subjects at high levels. But, we have a new day. Folk wisdom will not serve us anymore. We need a new way.”

“Where medicine has biochemistry, and where agriculture has plant genetics, education has cognitive psychology and brain science. These are the basic sciences that are going to revolutionize the practice of education in the 21st century.”

A primary goal of CASL, Whitehurst says, is moving “from our current research culture, which is oriented towards serving the needs and interests of other researchers, to a research culture which considers and serves the needs of practitioners. One of the things we want this competition to accomplish is to take those of you who are trained in cognitive science, who are accustomed to doing research in the laboratory, at the university, with samples of convenience – and to move that research endeavor into school-based work or applied learning. We hope you will do that.”

Bjork’s teaming with educational researcher Linn is an example of just such collaboration, but Bjork acknowledges that “the challenge of bridging the gap between basic research and translated educational practice is great. In my opinion, the gap is huge right now between where research stands and what kinds of principles often guide educational practice. In part, that gap is huge because there has been so much progress in basic research across the last couple of decades.”

There are also other reasons the challenge is great, according to Bjork, who is chairing the panel that selects this year’s CASL grantees.

“Bridging the gap demands a level of cooperation and communication and good will between researchers and educators that, for a variety of reasons, has not existed, and sometimes has been actively resisted,” he says. “There’s plenty of blame on both parties for that lack of communication and cooperation. Part of the problem is the sort of sociology of science, where, sort of working in this particular gap between basic research and educational practice is often not rewarded to the same degree in the university setting that the basic research itself is.”

Yet another obstacle is the “prevailing societal attitude” that gets in the way of applying good educational science, he says. “In my opinion, there is a dramatic over-tendency and over-appreciation in our society to attribute differences in performance to differences in innate ability between individuals – and, a wild under-appreciation of the power of practice, experience, and the basic capability of all of us to learn.

“We come equipped with this remarkable kind of ability” to learn, but students who do poorly in an early math test, for example, will decide that’s not “their thing” and spend the rest of their lives turning that into a self-fulfilling prophecy. “Ethnic groups, families, decide ‘their thing’ is to be not good at this or good at that. “It nullifies this remarkable ability that almost all of us have to learn in all sorts of domains. If you could change the culture to just have people understand the potential to learn, I think it would be a terrific contribution and would change things dramatically in terms of the personal education we give ourselves and in the broader educational practice.”

Some obstacles are purely logistical: classroom students away on field trips or sick at home on the day the experimental work is planned, and the paperwork involved in securing permissions and informed consent.

“Even though, at one level, it’s hard to imagine research that is more critical and of more national importance,” says Bjork, “everyone is aware of the difficulties of actual interventions in school. If you’re going to do what amounts to good experimental research using students in schools, there’s a huge amount you have to go through in terms of gaining cooperation.

“Speaking personally, over the years that has been a bit of a deterrent for me. Schools have to be very, very careful, their permission procedures can get very complicated. It hasn’t been very attractive for psychologists from that standpoint. You lose control over lots and lots of things, so people get pushed toward working in their constrained laboratories where they can control everything.”

The secret, he says, is to collaborate with an educational researcher who already has approvals and connections in place, someone like Linn. “What she knows about dealing with the public schools, it so swamps what I know there’s just no comparison,” Bjork says.

As wary as basic researchers might be of school-based entanglements, educators and educational researchers have their skepticism about laboratory scientists, too.

“On the one hand, they’re positive about efforts to join in basic research,” says Bjork, “but on the other hand they tend to think that people who have done basic research all their lives are naïve with respect to the realities of the classroom. There’s truth in that. I do think that’s a burden in this collaboration, that laboratory-bound types have a Pollyanna sort of incomplete view of the realities of the educational process. A basic researcher like me can’t just say look at this awful gap, look at what they don’t know and what they could apply.”

The bridge between basic research and educational practice may be fraught with obstacles, Bjork says, but “there’s a kind of readiness. I think educators in the trenches are more ready to cooperate than they’ve been. [And] people like myself and others who’ve been buried in our laboratories with our simple materials are eager to try to make a difference. There’s a whole lot to learn from each other, and to the extent that this current program could achieve that, that would be good progress all by itself.”

Teachers judge whether their students are learning from their classroom performance, using such measures as test scores. That can be “a notoriously poor guide,” warns Robert Bjork of the University of California, Los Angeles, because performance in class doesn’t show that the student will remember what was studied later in life.

“Instead, we can improve retention of what’s learned by teaching with desirable difficulties,” says Bjork. Desirable difficulties are counter-intuitive, “things that appear during instruction or training to slow the learner’s progress, sometimes upset or possibly even discourage the learner, but then are associated with better long-term retention and transfer.”

Under half a million dollar grant from the U.S. Department of Education’s Cognition and Student Learning program, Bjork has teamed with Marcia Linn of the University of California, Berkeley. Their collaboration applies Bjork’s and his colleagues’ IDDEAS (Introducing Desirable Difficulties for Educational Applications in Science) to Linn’s and her colleagues’ WISE (Web-based Inquiry Science Environment) set of instructional modules for middle- and high-school students.

The proposal partners cognitive researchers, educational researchers, and classroom teachers “who jointly design and carry out experiments in progressively more complex educational settings.” If successful, IDDEAS will develop theory-based principles to guide future instructional designers working in new contexts.”

The desirable difficulties Bjork’s team is testing include introducing variation and unpredictability; other “contextual interference” such as interleaving materials and adding to task demands, distributing practice sessions over time, reducing feedback, and using tests as learning events. They are testing them in two WISE modules: one that teaches about malaria transmission by mosquitoes and the roles of DDT, developing a vaccine, and instituting behavioral solutions; and another that contrasts models of light propagation. Over its three years, the study will:

Year One: Conduct baseline testing of high school students from selected classes and manipulate a variety of desirable difficulties with introductory psychology students at UCLA.

Year Two: Refine the manipulations based on year-one results and assign them to half the high school classes selected from the first year’s baseline data for comparison against control classes using the standard WISE modules, test desirable difficulties on college students in two additional areas to determine whether they can be generalized, and conduct baseline testing in a Hispanic school.

Year Three: After further refinement of the manipulations, conduct a pilot project asking a new cohort of WISE project developers to try the principles; add to the pre-college participants students from the primarily Hispanic schools that carried out the baseline test in the second year.

An important feature, Bjork says, is that the study addresses a major concern of educators. “Teachers are skeptical of innovation,” he says, “and properly so. Sometimes what amounts to quackery motivates big changes, and they become skeptical. They get all kinds of rules about check lists – did they cover this, that, and the other thing. They reach a point where they say, ‘Leave me alone! Let me teach!’

“One nice thing about the WiSE modules we’ve proposed, is that we don’t have to place any additional burdens on the teachers. If they’re using the WISE project on light propagation as a tool they wouldn’t have otherwise, and whether they’re using the standard one or the alternative one we think would be better, it doesn’t matter from their standpoint.”

Children’s difficulties with math and science problems often stem from confusion over fundamen-tal concepts, says Robert Siegler, Carnegie Mellon University.

In the United States, most fourth through sixth graders believe that because whole numbers with more digits are larger than those with fewer, the same is true about decimal fractions. Research shows that children who have such misconceptions have less success later on learning correct math procedures.

Studies with children in the United States, Japan, and Israel also reveal that most 5- to 6-year-olds and about half of 7- to 8-year-olds think either that everything is a living thing or that only animals are living things.

Siegler, an APS Fellow and Charter Member, and his colleagues want to see if students’ initial misconceptions about whole numbers and fundamental concepts in science also affect their learning, and whether targeting those misconceptions early could improve learning outcomes down the road. They are conducting two series of experiments – three in math, three in science – under a $429,000 grant in the Department of Education’s Cognition and Student Learning Research program.

The first in each series tests targeting teaching to overcome misconceptions, the second investigates whether the misconceptions are central to other misconceptions, and the third examines whether correcting misconceptions early on can improve later learning.

THE MATHEMATICS SERIES WILL:

1. Test the ability of 64 second graders and 64 fourth graders to correctly place a number on a line from 0 to 1,000.
2. Test 48 third graders and 48 fourth graders to determine if their ability in the first test correlates with their ability to estimate other things – for example, the width of their desktops after being shown how long an inch is.
3. Test 40 fourth graders to see if those who did best on number line placements also learn more quickly to solve more complex math problems.

“So far, we’ve conducted two of these proposed [mathematics] experiments,” Siegler says. “The results have met or exceeded our expectations. For example, we’ve found surprisingly strong and consistent relations between number line estimation and performance on standardized math achievement tests.”

THE SCIENCE SERIES WILL:

1. Show 144 kindergarten children cards containing three photographs – an animal, a plant and an artifact, then ask them which represent living things. After feedback to correct misconceptions, they will be retested one and six weeks later to see if the feedback helped.
2. Test 80 kindergarten children with the photo cards. Those who make errors will be retested and asked to say whether the photographed items can move toward goals and why. Some will have wrong answers corrected, some their explanations, some both and some neither. They’ll be retested immediately, after a week and after six weeks to determine the impact. “Learning why categories are defined as they are may be more important for enhancing subsequent learning than simply learning the category definition per se,” Siegler explains.
3. Ask 60 kindergarten children which photos represent things that have “golgi” – a microbiological term they are hardly likely to know. Some will be taught the term “organism” and asked to say which are organisms, then re-tested after getting feedback. Others will substitute “living thing” for “organism,” and a third group will get no labels. All will then be retested on “golgi,” first without and again with feedback. All of which, Siegler explains, “will test whether learning a category label promotes generalization of a novel property to living things, and whether novel labels are particularly effective in promoting such generalization.”

“We suspect a lot of educational failures reflect people forgetting things they once knew, to at least some degree,” says Harold Pashler of the University of California, San Diego.

“Take these studies one often sees in the media showing that students cannot find France on the map, or don’t know that the United States gained independence from Great Britain rather than, say, China. We suspect that the vast majority of children at least briefly knew these things and then later forgot them. The same happens in the workplace: millions of workers learn to use Microsoft Excel but then, when they really need to use it, they can’t remember exactly how to do what they want.”

With a $500,000 grant from the Department of Education’s Cognition and Student Learning program, Siegler and colleagues John Wixted and Nicholas Cepeda at UC-San Diego and Douglas Rohrer at the University of South Florida are studying two educational strategies aimed at improving retention: spacing instruction over time and “over-learning,” studying beyond the point of 100 percent recall.

“Our goal here is to derive concrete principles for reducing forgetting by the optimal use of study time,” Pashler says. “We see no reason to believe our findings are necessarily restricted to rote learning, and we will do some studies to see whether they are.”

The “spacing effect” has been widely investigated, APS Fellow Pashler says, “but the results have had depressingly little impact on educational practice,” largely because of “important limitations” in the studies themselves that his team seeks to remedy.

In both the laboratory and in Internet courses, where they expect more diverse subjects, they are teaching words in unfamiliar languages (Yupik, Luganda and Chocktaw), little-known facts and obscure English vocabulary, even training subjects in visuospatial categories and teaching them to identify melanoma from dermatological photographs, all in efforts to examine the effects of both spacing instruction out over days, weeks and even months, and of over-learning.

“We want to understand how you can predict when something has been learned enough that it won’t be forgotten over some period of time,” Pashler says. “For example, suppose you want to study some Czech words on two occasions, and you hope to remember them next summer when you go to Prague. How long should the interval be? We already know that 10 minutes probably isn’t enough, but how about a day? A week? A month? These kinds of questions have not been answered.”

Studies also show that over-learning does work, he says, “but it isn’t clear whether it is an efficient use of time or not. It may be better not to practice much after you’re getting something right, but rather to wait a while before testing some more.”

In the end, his team hopes to devise formal models of how key variables affect retention and to find whether instruction can be adapted to individual differences in learning and forgetting. “We will look at the amount of forgetting over periods of up to two years, based on the timing of the sessions and the test,” Pashler says.

“Also, we’d like to be able to predict which learning is most vulnerable to forgetting, based on what happens during learning. The old literature showed that when you learn everything to the same criterion, such as getting it right once on a test, what you learned the slowest is what you forget the fastest. We hope to figure out algorithms to compensate for this in the learning process.”