Meta-Analysis Helps Psychologists Build Knowledge

When scientists want to know the answer to a question that’s been studied a great deal, they conduct something called a meta-analysis, pooling data from multiple studies to arrive at one combined answer. Some people think this creates a chilling effect, shutting off further inquiry. But the authors of a new paper published in Perspectives in Psychological Science, a journal of the Association for Psychological Science, write that meta-analysis actually helps scientific fields develop.

There are many meta-analyses in psychology and medicine, areas where studies find often conflicting results. For example, some experiments might find that a particular drug decreases the risk of heart attack, while other experiments might find the drug doesn’t have any particular effect. A meta-analysis takes the results from all published studies on the same question and combines them; it’s as if someone had done a single study with a much larger sample size.

The answers that a meta-analysis comes up with can guide the field. But it may also stop scientists from looking at questions related to whatever the meta-analysis was about. “We started out asking the question, does it have a chilling effect? What happens after a meta-analysis?” says Richard D. Arvey of the National University of Singapore, who co-wrote the article with graduate student Meow Lan Evelyn Chan. They looked at a particular area and found that researchers did continue to conduct studies after the meta-analysis was published.

Meta-analysis can be very useful, Arvey and Chan argue. Arvey gives an example from his own experience: He was an expert witness in an age discrimination case. The workers who thought they had been discriminated against had an expert witness who presented a study that found the kinds of things the company was accused of were because of age discrimination. But Arvey used a meta-analysis to show, he says, that “The data this person had produced was an outlier, a very unusual result compared to all the other studies.” He showed that this study did not represent what had been found by the field as a whole.

This method can also help guide scientists on what kinds of work to subsequently explore. Arvey is starting to study the neurological basis of leadership, and he’s using meta-analysis to find out what personality characteristics are generally associated with leadership. “I will start looking at specific measures of those traits in my own research,” he says.

Just the fact that it’s possible to do meta-analyses in psychology shows that psychology is a fairly mature field, Arvey says. “In order to do meta-analysis, you have to have enough scholars who have studied the phenomenon in the first place.” Meta-analysis has helped psychology mature over the last 20 years, he says, helping scientists to develop paradigms for understanding human behavior and focus more on important questions.