Myth: Brain Training Will Make You Smarter

SUGGESTED LESSON PLAN

Clearly most of us would like to be more successful in school and at work, especially if we don’t have to work hard for guaranteed achievement. That’s pretty much what companies promoting “Brain Training” are offering. Rather than engaging in the difficult task of thinking and learning in varied and challenging educational environments, we can “get smarter” by playing games and “having fun” while doing it. The assumption in the claim (sometimes implied and sometimes explicit) is that, just as muscles get stronger by working out, the brain gets stronger with these exercises. But the brain is not a muscle and there is no “intelligence” equivalent of strength. Although practicing a cognitive task is likely to improve performance on that task, there is little compelling evidence that the training transfers to solving real world problems. The claims made by brain training sites are not supported by the data.

Researching this persuasive psych myth is an exercise in using the tools of scientific reasoning to think about popular claims regarding intelligence. This proposed module will follow the model of Activate, Question, Search, & Conclude. The focus is on critical reasoning, research methods, and defining intelligence. Many additional topics could also be addressed.

Pre-session: Activate

At the end of session before the Brain Training Module, activate prior knowledge for this new myth by asking students to think about the claim.

For Example: “How many of you have heard the claim that ‘Brain training makes you smarter’ or some version of it? Where? What versions? Do you believe it? Do you think others would believe it? Why would someone believe it?”

Assignment for Day 1
For next time, please find and be ready to share a link to a site that claims, “Brain Training makes you smarter” (or some version of the claim). After finding your site, you should review readings regarding the nature of making claims to prepare for our discussion. In class we will discuss whether and how the sites support their claim and how good you think that evidence is.

During our next three class sessions, we are going to try to answer the following questions:

1. What evidence supports the claim and how strong is that evidence?
2. Is there evidence that does not support the claim and how strong is the opposing evidence?
3. Do we need additional evidence before drawing conclusions about the claim, and if so, how could we collect it?
4. What conclusions are most reasonable given the evidence available so far?

Suggested reading assignments
Students should read or review readings in the textbook or additional sources relating to critical thinking, the nature of the scientific process, problems with “common sense,” and cognitive biases. For example, Chapter 1 in Bensley (2018) reviews making claims and evaluating evidence.

DAY 1: Question

Ask the class to describe the claims they found. What is the actual claim being made? Does the site attempt to justify the claim? Does the site give evidence to support the claim? What types of evidence? How clearly is the evidence related to the claim? Does the site provide a source for the evidence or a way of accessing the source? As students present findings, take the opportunity to link the discussion to material from the reading assignment. Why is it easy to accept claims without evidence (e.g., heuristics and biases)? How does the scientific process provide tools that reduce our biases when evaluating claims?

Sources Students Are Likely to Find to Support the Claim
Two common sources students are likely to find supporting the claim are sites promoting the games themselves and sites promoting health claims regarding the games. We suggest that you let the students find these and other sources on their own, but remind them to look for any references cited in each source, not just the sources themselves.

• Brain Games & Brain Training – Lumosity
• Elevate – Brain Training on the App Store – iTunes – Apple
• Brain Training Centers | LearningRx
• NeuroNation – Scientific Brain Training Exercises
• Brain Exercises, Brain Training, Brain Health – Brain HQ from Posit Science
• 7 Brain Games to Make You Smarter – Prevention
• 15 games that might make you smarter just by playing them – Business Insider
• Which Brain Games Can Make You Smarter? – Big Think
• Students might find this statement
• A Response to “A Consensus on the Brain Training Industry from the Scientific Community”

Supporting student discussion

• The claim: Although students may not see the point at first, it is important that they identify the actual claim being made. In order to know whether the claim is supported by evidence provided, you have to know what the claim is.
  Some sites are likely to make causal claims other sites will make statements that rely on reader assumptions and the reader will need to fill in the missing links. Here’s an example:

“For decades researchers have created tasks that measure cognitive abilities. We transform science into delightful games…engineered to train a variety of core cognitive functions.”

The site does not actually claim improvement in cognitive function but is relying on the reader to infer that use of the product will result in improvement.

• The evidence: What is the evidence supporting the claim? Some sites rely on testimonials such as, “I feel snappy!” Others allude to research, “Researcher at the University of Michigan …..” without providing an actual source. And some sites will provide lists of reported research. Students will likely need help recognizing that even published research needs to be evaluated to determine the relation between claim and support (the focus of Day 2). As students attempt to identify the types of evidence, including research evidence, they should think about (1) why it matters, (2) what types of evidence support what types of claims, and (3) how they would recognize the differences.At this point you could inform students that the Federal Trade Commission fined Lumosity in 2016 for making claims that its games reduce the risk of Alzheimer’s disease and improve achievement at work and school. Ask students why the FTC might have judged Lumosity’s claim as unjustified? What type of research would Lumosity or other sites need to conduct or cite in order to be able to make such claims?

Summary
Students should have generated questions about justification, assumptions, and evidence. What do we need to know to be able to say whether the evidence supports the claims being made? What types of studies would the companies need in order to make their claims?

View the full collection: Busting Myths in Psychological Science

Assignment for Day 2
Given the questions raised, there is a need for a more targeted search. Is there really empirical evidence that brain training makes you smarter or are the claims going beyond the data? In evaluating research, we need to know how research allows us to make and support claims. What is the relation between the design and the claim? How are scientific claims limited by data? There are now many articles that refute the brain training claims by addressing the research evidence. Look for and be ready to discuss one of these articles.

Suggested Reading
Students should read/review readings on the scientific process, research design & methodology, and conceptualizations of intelligence. 

DAY 2:  Search

 Students find sources that critique the brain training claims and that address the evidence “Brain Training” proponents are providing to support their claims.

Here are some sources students are likely to find that refute the “Brain Training” claim.

News Articles

• The Weak Evidence Behind Brain-Training Games – The Atlantic
• Brain Training Doesn’t Make You Smarter – Scientific American
• Think brain games make you smarter? Think again, researchers say – Science Daily
• Brain-Training Apps Won’t Make You Smarter – MIT Technology Review
• Do brain-training exercises really work? -CNN.com
• Brain Games are Bogus

The FTC Ruling

• Press Release
• If you weren’t smart enough to know Lumosity was making bogus claims, the FTC has your back – Los Angeles Times
• The Stanford/Max Plank Consensus Statement
• The full statement:
  • “A Consensus on the Brain Training Industry from the Scientific Community,” Max Planck Institute for Human Development and Stanford Center on Longevity, accessed (March, 18, 2018),

Ask students to share what they have learned about the justification of claims made by brain training sites. Are the claims supported by the data? Why is it important that the research methodology relates directly to the specific claim being made? What are the important characteristics of the research methodology that most require our attention?

Supporting student discussion

• Claims versus evidence. Although much advertising can justify its claims as mere “puffery” that no reasonable person would actually believe (Polar bears don’t really drink Coke), health claims can be held to a different standard (Simons, et al., 2016). As the FTC ruling against Lumosity notes, health claims must be backed by research that rules out alternative explanations. How does this point relate to the importance of limiting claims to what can be claimed by the study design? Why is the sample important? Why do operational definitions of variables matter? Why does the nature of the comparison group matter? Some of the articles students found may have discussed the importance of an “active control group.” In discussing the concept, students might be asked to come up with their own active controls (or discuss some of the controls cited, such as “occasional coffee with the experimenters”!) Why is it important to consider placebo effects? (Foroughi, et al., 2016).
• The nature of scientific claims. Students should be encouraged to think about how the scientific process relies on empirical evidence and recognizes uncertainty. An evaluation of a body of evidence contributes to the degree of confidence in claims. If students have not found the consensus statement on their own, you should direct them to the it. What is the value of consensus in science? What is the importance of reviewing the body of evidence? There is also an alternative consensus statement (www.brainhq.com/longevityresponse), and a response to the alternative consensus (Simons et al., 2016). How do these statements and critiques demonstrate the nature of science? Why does the consensus end with a statement about opportunity costs (the conclusion that any action should account for confidence in the claim in relation to alternatives)?
• Intelligence. A key problem in the “Brain Training” claim is the assumption that intelligence is “one thing” and the brain can be trained like a muscle. Students should be encouraged to discuss the nature of intelligence, overgeneralizing from findings (sample, operational definitions), and what would actually need to be demonstrated to justify the claim that the training will make one “smarter.” The textbook chapter on intelligence is likely to point out the challenges in defining the term “intelligence,” yet also note that there is a consensus view that intelligence involves the ability to learn, remember, and use information, to solve problems, and to adapt to novel situations (Neisser et al., 1996).  The many cognitive abilities contributing to intelligence are not observed directly, but are inferred from performance on various cognitive tasks. The finding that performance on these tasks tends to be related suggests a common underlying ability. However, the finding that there are differences in performance across cognitive tasks suggests their uniqueness. Students should think of the problems with the “strength-training” metaphor, which they are likely to find in articles. The assumption is that training on one cognitive ability (e.g., memory span, speed of processing) will improve performance on other abilities and that training on specific cognitive tasks will improve general and real world problem solving. Point out the challenges researchers have faced in demonstrating near transfer (to similar tasks) and far transfer (to different tasks) and what these challenges mean for “Brain Training” claims.

Summary
Students should have identified the type of research that “Brain Training” sites would need in order to support their claims and whether or not research has provided that evidence.

Assignment for Day 3
Students should complete any additional readings for the final session and be ready to share a justified conclusion. Additional reading for the final session could include sections of the textbook on concepts that the class discussions may have touched on but not covered in depth, such as persuasive communication. However, it’s also important to expose students to alternative proposals for the development of thinking. The final discussion could include life span development, (e.g., how healthy and actively lifestyles relate to cognition); research on learning and cognition, (e.g., how we learn and remember); or findings from research on thinking and problem solving, (e.g., heuristics, roadblocks, and strategies).

DAY 3: Conclusions

Ask the students to state and justify what they think is the best conclusion to draw about the unit’s claim. Remember to point out that, in some cases, final conclusions must await further data and should reflect the strength and quality of evidence now available.

Final discussion should pull together what has already been discussed and provide an alternative recommendation. Additional student readings could be selected on a variety of related issues. You might want to present a brief capstone lecture/discussion that includes some of the following topics:

• Evidence evaluation and why claims seem believable despite their providing weak evidence.
  Providing students with information on types and quality of evidence is not enough to promote use of the information. People find it difficult to use tools for evaluation even when they are aware of the tools (Shah et al., 2016). Although we may know to look for evidence, we may mistake explanations for evidence (Kuhn, 2010). Our tendency to have trouble thinking critically is related to our being “cognitive misers.” We think with heuristics that can mislead when applied to the evaluation of scientific claims (Kahneman, 2011). Our tendency not to think hard is also described in the persuasion literature (Petty & Wegener, 1998). The dual process model of persuasion illustrates how superficial aspects of claims can be persuasive when motivation to engage elaboration is low. Some of these points are reflected in the marketing techniques easily identified in the “Brain Training” sites, including reference to quick fixes, making grand claims, and using scientists as authorities. Research has even shown that the public is persuaded by the mere reference to neuroscience (Weisberg, et al., 2008). Suggestions for improving reasoning about evidence include opportunities to search for and evaluate evidence for claims and counterclaims (Kuhn, 2010).
• The nature of intelligence
  Intelligence is an important concept for introductory students to explore. It has a history of good, bad, and ugly (something to include in discussions of history, stereotyping, ethics), it is multi-dimensional in definition and measurement, and it has social implications (Nisbett, et. al., 2012). Individuals differ from one another in their ability to solve problems, adapt to their environment, and use forms of reasoning (Neisser, et al. 1996); intelligence is the inferred process humans then use to explain these differences (Myers, 2018). Psychometric models of intelligence have described differences by measuring and correlating various abilities and interpreting these relations. Carroll’s (1993) Three-Stratum model describes degree of intercorrelations within and between levels that form a hierarchy. With various interpretations of how to represent and assess intelligent behavior, we cannot make claims that intelligence is the same as any single ability or set of abilities. Abilities relate to one another, not necessarily highly, and individuals may vary on performance at different times and in different domains. Individuals may also use some specific abilities differently than others. As research summarized in some of the articles indicates, improving performance on one task may produce some minimal improvement on a similar task (near transfer), but improvement on a dissimilar task or on adaptive behavior is not a claim about which we can be confident.
• Learning and development
  Given  the limited evidence that playing (and paying for) brain games can increase likelihood of making one “smarter,” are there at least recommendations for improving thinking and even reducing the risk of mental deterioration in later years? Life span developmental research, including work on cognitive reserve (Tucker & Stern, 2011), suggests that a lifetime of engagement relates to positive outcomes in later cognitive abilities. Many sources suggest that engagements in relatively complex environments that provide consistent and deliberate practice with domain-related challenges are the most likely route to expertise (Ericsson, 2006). Studies of thinking and problem solving identify common roadblocks to clear thinking and suggest how to avoid those impediments when thinking. And experimental research in cognition and memory shows the value of strategy use, metacognition, and domain knowledge in memory and thinking (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). So as Elizabeth Stine-Morrow is quoted in the Atlantic (jokingly) saying in response to the question, Does brain training work? “Yes. It’s called school” (see also Ritchie & Tucker-Drob, 2018).

References

Bensley, D. A. (2018). Introduction to critical thinking in everyday life: A guided to effective thinking. New York: Worth Publishers.

Carroll, J. B. (1993), Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4–58.

Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman R. R. (2006). The Cambridge handbook of expertise and expert performance. Cambridge, England: Cambridge University Press.

Foroughi, C. K., Monfort, S. S., Paczynski, M., McKnight, P. E., & Greenwood, P. M. (2016). Placebo effects in cognitive training. PNAS, 113(27), 7470-7474.

Kahneman, D. (2011). Thinking, fast and slow. New York: Macmillian.

Kuhn, D. (2010). Teaching and learning science as argument. Science Education, 94(5), 810-824.

Myers, D. (2018). Psychology, 10th Ed. New York: Worth Publisher.

Neisser, N. et al. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51(2), 77-101.

Nisbett, R. E. et al. (2012). Intelligence: New findings and theoretical developments. American Psychologist, 67(2), 130-159.

Petty, R.E. & Wegener, D. T. (1998). Attitude change: Multiple roles for persuasion variables. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), The handbook of social psychology, pp. 323-390, New York: McGraw-Hill.

Ritchie, S.J. & Tucker-Drob, E.M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, xx, 1-12. https://doi.org/10.1177/0956797618774253

Shah, P. et al. (2016). What makes scientific reasoning so challenging? The Psychology of Learning & Motivation, 66, 251-299.

Simons, D. J., et al., (2016). Do “Brain-Training” programs work? Psychological Science in the Public Interest, 17(3), 103–186

Tucker, A. M. & Stern, Y. (2011). Cognitive reserve in aging. Current Alzheimer Research, 8(3), 1-7.

Weisberg, D. S., Keil, F.C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20(3), 470-477.