Innovative Methods and Scientific Progress

We are living in a time of unprecedented methodological innovations in psychology. Neuroscience has come onto center stage in the field. Epigenetics, an area of science that examines how environments influence genes, is no longer a mere theoretical possibility. Big data is increasingly common, which has forced rapid progress in advanced statistics. And many of us are increasingly aware of the need to sample subject populations widely and broadly, both to represent everyone around the globe in our science and, simultaneously, to enrich and improve our theories. Our time, I believe, is remarkable because of these methodological innovations. Or is it?  

To explore this question, it is worth reflecting on what methodological innovations in science might do to both the field and the scientists working in it. Why are methodological innovations so essential? And why are we so fortunate to work as scientists at this time of methodological innovations?  

Prior analyses on this matter in science in general (e.g., Kuhn, 1962) and psychology in particular (e.g., Greenwald, 2012) undoubtedly carry some truth. Methodological innovations provide scientists with fresh ways of addressing the mysteries of nature (the human mind included) one by one, inviting and motivating new ways of thinking. As Tony Greenwald (2012) put it, “there is nothing so theoretical as a good method” (p. 99). Further, new methods sometimes invite Kuhnian paradigm shifts by producing data that do not fit existing paradigms. To this, however, I want to add emotional and social dimensions. To make this point, I first want to invite readers to revisit one scene from President Biden’s inauguration on January 20. There is something in there to be learned about how science might evolve. 

See all columns by Shinobu Kitayama

“The Hill We Climb” 

The occasion was amazing in many ways. After the chaos and confusion that permeated the preceding weeks and months—most notably, the riot on January 6 targeting the U.S. Capitol—the ceremony was a welcome reminder that there remains light to shine over this country. The new president’s address was a forceful plea for national unity. Lady Gaga pulled off a passionate rendition of the national anthem. However, the highlight of the day was claimed by Amanda Gorman, a 22-year-old Black female poet who wrote and recited “The Hill We Climb”—a powerful manifesto for redemption and hope for the future. 

The Hill We Climb” brings forward multiple insights for our science—where it stands and how we shall try to further it in the years to come.

After witnessing Gorman’s brilliant performance, I paused to consider what it was that struck everyone’s minds and captured their hearts. As I see it, three components, presented in perfect balance, made her poem so moving. 

Expanded consciousness. First and foremost, her words were powerful. They expanded the consciousness of everyone who listened to them. “A skinny Black girl descended from slaves” who “can dream of becoming president” is now reciting the poem for one. Here, the dark history of slavery and racism is being juxtaposed with the ideal of the American Dream. The verse further reminds everyone that the country, although currently in trouble, is not “broken but simply unfinished.” Remarkably, such complex ideas are weaved into a seamless stream of unpretentious lines. When we digest the meaning that comes out of it, we are stunned by its scope. It encompasses the entire history of the United States. It shows both dark and bright elements of the nation in a single panorama.  

The emotion of awe. This panoramic view of the country’s history defines the present moment, where we stand today as individuals and as a nation, while casting light into the future. By opening up this panoramic view, Gorman’s words reach out to our hearts and souls. They evoke awe, which for many of us is exclusively reserved for natural wonders of massive scale. We are stunned and left motionless by the range of both the events that have occurred over this country’s history and the people who lived through them. The weight of history is viscerally felt. In a single brushstroke, Gorman then placed the new president’s inauguration in that history. As Dacher Keltner and Jon Haidt (2003) have proposed, the emotion of awe makes you feel small, bringing into sharp relief something that transcends any single individual and defines the moment. 

Shared mission. Just as important, the awe that was induced was socially shared: Everyone on the scene, so it would appear, had this sense of awe and wonder, whether they were attending the occasion in person or watching it on TV or online. Once shared, awe fosters a collective sense of mission (Rimé, 2009, 2018). The United States has a long, problematic history, but it was founded with a purpose. It has faced difficulties, yet, as Gorman puts it, it is “simply unfinished.” We all “lift our gazes not to what stands between us, but what stands before us.” The sense of mission conveyed and shared among all those who watched her perform afforded a decisive moment of unity. This moment helped people transcend their personal identities to form a collective identity—a phenomenon addressed under the rubric of “identity fusion” (Swann et al., 2012). No matter how transient it might have been, a moment like this would be needed to get the social experiment of America started all over again.  

Scientific progress: What is it, and how does it happen? 

Gorman’s “The Hill We Climb” brings forward multiple insights for our science—where it stands and how we shall try to further it in the years to come. For example, the replication crisis has reminded us, as Gorman did for the country, that the project of psychological science is still “unfinished.” Also, her resolve to pursue diversity would resonate strongly with us as a field (see my December 2020 column on systemic racism). However, above all these insights, I want to make one fundamental point about this column’s central theme: What might innovative methods do to our science? Remember that Gorman used her words to expand the nation’s consciousness, which evoked awe and wonder, defining the crucial moment of national unity. I suggest that at their best, innovative methods also have these effects. Such methods will expand the conceptual scope of our field. This expansion will evoke awe and wonder, which will, in turn, bring us together on a shared mission of seeking the truth about the human mind.  

Reflecting on the President’s Column in the March/April issue of the Observer, NPR science correspondent Joe Palca, who earned a PhD in psychology from the University of California, Santa Cruz, offers his unique take on the evolution of methodologies in science, from landers on Mars to big data and psychological science. 

Scientific evolution. To be more concrete, consider a bit of the history of persuasion research. A cursory look at this literature shows a noteworthy transformation of the field over the years. Around World War II, the stimulus-response learning theory, championed by some luminaries, such as Clark Hull and Kenneth Spence, was a dominating force in psychology. Influenced by this theory, some social psychologists, most prominently Carl Hovland and colleagues (1953), hypothesized that for messages to impact attitudes (i.e., to be persuasive), they must be learned. The main method available back then to test this idea was to examine how well people remembered persuasive messages. To the scholars’ surprise, data based on this method revealed no relationship between memory for a message and its effect on attitudes. This observation presented an empirical anomaly, which was an impetus for various innovations. One such innovation involved the notion of “cognitive response.” Greenwald (1968) argued that it is not the information contained in a message but the cognitive elaborations performed on it that count. A critical methodological innovation was to have subjects list all thoughts that occurred to them while listening to and comprehending persuasive messages. Indeed, this method resolved the early anomaly by showing that the thoughts listed retrospectively predicted the observed attitude change.  

Subsequently, researchers in this area have advanced the additional insight that cognitive elaboration does not account for the persuasive effect of a messenger’s attractiveness or expertise. This realization led to versions of dual-process theory in this area (e.g., Petty & Cacioppo, 1986). Combined with other related ideas, these theories have coalesced into a broader notion of “thinking, fast and slow,” popularized by Daniel Kahneman (2011) with his book of the same title.  

Lessons from history. It bears emphasis that the target of the investigation, persuasion, has remained the same. But what researchers saw in this phenomenon varied tremendously depending on the methods brought to bear on it—in our example, recall versus thought-listing. Very different sceneries may unfold in front of you depending on what methodological lenses you wear. Thomas Kuhn (1962) famously saw the evolution of science as a series of revolutions. But no such revolution has happened in the persuasion field, and yet progress is visible. Perhaps his analysis is reserved for seismic changes that happen only once in any given historical time (as in the Copernican revolution). If so, Kuhn’s analysis must surely be supplemented with something more pertinent to science in action on a day-to-day basis. His analysis is often contrasted with more dominant views of scientific progress as linear and steady, but those do not seem right, either.  

Progress in the persuasion field is evident, but it is based on repeated, incessant efforts to get something more systematic by using many different methods. There are a lot of trials and errors in the process. Equally important, observations based on a newer method rarely refute or falsify those based on an earlier one: In our example, cognitive elaboration (thought-listing) would require, and thus presuppose, the learning (recall) of persuasive messages. The new method adds depth and conceptual richness to the content matter to encourage more comprehensive theorizing. The most current, up-to-date analysis incorporates prior analyses.  

Progress in the persuasion field is evident, but it is based on repeated, incessant efforts to get something more systematic by using many different methods. There are a lot of trials and errors in the process.

Moreover, if you think all this evolution of science is a matter of pure intellect, as is typically conceived in the philosophy of science, your view would be too myopic. In reality, scientific progress is deeply emotional. Many of us—active researchers—are excited, stunned, dismayed, moved, and, if we are lucky, deeply satisfied by some resolution after a long period of agony and cognitive dissonance. Clearly, some great scientists in history saw something bigger and vaster than anything mortals have at their disposal behind what they studied. Remember Einstein, who noted that “God does not play dice.” It is also clear that none of this happens in a social vacuum. Increasingly, science is a highly social endeavor. Today, dual-process theory is a linchpin that unites multiple schools of thought that address different areas with similar ideas. For our science to be productive, we must bring together experts of all disciplinary and methodological stripes in a dynamic whole. Collaboration is an integral part of science. 

Why innovative methods are crucial. The best metaphor I have to describe this state of affairs compares scientists to the crew members of a spaceship circling Earth in its orbit. Science may not proceed like an arrow. Progress is not linear. Nor does it advance only through revolutions. Science evolves on the shoulders of preceding researchers through social engagement with colleagues. This evolution takes a spiral movement: Progress becomes possible through circling around the same object. As the spaceship metaphor makes apparent, every time we circle the same object—say, persuasion—we try out new observation methods. These methods serve as new lenses through which to observe the object. We make very different observations as a result, even though the target object remains identical. These observations invite new questions, which in turn may motivate newer methods. Every time we circle the object, we deepen our understanding and broaden our conceptualization; eventually, we approximate the theoretical framework that can accommodate all new observations. This circular effort gradually moves science forward. That progress, however, may become visible only when seen from afar.  

As we observe the same object through different methodological lenses, we will be amazed by new ways in which the object presents itself. This amazement translates into a sense of awe and wonder. Many readers of my generation may remember the magical power of enchantment in Carl Sagan’s (1980) exposition of celestial systems. I should add that sometimes, the innovative methods themselves are awe-evoking. For example, remember the first time you saw a colorful heat map of the brain on your computer screen. Science can be “awe-some,” as Piercarlo Valdesolo and colleagues (2017, p. 215) observed. The awe will be augmented as the method is repeatedly tried and variously applied. Further, the awe will be shared by an increasingly large number of researchers in the lab. Upon further repetition and application, the sense of excitement may spread to scholars outside the lab. It may eventually reach many in the field at large. The emotions of awe and wonder may cultivate many scientists’ imaginations. They also induce in-group cooperation (Piff et al., 2015) and “identity fusion” (Swann et al., 2012). The resulting collaboration will forge a commonly shared sense of mission and direction for the future. When all this happens, a new area of research will have emerged. Ultimately, the field-forming force of awe and wonder may even influence popular  
conceptions of the issue at hand. 

Three innovations that may transform psychological science 

The preceding discussion of scientific progress leads me to my next questions: Where do we stand today in psychological science? What methodological innovations will expand the scope of our understanding of the human mind? What methods could bring about awe and help us renew the sense of our mission to further the understanding of the mind? I think everyone has informed guesses. These guesses may differ, but I suspect the following three are common.  

Neuroscience. My first pick is neuroscience, broadly conceived to include not only brain imaging methods, such as MRI and electroencephalogram, but also various methods in genetics, epigenetics, and neuroendocrinology. The use of brain imaging methods in particular has created a new field of cognitive neuroscience. However, the impact of neuroscience can be felt everywhere, from clinical and developmental psychology to social and cultural psychology. A key idea is neuroplasticity. Experience can and does change your brain (and body), and through those changes, it changes who you are and how you think and feel. All this happens within the constraints put forward by our evolutionary past. Importantly, however, the brain is now conceptualized as a far more open system than it once was. As I discussed in my last column, the human brain (and thus human psychology) is closely interdependent with its environment. Moreover, it is now empirically tenable to specify how life experiences affect the activation or silencing of genetic codes responsible for creating brain networks and the connections among such networks. Pioneered by Michael Meaney (2001) and Steve Cole (2014), among many others, this new approach and all insights that come with it carry the potential to transform psychological science.  

Big data. With the advent of super-powered computers, the amount of available data has increased exponentially. Brain network analysis is increasingly common, supplementing or even replacing more traditional regions-of-interest analysis. Genome-wide scanning is becoming more commonplace. And we must not forget physiological and neuroendocrine data. The online collection of data from wearable devices, such as watches and wristbands, that track physiological signals is allowing us to examine mind-body connections in new ways and at levels never seen before. Just as important, a similar expansion is evident in social domains. Facebook and Twitter have supplied tons of data for careful analysis. For example, automated computer-mediated coding of millions of tweets is now among the tools added to our science. Ecological experience-sampling methods carry the potential to test theories of, say, emotion in ecologically valid settings. When all these developments are brought together, one can do some highly innovative things. For example, one can use wearables that monitor physiological data to assess the effect of experimental interventions by randomly assigning the moments that meet certain on-line criteria (e.g., momentary heart rate) to the experimental (vs. control) condition (Nahum-Shani et al., 2020). And we should not forget machine learning, which has emerged as a powerful method for data-driven inductive reasoning and hypothesis generation (Sheetal et al., 2020). Altogether, the big data has changed the field both rapidly and massively, offering tremendous opportunities for everyone interested in the mind and body at work in the real world. 

Diverse samples. This last one may not come across as a methodological innovation to many in the field. But it is! Testing existing theories across divergent samples can immediately challenge those theories. In my career, an initial failure to replicate standard findings—say, the fundamental attribution error, self-serving bias, or cognitive dissonance—in Asia always forced a moment of reckoning, which was turned into new research opportunities (Markus & Kitayama, 1991). It is now abundantly clear that there is substantial psychological diversity across the globe. Hence, theories and observations based on WEIRD (Western, educated, industrial, rich, and democratic) samples can, more often than not, be myopic and, worse, ethnocentric. To address this issue, scholars have traditionally used international surveys. However, such surveys must be supplemented with experimental procedures, as well as neuroscientific, physiological, genetic, or epigenetic measurements. Some promising efforts are underway (Reinecke & Gajos, 2015). Just as the exploration of psychological mechanisms within the brain requires close collaboration with those who know the brain well (i.e., neuroscientists), exploring such mechanisms across societies demands close collaboration with sociologists, anthropologists, and other social scientists. Cross-disciplinary fertilization will for sure promote further theoretical advancement and enrich psychological theories. 

Concluding thought 

In “The Hill We Climb,” Gorman used “hill” in two disparate senses. First, and most literally, she referred to Capitol Hill, where she stood along with all those who were to govern the nation. However, more metaphorically, she also referred to people’s struggles, both past and present, to realize America’s ideals, including diversity and equality. This juxtaposition of the hill that governs this country and the hill many oppressed have tried to climb foreshadowed the awe she was to inspire in the audience. Her poem expanded the consciousness of everyone who listened, struck them with new insight about who we are and, more importantly, who we want to be, and produced a common sense of mission to the future.  

I cannot help but feel an equally strong hope for our field. Psychology has struggled to understand this mysterious thing called “mind” for so long. However, new methods are now available, and others are on the horizon. They promise to expand the consciousness of the field. When they do, they will undoubtedly foster the emotions of awe and wonder, which will be widely shared by all those seekers of the truth about the human mind. Methodological innovations will help our science grow and climb a hill of its own. This prospect may no longer be merely aspirational. It may be within our reach.

Feedback on this article? Email [email protected] or scroll down to comment.

References 

Cole, S. W. (2014). Human social genomics. PLoS Genetics, 10(8). https://doi.org/10.1371/journal.pgen.1004601 

Greenwald, A. G. (1968). On defining attitude and attitude theory. In A. G. Greenwald, T. Brock, & T. Ostrom (Eds.), Psychological foundations of attitude. Clark University Press. 

Greenwald, A. G. (2012). There is nothing so theoretical as a good method. Perspectives on Psychological Science, 7(2), 99–108. https://doi.org/10.1177/1745691611434210

Hovland, C., Janis, I., & Kelley, H. (1953). Communication and persuasion: Psychological studies of opinion change. Yale University Press. 

Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux. 

Keltner, D., & Haidt, J. (2003). Approaching awe, a moral, spiritual, and aesthetic emotion. Cognition and Emotion, 17(2), 297–314. https://doi.org/10.1080/02699930302297 

Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press. 

Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98(2), 224–253. https://doi.org/10.1037/0033-295X.98.2.224 

Meaney, M. J. (2001). Maternal care, gene expression, and the transmission of individual differences in stress reactivity across generations. Annual Review of Neuroscience, 24(1), 1161–1192. https://doi.org/10.1146/annurev.neuro.24.1.1161 

Nahum-Shani, I., Almirall, D., Yap, J. R. T., McKay, J. R., Lynch, K. G., Freiheit, E. A., & Dziak, J. J. (2020). SMART longitudinal analysis: A tutorial for using repeated outcome measures from SMART studies to compare adaptive interventions. Psychological Methods, 25(1), 1–29. https://doi.org/10.1037/met0000219 

Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 19, 124–205. 

Piff, P. K., Dietze, P., Feinberg, M., Stancato, D. M., & Keltner, D. (2015). Awe, the small self, and prosocial behavior. Journal of Personality and Social Psychology, 108(6), 883–899. https://doi.org/10.1037/pspi0000018 

Reinecke, K., & Gajos, K. Z. (2015). LabintheWild: Conducting large-scale online experiments with uncompensated samples. In Proceedings of the 18th ACM Conference on Computer-Supported Cooperative Work and Social Computing (pp. 1364–1378). Association for Computing Machinery. 

Sagan, C. (1980). Cosmos. Random House. 

Sheetal, A., Feng, Z., & Savani, K. (2020). Using machine learning to generate novel hypotheses: Increasing optimism about COVID-19 makes people less willing to justify unethical behaviors. Psychological Science, 31(10), 1222–1235. https://doi.org/10.1177/0956797620959594 

Swann, W. B., Jetten, J., Gómez, Á., Whitehouse, H., & Bastian, B. (2012). When group membership gets personal: A theory of identity fusion. Psychological Review, 119(3), 441–456. https://doi.org/10.1037/a0028589 

Rimé, B. (2009). Emotion elicits the social sharing of emotion: Theory and empirical review. Emotion Review, 1(1), 60–85. https://doi.org/10.1177/1754073908097189 

Rimé, B. (2018). Comment: Social integration and health: Contributions of the social sharing of emotion at the individual, the interpersonal, and the collective level. Emotion Review, 10(1), 67–70. https://doi.org/10.1177/1754073917719330

Valdesolo, P., Shtulman, A., & Baron, A. S. (2017). Science is awe-some: The emotional antecedents of science learning. Emotion Review, 9(3), 215–221. https://doi.org/10.1177/1754073916673212