Online Research: From Funding to Data Collection

Be creative and keep methodology in mind: two key takeaways from the APS webinar Online Research: Tools and Techniques. Robert Ariel (Virginia Wesleyan University) and Joshua VanArsdall (Elmhurst University) and shared their experiences and tips for conducting research online, and Elise Rice, a program officer at the National Institutes of Health (NIH), discussed how online research may impact funding options.

Taking Experiments From the Lab to Online

Ariel is a cognitive scientist whose research focuses on metacognition, self-regulation of learning, and the application of cognitive psychology to education. He outlined challenges researchers may encounter while trying to adapt experiments to online settings, such as needing to use specialized software that participants cannot not access on their own computers. For such challenges, he recommended the readily available solution of using a videoconferencing platform like Zoom, which can allow subjects to see and interact with the researcher’s computer. Despite potentially slowing down data collection (only one participant at a time) and not being appropriate for studies that involve measuring reaction times, Zoom “gives subjects something that approximates the lab experience,” he said. Another advantage is that Zoom doesn’t require researchers to code new experiments or set up servers and databases. Asked about possible security issues during experiments, he added that experimenters can regain immediate control of their computer simply by clicking their mouse.

For researchers who wish to collect data in larger batches, Ariel recommended programming tasks using free tools that require minimal coding knowledge, or avoiding the need for servers by developing online experiments. He gave a tutorial about how to use jsPsych, a library of commonly used tasks already coded and ready to be used in experiments, and Cognition.run, an online experiment platform that uses the jsPsych architecture to host experiments and store data online. 

Using Data Available Online

VanArsdall, who studies the processes at the intersection of cognitive psychology, social cognition, and memory, has mastered the use of existing online data to answer research questions. Content analysis (quantitative analysis) helps him identify the frequency of hashtags or the occurrence of different words, for example, and thematic analysis (qualitative analysis) helps him parse data publicly available on social media platforms. VanArsdall also underscored the importance of rethinking research questions to explore how they can be answered by publicly available data, and thoroughly defining the phenomenon that will be measured. He shared the example of a colleague who studies parenting and collects data from “mom blogs”—an approach that provides access to a lot of data that is often subjective. “You’re only seeing what people are presenting to you,” VanArsdall said. “Keep context in mind.”

More tips on using online data:

• Keep in mind methodological and ethical considerations, such as the fact that most online data was not initially intended to be used in research and may not fully represent the individuals behind them. 
• Do not make causal inferences, as you would in a laboratory experiment. You can correlate variables, but you cannot manipulate variables when analyzing data already available.
• Consider whether random sampling is possible and how it will affect data interpretation, and think about linking to relevant data, such as historic crime, weather, or laws and policies, as well as to contingencies or time (when were tweets posted? in response to what?). 
• Be careful not to misrepresent data, and always seek Institutional Review Board (IRB) approval—even for publicly available data.

Applying for Funding for Online Research

Rice studied social psychology and now serves as a program officer for the Behavioral and Social Sciences Research Program at the NIH’s National Institute of Dental and Craniofacial Research. She talked about funding considerations for online research, whether for primary data collection (i.e., experiments) or secondary data analysis (i.e., data publicly available). The foremost step toward getting funding? Connect with a program officer, just as you would for in-person research funding.

For primary data collection, consider whether your topic and methods are within the scope of the funding opportunity and whether your methods and rationale are scientifically sound. For secondary data analysis, the funding mechanisms may differ but the principles are similar: You still must ensure that “your proposed analyses and inferences are scientifically sound,” said Rice. Moreover, instead of focusing on the online aspect of your research, focus on the soundness of your methods and the importance of your research questions.

View a recording of the Research Online: Tools and Techniques webinar and the full list of past and upcoming APS webinars here.