Mixed Methods Research

Traditionally, there are three branches of methodology: quantitative (numeric data), qualitative (observational or interview data), and mixed methods (using both types of data). Psychology relies heavily on quantitative-based data analyses but could benefit from incorporating the advantages of both quantitative and qualitative methodologies into one cohesive framework. Mixed Methods (MM) ideally includes the benefits of both methods (Johnson, Onwuegbuzie, & Turner, 2007): Quantitative analyses employ descriptive and inferential statistics, whereas qualitative analyses produce expressive data that provide descriptive details (often in narrative form) to examine the study’s research objectives. Whereas quantitative data may be collected via measures such as self-reports and physiological tests, qualitative data are collected via focus groups, structured or semistructured interviews, and other forms (Creswell, 2013).

MM hypotheses differ in comparison with solely quantitative or qualitative research questions. Not only must the quantitative and qualitative data be integrated, but the hypotheses also must be integrated. MM practitioners promote the development of a theory-based set of three hypotheses. Hypotheses should be conducted a priori and be both logical and sequential research questions (for more information, see Onwuegbuzie & Leech, 2006). Specialists encourage researchers to construct three separate types of hypotheses for an MM research project. There can be more than three hypotheses but there must be at least one of each type. The first hypothesis should be quantitative and the second should be qualitative. The third hypothesis will be an MM hypothesis.

Integration of these data is often complex, even when there is a strong theoretical rationale for doing so. Data integration occurs when quantitative and qualitative are combined in a data set. There are multiple ways for this to occur, including triangulation, following a thread, and the mixed methods matrix (see O’Cathain, Murphy, & Nicholl, 2010, for a brief review). Yet understanding the overall reasoning for using MM and how to best combine the approaches in practice can help lessen the challenge of MM data integration (Bryman, 2006).

Types of MM Research

• There are dozens of MM designs, but for the purpose of this article, six MM designs will be presented:
• The sequential explanatory method employs two different data-collection time points; the quantitative data are collected first and the qualitative collected last.
• The sequential exploratory design is best for testing emergent theory because both types of data are interpreted during the data integration phase.
• The sequential transformative approach has no preference for sequencing of data collection and emphasizes theory.
• Concurrent triangulation is the ideal method for cross-validation studies and has only one point of data collection.
• The concurrent nested design is best used to gain perspectives on understudied phenomena.
• The concurrent transformative approach is theory driven and allows the researcher to examine phenomena on several different levels.

Strengths and Challenges of MM Research

An MM approach is helpful in that one is able to conduct in-depth research and, when using complementary MM, provide for a more meaningful interpretation of the data and phenomenon being examined (Teddlie & Tashakkori, 2003).  Another strength of MM is the dynamic between the qualitative and quantitative portions of the study. If the design is planned appropriately, each type of data can mirror the other’s findings, so the methodology can benefit many types of research. However, interpreting data using the MM framework can be complicated and time intensive given that the data and interpretations are often abstract. Additionally, conducting MM research requires training and mastery of the methodology, so there can be a learning curve for researchers who traditionally use only quantitative or qualitative methods. Sticking to the theory-based and evidence-based designs will aid in your understanding and interpretation of the data.

Qualitative Data Analysis

Qualitative coding is a multistep process that includes different types of analyses depending on the nature of your data. Codebooks are important before, during, and after qualitative coding due to the detailed nature of the qualitative data. It is also important to know your expected codes and themes in order to promote interrater reliability (Hruschka et al., 2004). Expected codes are based on the theoretical foundation of your project. I suggest including the expected codes and themes in your codebooks. As previously mentioned, research designs involving this type of data can vary greatly, but in general, the following is a framework of how to conduct a thematic data analysis: Know your data inside and out, generate codes, search for themes, and review themes with a research team (Braun & Clarke, 2006). For more detailed instructions on conducting a qualitative analysis, please refer to last month’s Student Notebook article (Heydarian, 2016).

Lessons Learned

From the start, the researcher or research team must have a clear idea of their resources and the pros and cons of each method. Researchers also must be flexible. I am interested in examining the factors that compose seeking health information online. To investigate this topic, I developed an online, two-part study. Information obtained from qualitative prompts was used to inform the development of a scale measuring
health-information-seeking behavior online. The first study used MM, and the data collection occurred on Amazon Mechanical Turk, a marketplace where researchers can post their available studies. Potential participants are paid a small fee, and data collection usually is completed in less than a week. I expected to conduct magnitude coding — a type of qualitative coding that evaluates the emphasis of content — but instead I had to choose a more appropriate type of coding because the participants provided extremely brief responses.

In closing, the design of your study (quantitative, qualitative, or MM) should align with your training and your research objectives. MM has the potential to bring your research to the next level by combining the strengths of quantitative and qualitative methodologies.

Suggestions for Conducting MM Research

Be proficient in MM research by keeping up to date with the latest techniques, software, textbooks, and manuals.

Think “outside the box” and consider other data-analytic approaches that are not used in your field.

Choose the research design that best fits the hypotheses, and know the assumptions and limitations of that design.

Incorporate figures and tables into your qualitative codebook to deepen the conceptualizations for the coders and provide a few examples of already coded data in order to provide thorough instructions.

Create and use summary statements for each participant to help with the abstract portion of the analyses. Summary statements should be a few sentences that describe the participant’s statement and provide an overall gist of the available qualitative information.

References

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. doi:10.1191/1478088706qp063oa

Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6, 97–113. doi:10.1177/1468794106058877

Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage Publications.

Heydarian, N. (2016). Developing theory with the grounded-theory approach and thematic analysis. Observer, 29(4), 38–39.

Hruschka, D. J., Schwartz, D., John, D. C. S., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods, 16, 307–331. doi:10.1177/1525822X04266540

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112–133. doi:10.1177/1558689806298224

O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ, 341, c4587. doi:10.1136/bmj.c4587

Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking research questions to mixed methods data analysis procedures 1. The Qualitative Report, 11, 474–498.

Teddlie, C., & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 3–50). Thousand Oaks, CA: Sage Publications.