"Mixed methods design"

Clarification of a complex research design – Notes crafted by Prof CH van Heerden

In recent years, more social and health sciences researchers have been using mixed-methods designs for their studies. By definition, mixed
methods is a procedure for collecting, analysing, and “mixing” or “integrating” both quantitative and qualitative data at some stage of the research
process within a single study for the purpose of gaining a better understanding of the research problem (Tashakkori and Teddlie 2003;
Creswell 2005). Integration is the distinguishing factor that separates mixed methods from all other methodologies. Mixed methods is not simply
gathering quantitative and qualitative types of data: It involves linking or mixing the two databases (Creswell, 2016).

Why do you want to follow a mixed methods research design? Read widely and in-depth about mixed methods research design.
Does your research problem or question merit a mixed methods study - Choose a method based on your Molina Azorín,J, M and Cameron, R. “The Application of Mixed Methods in Organisational
problem statement. Research: A Literature Review” The Electronic Journal of Business Research Methods
When is mixed methods suitable for a problem or question - When qualitative research or quantitative Volume 8 Issue 2 2010 (pp.95-105), available online at www.ejbrm.com
research is insufficient to fully understand the problem
Consult the Journal of Mixed Methods Research (Scopus, ISI, and Clarivate Analytics)

Schoonenboom, J. & Johnson, JB. (2017) How to Construct a Mixed Methods Research Design. Köln Z Soziol. (Suppl 2) 69:107–131. DOI Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking
10.1007/s11577-017-0454-1 Research Questions to Mixed Methods Data Analysis
Cameron, R. (2011). Mixed Methods Research: The Five Ps Framework” The Electronic Journal of Business Research Methods. 9(2), 96-108, Procedures 1. The Qualitative Report, 11(3), 474-498.
available online at www.ejbrm.com Retrieved from
https://nsuworks.nova.edu/tqr/vol11/iss3/3

Framing the research question Sampling

1. Definitions of terms
2. Underlying theory/theories
3. Important and appropriate claims,
conclusions, and findings about the scientific
constructs you are investigating
4. Explore calls for follow-up studies relevant
to your project
5. Describe gaps you notice in the literature
6. Identify scholarly disagreement about the
constructs you are investigating
Draw a diagram of your proposed Mixed Method Design
Research design starts with the research problem and the research question(s)
Teddlie, C. and Yu, F. (2007) Mixed Methods Sampling:
A Typology with Examples. Journal of Mixed
To answer a research question
Methods Research, 1, 77-100.
What information is already in place? / What information do we require?
https://doi.org/10.1177/2345678906292430
What sort of information can we use to quantify the problem?
Onwuegbuzie, A. J., & Leech, N. L. (2007). Sampling
Designs in Qualitative Research: Making the
Research script - Research questions dictate the type of research design used, the Sampling Process More Public . The Qualitative
sample size and sampling scheme employed, and the type of instruments administered Report, 12(2), 238-254. Retrieved from
as well as the data analysis techniques (i.e., statistical or qualitative) used. https://nsuworks.nova.edu/tqr/vol12/iss2/7
Onwuegbuzie, A. J., & Collins, K. M. (2007). A Typology
Observe: Observe: of Mixed Methods Sampling Designs in Social Science
Zohrabi, M. (2013). Mixed Method Shenton, AK. (2004). Research .The Qualitative Report, 12(2), 281-316.
Research: Instruments, Validity, Strategies for ensuring Retrieved from
Reliability and Reporting Findings. https://nsuworks.nova.edu/tqr/vol12/iss2/9
trustworthiness in
Theory and Practice in Language
qualitative research
Studies, 3(2), 254-262, February
2013..doi:10.4304/tpls.3.2.254- projects. Education for
262. Information, 22( 2), 63-75.

O'Cathain, A., Murphy, E. & Nicholl, J. (2008) The quality of mixed methods studies in health services research. J Health Serv Res
Policy, 13, 92-8
 The following should be clearly depicted (illustrated) and substantiated in the research proposal

1. A mixed methods research design script

Example 1.1 - A convergent mixed methods design

This mixed methods study will address ______________ (overall content-aim of the study). A convergent mixed methods design will be used, and it is a
type of design in which different but complementary data will be collected on the same topic.
In this study, ___________ (quantitative instruments) will be used to test the theory of ____________ (the theory) that predicts that _____________
(independent variables) will influence ______________ (positively, negatively) the __________ (dependent variables or outcomes) for __________
(participants) at __________ (the research site).
Concurrent with this data collection, qualitative _____________ (type of qualitative data, such as interviews) will explore ____________ (the central
phenomenon) for ______________ (participants) at _____________(site).
The reason for collecting both quantitative and qualitative data is to bring together the strengths of both forms of research to _________________ (e.g.,
compares results, validate results, corroborate results).

Example 1.2 - An explanatory mixed methods design

This study will address ______________ (content-aim of the study). An explanatory mixed methods design will be used, and it will involve collecting
qualitative data after a quantitative phase in order to explain or follow up on the quantitative data in more depth. In the first quantitative phase of the
study, ____________ instrument data to be collected from _____________ (participants) at ___________ (research site) to test _______ (the theory) that
explains why ____________ independent variables) relate to the ____________ (dependent variables).
The second qualitative phase will be conducted because ______________ (intent of the qualitative phase). In this exploratory follow-up, the
______________ (central phenomenon) will be tentatively explored with ___________ (participants) at _____________ (the research site). The reason for
the exploratory follow-up is to ______________ (e.g., to help explain or build upon initial quantitative results).

Example 1.33 - Exploratory sequential design

This study addresses _______________ (content-area of the study). The purpose of this exploratory sequential design will be to ________________ (e.g.,
develop and test an instrument, generate a taxonomy).
The first phase of the study will be a qualitative exploration of ________________ (the central phenomenon) by collecting _________________ (types of
data) from _________________ (participants) at _____________ (the research site).
The second quantitative phase will follow up on the qualitative phase for the purpose of __________(intent of this follow-up). In the quantitative phase,
___________ (instrument data) will be collected from ____________ (participants) at ______________ (research site).
Quantitative research questions/hypotheses will be formulated after the completion of the initial qualitative phase. The reason for collecting qualitative
data initially is that ________________ (e.g., instruments are not available, variables are not known, there is little guiding theory).
 The following should be clearly depicted (illustrated) and substantiation in the research proposal

2. Population and sampling

Design 1
Design 2 Teddlie, C. and Yu, F. (2007) Mixed
Design 3 Methods Sampling: A Typology with
Examples. Journal of Mixed
Design 4 Methods Research, 1, 77-100.
https://doi.org/10.1177/234567890
6292430
Onwuegbuzie, A. J., & Leech, N. L.
(2007). Sampling Designs in
Qualitative Research: Making the
Sampling Process More Public . The
Qualitative Report, 12(2), 238-254.
Retrieved from
https://nsuworks.nova.edu/tqr/vol1
2/iss2/7
Onwuegbuzie, A. J., & Collins, K. M.
(2007). A Typology of Mixed
Methods Sampling Designs in Social
Science Research .The Qualitative
Report, 12(2), 281-316. Retrieved
Design 5
from
Design 6 https://nsuworks.nova.edu/tqr/vol1
2/iss2/9
Design 7

Design 8
 The following should be clearly depicted (illustrated) and substantiation in the research proposal

3.

The following should be clearly depicted (illustrated) and substantiation in the research proposal

4.
A mixed methods script – problem statement and objectives
Three questions (from overall aim)
• Quantitative questions or
hypotheses
• Qualitative questions
• A mixed methods question
Data collection

Integration / Merging

Visual displays
Creswell’s view Creswell, John W., "Steps in Conducting a Scholarly Mixed Methods Study"
(2013). DBER Speaker Series. 48.
http://digitalcommons.unl.edu/dberspeakers/48
Definition Cameron, R. (2011). Mixed Methods Research: The Five Ps Framework” The
This paper provides a brief overview of some of the more common Electronic Journal of Business Research Methods. 9(2), 96-108, available online at
definitions of mixed methods research and methodology before www.ejbrm.com
introducing the conceptual framework of the Five Ps of mixed
methods research. The Five P framework will be used to structure
an exploration of some of the key challenges facing those who
choose the innovative path of mixed methods research and some of
the key areas for capacity building. The Five Ps include: Paradigms;
Pragmatism; Praxis; Proficiency; and Publishing.
Eight key considerations in planning and undertaking mixed methods Halcomb, E. & Hickman, L. (2015). Mixed methods research. Nursing Standard:
research, namely; promoting excellence in nursing care, 29 (32),
1) examine the rationale for using mixed methods; 41-47.
2) explore the philosophical approach;
3) understand the various mixed method designs;
4) assess the skills required;
5) review project management considerations;
6) plan and justify the integration of qualitative and quantitative aspects;
7) ensure that rigour is demonstrated;
8) disseminate mixed methods research proudly.

Figure 1. Reasons for using mixed methods research (page 5 of 17)

Framing the Literature Review

Research Questions
First, we discuss the ways that the goal of the study, the research Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking Research Questions to Mixed
objective(s), and the research purpose shape the formation of research Methods Data Analysis Procedures 1.The Qualitative Report, 11(3), 474-498.
questions. Retrieved from https://nsuworks.nova.edu/tqr/vol11/iss3/3
Second, we compare and contrast quantitative research questions and
qualitative research questions.
Third, we describe how to write mixed methods research questions, which
we define as questions that embed quantitative and qualitative research
questions. Finally, we provide a framework for linking research questions
to mixed methods data analysis techniques. A major goal of our framework
is to illustrate that the development of research questions and data analysis
procedures in mixed method studies should occur logically and
sequentially.

While designing a mixed methods study, three issues need consideration: Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003).
priority, implementation, and integration (Creswell, Plano Clark, Guttman, Advanced Mixed Methods Research Designs. In A. Tashakkori, & C. Teddlie
& Hanson, 2003). (Eds.), Handbook of Mixed Methods in Social and Behavioral Research (pp. 209-
Priority refers to which method, either quantitative or qualitative, is given 240). Thousand Oaks, CA: Sage.
more emphasis in the study.
Implementation refers to whether the quantitative and qualitative data
collection and analysis comes in sequence or in chronological stages, one
following another, or in parallel or concurrently. Integration refers to the
phase in the research process where the mixing or connecting of
quantitative and qualitative data occurs.
Research design
The visual model of the procedures for the sequential explanatory mixed
methods design of the study must be presented
Designing and conducting mixed methods research Creswell JW, Plano CV. 2011. Designing and conducting mixed methods research.
2nd ed. Thousand Oaks: SAGE Publications
This article provides researchers with knowledge of how to design a high Schoonenboom, J. & Johnson, JB. (2017) How to Construct a Mixed Methods
quality mixed methods research study. To design a mixed study, Research Design. Köln Z Soziol. (Suppl 2) 69:107–131. DOI 10.1007/s11577-
researchers must understand and carefully consider each of the dimensions 017-0454-1
of mixed methods design, and always keep an eye on the issue of validity.
We explain the seven major design dimensions: purpose, theoretical drive,
timing (simultaneity and dependency), point of integration, typological
versus interactive design approaches, planned versus emergent design, and
design complexity. There also are multiple secondary dimensions that need
to be considered during the design process. We explain ten secondary
dimensions of design to be considered for each research study.
We also provide two case studies showing how the mixed designs were
constructed.

Population and sampling

Mixed Methods Sampling: A Typology Teddlie, C. and Yu, F. (2007) Mixed Methods Sampling: A Typology with
Four MM sampling prototypes are introduced: Examples. Journal of Mixed Methods Research, 1, 77-100.
basic MM sampling strategies, https://doi.org/10.1177/2345678906292430
sequential MM sampling,
concurrent MM sampling, and
multilevel MM sampling.
Examples of each of these techniques are given as illustrations of how
researchers actually generate MM samples.
Finally, eight guidelines for MM sampling are presented (p96).
Sampling Onwuegbuzie, A. J., & Leech, N. L. (2007). Sampling Designs in Qualitative
Research: Making the Sampling Process More Public . The Qualitative Report,
12(2), 238-254. Retrieved from https://nsuworks.nova.edu/tqr/vol12/iss2/7
First, we present sampling schemes that have been associated with Onwuegbuzie, A. J., & Collins, K. M. (2007). A Typology of Mixed Methods
quantitative and qualitative research. Sampling Designs in Social Science Research .The
Second, we discuss sample size considerations and provide sample size Qualitative Report, 12(2), 281-316. Retrieved from
recommendations for each of the major research designs for quantitative https://nsuworks.nova.edu/tqr/vol12/iss2/9
and qualitative approaches.
Third, we provide a sampling design typology and we demonstrate how
sampling designs can be classified according to time orientation of the
components and relationship of the qualitative and quantitative sample.
Fourth, we present four major crises to mixed methods research and
indicate how each crisis may be used to guide sampling design
considerations.
Finally, we emphasize how sampling design impacts the extent to which
researchers can generalize their findings.
Reliability and Validity
The mixed method approaches have recently risen to
prominence. The reason that more researchers are opting
Mixed methods researchers presently have 24 sampling schemes from which to
choose. These 24 designs comprise 5 probability sampling schemes and 19
purposive sampling schemes.
Table 3
Minimum Sample Size Recommendations for Most Common Quantitative and
Qualitative Research Designs
Figure 2
Two-dimensional mixed methods sampling model providing a typology of mixed
methods sampling designs.
Table 5
Crises Faced by Mixed Methods Researchers
Zohrabi, M. (2013). Mixed Method Research: Instruments, Validity,
Reliability and Reporting Findings Theory and Practice in Language
Studies, 3(2), 254-262, February 2013..doi:10.4304/tpls.3.2.254-
for these types of research is that both qualitative and 262
quantitative data are simultaneously collected, analyzed and
interpreted. In this article the main research instruments
(questionnaire, interview and classroom observation) usually
used in the mixed method designs are presented and
elaborated on. It is believed that using different types of
procedures for collecting data and obtaining that
information through different sources (learners, teachers,
program staff, etc.) can augment the validity and reliability
of the data and their interpretation. Therefore, the various
ways of boosting the validity and reliability of the data and
instruments are delineated at length. Finally, an outline of
reporting the findings in the mixed method approaches is
sketched out. It is believed that this article can be useful and
beneficial to the researchers in general and postgraduate
students in particular who want to start or are involved in
the process of conducting research.
Trustworthiness in qualitative research Shenton, AK. (2004). Strategies for ensuring trustworthiness in qualitative
research projects. Education for Information, 22( 2), 63-75. Although many
critics are reluctant to accept the trustworthiness of qualitative research,
frameworks for ensuring rigour in this form of work have been in existence for
many years. Guba's constructs, in particular, have won considerable favour and
form the focus of this paper. Here researchers seek to satisfy four criteria. In
addressing credibility, investigators attempt to demonstrate that a true picture of
the phenomenon under scrutiny is being presented. To allow transferability, they
provide sufficient detail of the context of the fieldwork for a reader to be able to
decide whether the prevailing environment is similar to another situation with
which he or she is familiar and whether the findings can justifiably be applied to
the other setting. The meeting of the dependability criterion is difficult in
qualitative work, although researchers should at least strive to enable a future
investigator to repeat the study. Finally, to achieve confirmability, researchers
must take steps to demonstrate that findings emerge from the data and not their
own predispositions. The paper concludes by suggesting that it is the responsibility
of research methods teachers to ensure that this or a comparable model for
ensuring trustworthiness is followed by students undertaking a qualitative inquiry.
Mixed methods research is becoming an important
methodology to investigate complex health-related topics, yet Guetterman TC,Fetters MD,Creswell JW. (2015). Integrating Quantitative and
the meaningful integration of qualitative and quantitative data Qualitative Results in Health Science Mixed Methods Research Through Joint
remains elusive and needs further development. A promising
Displays. Ann Fam Med. Nov;13(6):554-61. doi: 10.1370/afm.1865.
innovation to facilitate integration is the use of visual joint
displays that bring data together visually to draw out new
insights. The purpose of this study was to identify exemplar
joint displays by analyzing the various types of joint displays
being used in published articles.

Mixed-Methods Sequential Explanatory Design Ivankova, N.V; Creswell, J & Stick, SL. 2006. Using Mixed-Methods Sequential
Explanatory Design: From Theory to Practice. Field Methods, 18(1), February: 3-
20

Dhanapati S. 2016 Explanatory Sequential Mixed Method Design as the Third

Research Community of Knowledge Claim.” American Journal of Educational
Research, 4(7): 570-577.
Ivankova, N.V, 2014, Implementing Quality Criteria in Designing and Conducting
a Sequential QUAN → QUAL Mixed Methods Study of Student Engagement
With Learning Applied Research Methods Online. 8(1): 25-51
Qualitative data analysis
Quantitative data analysis

Integration / Merging
………. that qualitative and quantitative components are integrated. Some Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a Definition
primary ways that the components can be connected to each other are as of Mixed Methods Research. Journal of Mixed Methods Research,1(2), 112–133.
follows: https://doi.org/10.1177/1558689806298224
(1) merging the two data sets,
(2) connecting from the analysis of one set of data to the collection of a
second set of data,
(3) embedding of one form of data within a larger design or procedure, and
(4) using a framework (theoretical or program) to bind together the data
sets (Creswell and Plano Clark 2011, p. 76).
Guetterman TC, Fetters MD, Creswell JW. 2015. Integrating quantitative and
qualitative results in health science mixed methods research through joint displays.
Annals of Family Medicine. 2015;13(6):554–61
Driscoll, D.L.; Appiah-Yeboah, A; Salib, P; & Rupert, D. J.. 2007. Merging
Qualitative and Quantitative Data in Mixed Methods Research: How To and Why
Not. Ecological and Environmental Anthropology (University of Georgia). 18.

Sample proposal Ivankova NV. 2002. A Sample Mixed Methods Dissertation Proposal (Prepared by
Nataliya V. Ivankova).

Do a Google Scholar search (Type the following in the Search Bar - A Sample
Mixed Methods Dissertation Proposal Prepared by Nataliya V. Ivankova

Mixed Methods Analysis and Information Visualization: Graphical Display Onwuegbuzie, A. J., & Dickinson, W. B. (2008). Mixed Methods Analysis and
for Effective Communication of Research Results Information Visualization: Graphical Display for Effective Communication of
Research Results . The Qualitative Report, 13(2), 204-225. Retrieved from
http://nsuworks.nova.edu/ tqr/vol13/iss2/5

Reporting of a Mixed Methods Study O'Cathain, A., Murphy, E. & Nicholl, J. (2008) The quality of mixed methods
studies in health services research. J Health Serv Res Policy, 13, 92-8
Data integration is a crucial element in mixed methods Fielding, N. G. (2012). Triangulation and Mixed Methods
analysis and conceptualization. It has three principal Designs: Data Integration With New Research
purposes: illustration, convergent validation Technologies. Journal of Mixed Methods Research, 6(2), 124–
(triangulation), and the development of analytic density 136.https://doi.org/10.1177/1558689812437101
or “richness.” This article discusses such applications in
relation to new technologies for social research, looking
at three innovative forms of data integration that rely on
computational support: (a) the integration of geo-
referencing technologies with qualitative software, (b)
the integration of multistream visual data in mixed
methods research, and (c) the integration of data from
qualitative and quantitative methods.
Applications
This article examines the prevalence and characteristics of mixed van der Roest, J.-W., Spaaij, R., & van Bottenburg, M. (2015).
methods research in the relatively Mixed Methods in Emerging Academic Subdisciplines: The
new subdiscipline of sport management. A mixed methods study is Case of Sport Management.Journal of Mixed Methods
undertaken to evaluate Research, 9(1), 70–90.
the epistemological/philosophical, methodological, and technical https://doi.org/10.1177/1558689813508225
levels of mixed methods
design in sport management research. The results indicate that
mixed methods research is still
rarely used, poorly legitimized and often weakly designed in this
field. Our conclusions lead to
the hypotheses that the more central a research field is, the higher
the prevalence of mixed
methods, and that mixed methods only slowly trickle down from
central to more peripheral
subdisciplines. Implications of the research findings for both mixed
methods scholars and sport
management researchers are discussed, and directions for future
research are proposed.
The purpose of this paper is to review the application of Molina A.J, M and Cameron, R. (2010). The Application of Mixed Methods in
mixed methods research within organisational and Organisational Research: A Literature Review” The Electronic Journal of Business
management research. The study reported in this paper Research Method. 8(2), .95-105, available online at www.ejbrm.com
identifies the use of mixed methods in the leading journals in
these two organisational fields: the Strategic Management
Journal (SMJ) and the Journal of Organizational Behavior
(JOB). In addition, Organizational Research Methods (ORM)
is

This article describes integration principles and practices at three Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed
levels in mixed methods research and provides illustrative methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt
examples. Integration at the study design level occurs through three 2):2134–2156. doi:10.1111/1475-6773.12117
basic mixed method designs—exploratory sequential, explanatory
sequential, and convergent— and through four advanced
frameworks—multistage, intervention, case study, and participatory.
Integration at the methods level occurs through four approaches. In
connecting, one database links to the other through sampling.With
building, one database informs the data collection approach of the
other. When merging, the two databases are brought together for
analysis. With embedding, data collection and analysis link at
multiple points. Integration at the interpretation and reporting level
occurs through narrative, data transformation, and joint display. The
fit of integration describes the extent the qualitative and quantitative
findings cohere.