Faculty of Nursing, Universitas Hasanuddin

4th International Conference

Integrating analyses in mixed

methods research
A workshop presented by: Professor Pat Bazeley
Western Sydney University
 Mixed methods research
Mixed methods studies are those in which more than one
source or type of data, and/or more than one approach to
analysis of those data, are integrated throughout the study
in such a way as to become interdependent in reaching a
common theoretical or research goal.
(Bazeley, 2010: 432)

There are two key elements in this definition:

- It says something about your choices of data, and
- It emphasises the absolute necessity for integration.
 1. Phenomena (things seen and unseen)
– and data
• All phenomena – seen and unseen – have both qualities
and quantities. They are multidimensional.
• Therefore, we need words and numbers, and perhaps also
images or multimedia to record our observations.
• All data require interpretation.
 2. Integration – as interdependence –
the essential component for MMR
• Interdependence in mixed methods research means that one
approach, methodology, or method used without input from
another is inadequate to achieve the purpose of the research
– each is influenced by, or dependent on, the other.
• Interdependence is stronger than simply “linking” or “bringing
together” different analyses at the conclusion of a study.
• It is evidenced in the results and conclusions of a study.

Think of the interdependence of the two

strands of DNA as information is shared
between them through protein transfer.
 Design for integration
Plan questions, data sources, samples, management, analysis
strategies and eventual reporting all with integration in
mind.
• Set out a common purpose and common epistemology for all data.
• Focus questions on the topic/problem to be solved, not methods.
• Choose data that will help to achieve your purpose, regardless of type.
• Align samples by topic and/or by case.

Research is a planned but non-linear

process: design strategically, and
expect to adapt.
Integrative methods and processes
for mixed data and/or sources
7 ‘C’s – 3 are core for analysis
Converse across methods throughout the project
Construct one method based on another
Combine sources/data in complementary analyses
Compare across data types and sources
Convert data from one form to another
Compile using all sources together
Convey results arranged by topic, not method
 CONVERSE between methods
The iterative (back and forth) exchange occurring
throughout a project:
• Through reflection, as each approach prompts ideas to inform
the construction and/or analysis of other data
• Through team exchanges, discussions, and shared
experiences
This then:
• Informs further developments
• Informs interpretive understanding
• Resolves potential conflicts
CONSTRUCT one method based on another
To inform data collection (with participant focus)
• Usually qual -> quant for survey design (e.g., to generalise)
• Also survey -> interviews/other qual (to extend, clarify)
To inform data analysis
• Quant informs qual e.g., provides codes for interviews
(or theory informs both)
• Qual themes guide quant analysis/synthesis

-> Combine both sets of data/results in

final analyses/write up.
 e.g., Develop survey questions from interview data
Sub-theme Quote Scale Items
Balance of Work, Most PhD programs are not structured for people that Family responsibilities would make it difficult for me to
School, and Family work. They just aren’t. And that makes it extremely pursue a PhD in engineering.
Life difficult to pursue it. If you’re someone that has a
Balancing school, work and family time would be a factor in
family and has a job, and things go along with that like
considering a PhD.
a house. It’s extremely difficult.
Confidence & Self- I could work full-time while earning a PhD part-time. I am smart enough to complete a PhD.
efficacy And when you think that it’s unreachable or
My GPA is good enough to get admitted to a PhD program.
unattainable or you couldn’t—you know, it seems too
hard or I’m not smart enough or something like— I feel confident in my academic abilities.
even though you’re doing fine.
… … …
PhD-Level I guess, um, they could make it more interesting to A Professional Engineering license is more valued by
Engineering Work me if they could show a reason, a difference between industry than a PhD.
being a, just a PE, or being a PE and having a PhD. Like
I understand the kind of work that engineers with PhDs do.
I can’t see, I don’t know what difference there is
adding your PhD, they pretty much can do the same I think people with a PhD in engineering are overqualified
thing. for most engineering jobs.

I can do the same kind of work with a bachelor’s degree

that an engineer with a PhD can do.

Source: Howell Smith, M. C. (2011). Factors that facilitate or inhibit interest of

domestic students in the engineering PhD: a mixed methods study.
http://digitalcommons.unl.edu.au/dissertations/AA13466518 11, p. 69; Table 5
 COMBINE data and analyses
A process of creating something new, enhanced, or more
complete, from component parts:

Add/Extend Compose Juxtapose

• Illustrate • Weave • ‘Validate’
• Enhance • Merge or confirm
• Explain • Complete • Compare and
• Contextualise converge

– aided by keeping combined records/results organised

by topic or in an integrated QDAS database
COMBINE -
how to, without
qual software:
Use heading styles
and the document
map (navigation or
summary pane) in
MS Word to record
information and
insights, to facilitate
integration, analysis,
and writing.
If you begin by
recording different
data together, you
are more likely to
effectively integrate
your results and
conclusions.
Code retrieval in QDAS – brings sources together

Weaving or
merging
different
→
pieces of
information
into a
coordinated
report
 e.g., Combine through weaving
Data from: Group discussion, Initiating and maintaining involvement
The majority (47/92; 51%) of those coming to Melaleuca
Interviews, Literature, Surveys. Wellness in 2016 found out about the centre through friends or
relatives. Twenty-seven (29%) responded to an advertisement or
(Original sources discernible.) story in the newspaper, and nine (10%) came through other
health care or exercise class providers.
More interviewees spoke about social interaction than
physical activity in relation to their involvement in an OWN
Centre; all those who talked about physical activity also
mentioned social interaction. For MWC survey respondents,
exercise (43%) and social connection (13%) or a combination of
these (43%) were primary reasons given coming to the Centre: I
love the exercise and the talks. Make friends, meet people. Great
social life and interesting. Mental wellbeing was also referred to:
Improved my outlook on life, my mental wellbeing. Anxiety has
improved, so has mood. It helps relieve my anxiety and am not as
stressed as I used to be. It’s given me a reason to live.
Ninety per cent of those attending MWC participate in at least
one of the six types of active physical exercise classes: …. The
seven women who came only for the other less active classes (…)
were significantly older than others (by an average of eight years)
and included several of the older-old, long-term members. Those
coming to yoga and tai chi were more likely to indicate they were
coming to the Centre for the benefit of the exercise than for
social reasons (a proportion of those doing yoga came only for
that class). In contrast, social reasons were as much a motivator
for coming as physical reasons for those coming for strength and
balance, gentle exercise, relaxation, and Feldenkrais….
The MWC women spoke of how they kept coming because of
the friendship, company and social support, and its welcoming
and friendly environment: The friendship of the girls and learning
new things…Makes me get up and go, like company, talking. They
enjoyed the activities and could feel the tangible benefits to their
health and wellbeing: I feel good, and flexible after the exercises…
Makes me feel better…Makes me feel relaxed after a class and
better within myself. …The women, therefore, maintained their
participation because they perceived benefits to their physical
health, cognitive-mental health, personal motivation-
Reported contributions to health and wellbeing from attending MWC Changes over time for scores on WHO-5, a measure of general wellbeing
development, and social interaction …
 Combine through merging
e.g., modelling/theorising/conceptualising
Integrative model based on analysis and
interpretation of multiple data sources (surveys,
interviews, cultural domain data, literature).
(Sources are no longer discernable)
Following a thread
 COMPARE (and contrast)
Juxtapose different data in a joint display
Identify patterns of similarity and difference

Analyse discrepancies
Data are conceptually aligned (same topic), but might be from different
samples.

Matrix analyses
Identify patterns (contrasts and similarities) in qualitative data associated
with groups based on demographic, categorical, or scaled variations
Data can be conceptually different, but must come from linked cases.
Joint display: Compare to confirm or contrast

Fitzpatrick, K. R. (2016). Points of

convergence in music education:
the use of data labels as a
strategy for mixed methods
integration. Journal of Mixed
Methods Research, 10(3), 273-
291. doi:
10.1177/1558689814560264
 Joint display: Compare to identify patterns of association
Practice Practice Preventive Service Defining Characteristics of Approach to Delivering
1 2
Size and Characteristics Delivery, % Preventive Services
Location

9. Small PA or NP: Yes Screening: 65 Prevention is clinician driven with minimal office-
urban level protocols, and staff are not supportive. Charts
System owned: Smoking counseling:
are well organized, and clinicians rely on a mental
Yes 31
protocol to deliver preventive care opportunistically
High volume: Immunization: 31 in most visits. Few patients attend HCM visits, and
Yes many patients have complex, multiple chronic health
problems, so encounters have many competing
social and economic constraints that make
prevention difficult.
12. Small PA or NP: No Screening: 58 Clinicians often spend a great deal of time with
suburban patients in lengthy visits. Preventive care is delivered
System owned: Smoking counseling:
in all visits. Thorough HCM visits are provided, using
No 69
comprehensive sex- and age-specific prevention flow
High volume: No Immunization: 16 sheets. A reminder system is used for female annual
examinations. High-quality educational materials are
available. Charts are a mess and poorly filed, often
unfilled in piles around the floor. Staff are all part-
time and not involved in prevention
1PA= physician’s assistant, NP = nurse practitioner; 2HCM = health care maintenance
Source: Crabtree et al. (2005): Online Supplemental Table 1 (http://www.annfammed.org/cgi/content/full/3/5/430/DC1)
 Compare using a matrix or crosstab
• Use data (different types, same or different sources) from
linked (i.e., same or matched) samples.
• Identify patterns (contrasts and similarities) in qualitative
data associated with groups based on demographic,
categorical, or scaled variations.
• This usually requires software: Excel or preferably QDAS.
• Use all sources of data to explain similarities and differences
– e.g., how many differ, and how they differ.
Compare and Contrast using an Excel matrix
Explore patterns:
contrasts and
relationships
• Record cases in rows,
codes in columns and
relevant data in cells
• Use the categorical
variables to SORT the
data: then COMPARE
patterns of responses
for different groups.
and/or
• Use the split pane or
hide columns to bring
selected columns
together to allow
exploration of regular
ASSOCIATIONS within
cases.
Compare how many and how with a QDAS crosstab
 Use comparative analyses to:
Describe differences (what) -> further questions to explore (how, why)
• e.g., For older women: If the way in which participants talk about autonomy or
contentment is differentially associated with their age or their health status, what
else is associated with age or health that helps to explain these differences?
Validate and interpret scale scores
• If differences in scores on a scale to assess depression are associated with a
difference in verbal responses related to mood, do these differences support
(validate) the scaled measure?
• What does it mean experientially to be at a certain point as measured by, say, a
visual analogue scale for pain?
• Do verbal responses about satisfaction with a service correspond with and explain
concurrently gathered ratings of the same service?
Identify dimensions in a concept
• Does examining differences across subgroups of a sample reveal variations in (or
sub-dimensions of) a concept?
 CONVERT (transform) data
to gain a dual perspective using the same data

• Transform qual codes to numeric variables

- counts -> integrated description
-> statistical analyses
- description; comparison; prediction
- multivariate exploratory analyses

• Narrative or profile analysis from quant data

- profile individuals, groups;
- build a narrative from panel data
 Count (and illustrate) occurrences
(data from descriptors given for particular characteristics of high-performing researchers)

Issues raised: What/how to count – cases or volume or intensity?

What does zero mean?
Post-code (CONVERT) in new columns to allow further analyses (intensity coding)
 Convert (transform) interpretive codes to numeric
variables (using 0/1 or frequency counts) with QDAS

Using survey responses and interviews (e.g., using 2, t, r, etc)

• Code open-ended responses or supplementary interviews, convert to variables
(± add to existing database) → compare/relate to other measures and variables

Prediction/explanation (e.g., using regression)

• e.g., Identifying factors contributing to an outcome.

Be aware of statistical assumptions!

Always consider results in relation to original qualitative data.
 Employ exploratory multivariate statistics
- based on frequency of intersections in coding

Use a similarity matrix -> cluster analysis, multidimensional scaling

→ visualise patterns, clusters, dimensions

• Exploratory multivariate statistics do not have the same sampling or

data distribution requirements as other statistics.
• QDAS provides counts of exact intersections based on specific data,
not just for overall case(s).
 Cluster analysis
based on a similarity matrix

Cluster analysis
can also be
carried out
using a simple
case by variable
matrix
Two-dimensional output from MDS of descriptors
of researchers, based on similarity matrix

Dimension 1
Technical-Social

Dimension 2
Extrinsic-Intrinsic

Matrix transferred
from QDAS to SPSS
for processing
Use correspondence analysis based on a pattern matrix
Association between descriptors of researchers (inputs) and type of output

• Also based on frequencies of

intersections between codes
• Shows association between
demographic or other
variables and qualitative
characteristics

Matrix table from QDAS

requires further conversion
into a form suitable for SPSS
to allow processing.
 CONVERT and COMBINE
to consolidate, blend, and further analyse

Consolidate
• Create a consolidated database from variable data,
counts, and transformed codes -> analyse statistically.
• For a small sample, combine simple numbers with text
summaries in a metamatrix -> pattern identification.
Blend
• Combine code data with variable data to create blended
variables for further analyses -> resolve puzzles.
Consolidated case-based database (metamatrix)
using categorised qual and quant data
Case Mini Mental Caregiver Vigilance Cooperation Trust
Status Exam (hours on duty)
Jerry 8 (Low) 24 hrs (High) Moderate Low
directable, hides or loses
doesn’t resist, pills, forgets what
requires repeat he is doing
cues

Claire 28 (High) 3 hrs (Low) High High

reminds caregiver can take
to refill responsibility

Source: Happ et al., 2006: S46; adapted from Table 2

See also Louis, 1982: American Behavioral Scientist, 26(1), 101-120.

A 240-variable table is constructed from ‘patchy’ evaluation data to evaluate a reading
program intervention in schools -> pattern analysis.
 Strategies in combination
Construct +/- Compare +/- Convert -> Combine

• Many approaches involve combinations, e.g.,

• after construct -> combine;
• after convert -> review qual and combine/compare
• combine strategies in complex studies

• Hybrid methodologies (e.g., QCA, SNA, GIS, Q-factor)

• typically use combinations of convert and combine

• Resolving dissonance and divergence

• often involves using a combination of analytic strategies

Construct, Compare, Convert, and Combine are foundational

components of more complex analysis strategies
The processes of iterative exchange through
construction, comparison, and conversion, and the
analytic combination of data, methods, and results
from those processes:
• Allow you to make use of whatever data are available
• Deepen analysis through breaking down concepts and
exchange of ideas
• Benefit understanding through providing a more
complete, more holistic view of a phenomenon
 Finally: COMPILE and CONVEY
through analytic writing and modelling

• Write during analysis – a key to integration

• Initiate, reflect, deepen

• Integration of results – evidence of integration

• Design results around issues to be discussed rather than
the methods used to investigate them
• Keep a methods log / audit trail to support process and
evidence

• Journey toward a conclusion

• Build an argument e.g., Employ process tracing
• Tell a story Test alternative explanations
• Support with evidence Visualise in charts and models
 MMIRA
Mixed Methods International Research Association
• MMIRA aims to create an international community to promote and support
interdisciplinary mixed methods research
• Our vision includes bringing together diverse communities of scholars, students,
practitioners, policymakers, citizens, and other stakeholders, with the goals of
expanding knowledge and producing social betterment and social and global
justice
• Support provided through regional and international conferences, newsletters,
website resources, etc. Membership includes electronic access to Journal of
Mixed Methods Research

Further info: www.mmira.org

 Integrating analyses in MM research (Sage, 2018)
This book takes you beyond mixed methods research design and data
collection to provide a pragmatic discussion of the challenges of
effectively integrating data to facilitate a more comprehensive and
rigorous level of analysis. Showcasing a range of strategies for integrating
different sources and forms of data as well as different approaches in
analysis, it will help you plan, conduct, and disseminate complex analyses
with confidence.

A practical introduction to MM (Sage, 2019)

This book introduces the concepts and debates associated with
combining methods, and illustrates both benefits and the hazards of
undertaking a mixed methods study by drawing on example studies from
across business and management disciplines.
Students and researchers undertaking their own mixed methods
project are taken step-by-step through the mixed methods process from
developing a mixed methods study, through designing and conducting it,
to integrating and reporting on the results.