Qualitative Data Analysis Techniques

for Drawing Themes

[insert date]
Qualitative Methods
in Evaluation of Public
Health Programs
Session 6
The evaluation process
Learning objectives
By the end of this session,
participants will be able to:
 Explain qualitative data analysis
and its approaches
 Describe stages in conducting
qualitative analysis
 Develop a coding structure
for categorizing data
 Apply the analytical method
for drawing themes
Overview of qualitative analysis

 Organizing and interrogating

data in ways that allow evaluators
to see patterns, identify themes,
discover relationships, develop
explanations, and make
interpretations (Leech &
Onwuegbuzie, 2007)
 Systematic search for meaning
in data
Qualitative data
Large volumes of data are reduced
or configured.
 A process of selecting, focusing,
simplifying, abstracting, and
transforming data
Forms of qualitative data:
 Text (written field notes or transcripts)
 Image (video/pictures)
 Sound (audio recordings)
Approaches for qualitative analysis

 Grounded theory

 Framework analysis

 Discourse analysis

 Thematic analysis***

 Others…
Analysis techniques
 Grounded theory: Data are analyzed
and conceptualized to construct an
explanatory theory of basic social
processes
 Framework analysis: Adapts to
evaluation that has specific questions,
a limited time frame, a predesigned
sample, and a priori issues
 Discourse analysis: Concerned with
language use and how individuals
accomplish personal, social, and political
projects through language
Thematic analysis

 A method for systematically identifying,

organizing, and offering insight into
patterns of meaning (themes) across
a dataset (Braun & Clarke, 2012)
 Entails careful reading and rereading
of data (Guest & Macqueen, 2011)
 Themes are identified and used to
answer a specific evaluation question
Drawing themes
Stages of analysis
 Stages of data coding/analysis

Developing code book and

testing reliability of codes
Coding
processes
Summarizing data and
identifying initial themes

Connecting the codes

and identifying themes
Identifying
themes
Corroborating and
legitimizing coded themes

Iterative process, not linear

Coding
What is a code?
 Label for a feature of the data (e.g., qualitative
indicators) that is potentially relevant to the
research question—a segment of information that
gives a sense of direction
What to code?
 Field notes, transcripts, observational notes,
images, voice recordings, sound, etc.
Who?
 Somebody who participates in the fieldwork
 Somebody who has a deep understanding
of the program and evaluation questions
When?
 Coding must be done after one or two rounds
of “reading” raw data
Code identification

Thematic analysis

Inductive Deductive
•Exploratory Top-down
•Bottom-up Confirmatory
•Data driven Theory-based

E.g., women reporting

intimate partner
E.g., why are women in violence in District Y has
District Y not reporting cases Hybrid
declined owing to
of intimate partner violence combination increased awareness
at the facility? of IPV as a crime.
Code identification

Code identification and listing

can entail different approaches.
A combination, or hybrid, is
appropriate for qualitative
evaluation.
Hybrid approach for
Qualitative evaluation
Combining inductive and deductive
approaches:
 Commonly used
 Highly practical for qualitative
evaluation purposes
For example, once patterns and themes have been
established through inductive analysis, the final
confirmatory stage may be deductive and affirming
the authenticity and appropriateness of the inductive
(Patton, 2003).
Identifying codes for qualitative evaluation

 Codes are driven by the

evaluation questions
 Stay focused on primary and
specific evaluation questions/issues
(program processes, outcomes,
and impact)
 Qualitative themes: e.g., processes
leading to the outcomes, evolving
relationships/ involvement, etc.
Developing a codebook
 Begin codebook development
once there is agreement on codes—
however, definitions are still proposed
and reviewed
 Serves as data management tool for
organizing segments of similar or
related texts to assist in interpretation
 It provides a clear trail of evidence
for credibility of study
Codebook structure
Codebook may include the
following components:
1. A code
2. A brief definition
3. A full definition
4. Guidelines for when to use the code
5. Guidelines for when not to use
the code
6. Examples from the text
Codebook
Example of a codebook entry
Testing reliability of codes

 Determine the applicability of codes

to raw information

 Key to select some of the raw

documents as test pieces

 This should follow the coding

process of the documents using the
defined codes
Testing the codebook
Individual exercise:
Using the codebook in the case study,
practice applying codes A.04, A.05,
and/or A.06. to the transcript. (10 mins.)

Group discussion (10 mins.):

1. Which codes are relevant?
2. Which areas had no codes?
Applying codes

 Apply codes in the codebook to text

 Use to identify meaningful units of text

 Can be software assisted

(practical session is next)
Coding systematically
Ask yourself the following questions:
 What is this saying?
 What does it represent?
 What is this an example of?
 What do I see is going on here?
 What is happening?
 What kind of events are at issue here?
 What is this trying to convey?
 Identifying and
reviewing themes
What is a theme?

 Shows patterns within the data

 Broader than codes
 Connects codes across and
between data
 Captures something about the data
in relation to the evaluation question
 Tells a “story” from a group of codes
Examples of themes or patterns

Patterns in behaviors, experiences, views,

knowledge, or emotions in relation to the
intervention/program being evaluated.

 E.g., what are women saying about

spousal abuse after completing an
intimate partner violence educational
program?
Theme search
Searching for themes is an active process
 Evaluator generates or constructs
themes rather than discovers them
 Refocus the analysis at the broader
level of themes, rather than codes
 Entails sorting the different codes into
potential themes, and collating all the
relevant coded data extracts within
the identified themes
 Analysis of codes
Connecting codes and identifying themes

 Review coded data to identify

areas of similarity and overlap
between codes, and across all
sets of data

 Identify differences between

separate groups
Reviewing themes
Two levels of reviewing and refining
1. Reviewing at the level of coded
data extracts
 Read all the collated extracts for each
theme, and consider whether they
appear to form a coherent pattern

2. Consider the validity of individual

themes in relation to the data set
 Ensure your thematic map
“accurately” reflects the meanings
evident in the data set as a whole
Thematic maps
Using visual representation of themes
and relationships between codes.
These could be:
 Mind maps
 Charts
 Tables

Maps, charts, and tables are used to

construct a process-outcome matrix for the
program (Patton, 2003).
Drawing thematic maps
Data
Theme Codes extract/
quotes

?
?
???? ?
? ?
Evaluation question
How acceptable is VAW
among community members?
Small group activity
Connecting codes/extracts
to create themes
 Split into groups of 3–4
 Create various piles of the
codes/quotes related to each other
 Label piles with theme(s); a word
or phrase (20 mins.)
 Give a rationale for that theme
and discuss in plenary (20 mins.)
Key questions to ask
 Is this a theme?
 What is the quality of this theme?
 What are the boundaries of this theme?
 Are there enough data to support
this theme?
 Are the data too diverse or wide ranging?
 Do themes connect with one another
to enable the construction of process-
outcome matrix for the program?
Corroborating and legitimatizing
Coded themes
 Process of confirming results
 Selected themes must be related,
but not overlap, and directly address
an evaluation question
 Corroborate to avoid fabrication
of evidence
 This is an iterative process
Tips for conducting analysis

1. Start analysis immediately

2. Involve more than one person
3. Leave enough time and funds
for analysis and writing
 To be discussed further
in upcoming sessions
References
Leech, N.l., & Onwuegbuzie, A.J. (2007). An array of
qualitative data analysis tools: A call for data analysis
triangulation. School Psychology Quarterly; 22(4).
Braun, V., & Clarke, V. (2014). What can “thematic
analysis” offer health and wellbeing researchers?
International Journal of Qualitative Studies on Health
and Wellbeing; 9(26).
Braun, V., & Clarke, V. (2006). Using thematic analysis
in psychology. Qualitative Research in Psychology;
3(2):77–101.
Guest, G., Macqueen, K.M., & Namey, E.E. (2012).
Applied Thematic Analysis. Thousand Oaks, CA, USA:
SAGE Publications, Ltd.
Patton, M.Q. (2003). Qualitative evaluation checklist.
This presentation was produced with the support of the United
States Agency for International Development (USAID) under
the terms of MEASURE Evaluation cooperative agreement
AID-OAA-L-14-00004. MEASURE Evaluation is implemented by
the Carolina Population Center, University of North Carolina at
Chapel Hill in partnership with ICF International; John Snow,
Inc.; Management Sciences for Health; Palladium; and Tulane
University. Views expressed are not necessarily those of USAID
or the United States government.
www.measureevaluation.org