Research Design: Data

Analysis

Chapter 5
How do you identify the quantitative data analysis and results?

– Look to the Method section for a general

description of the data analysis process
– Examine the Results section to find the
results for the study’s research questions and
hypotheses
What are the steps to understand a study’s quantitative data analysis?

Look for how the researchers:

– Scored the data (assigned a value for each
response)
– Prepared the data for analysis
– Analyzed the data to answer descriptive research
questions
– Analyzed the data to test comparison and
relationship hypotheses
– Reported the results
How do researchers prepare the quantitative data for
analysis?

– Select a computer program to use for analysis

– Enter the collected data into the database
– Examine the dataset for missing data
(participants did not answer certain items)
– Check data for accuracy
– Prepare the types of scores needed for the
analysis
• Single-item scores
• Summed (scaled) scores
How do researchers use descriptive statistics to
answer descriptive research questions?

• Central tendency (overall tendency in the

data)
• Variability (how varied scores are)
• Relative standing (where one score stands
relative to the rest of the scores)
How do descriptive statistics describe central tendency?

– Two common statistics are the mean and the

mode
– Mean is the average score for the variable
across all the participants
– Mode is the score that is the common
response for a variable
How do descriptive statistics describe variability?

– Two common statistics are the range and the

standard deviation
– Range of scores is the difference between the
highest and the lowest scores obtained for a
variable
– Standard deviation is a measure of how
dispersed the data are about the mean value
for a continuous variable
How do descriptive statistics to describe relative standing?

– Two common statistics are percentile ranks

and z scores
– Percentile rank is a number describing the
percentage of participants in the distribution
who have scores at or below a particular
score
– Standard z score is a measure of relative
standing that is calculated by converting a
participant’s score into a relative score
measured in units of standard deviations
How do researchers use inferential statistics to answer
comparison and relationship research questions?

1. Identify a null and alternative hypothesis

2. Set the criterion for making a decision, called the
alpha value or level of significance
3. Collect data from a sample
4. Compute a statistic for the sample’s data
5. Make a decision about rejecting or failing to
reject the null hypothesis
 What statistical tests are used for comparing groups?

Statistic name Use

T-test Test difference between 2 groups, 1
dependent variable
Analysis of Variance Test difference among 2 or more groups, 1
(ANOVA) dependent variable
Analysis of Covariance Test difference among 2 or more groups, 1
(ANCOVA) dependent variable, controlling for at least 1
control variable
Multiple Analysis of Test difference among 2 or more groups, 2 or
Variance (MANOVA) more dependent variables
Chi Square Test difference among groups in terms of a
categorical dependent variable
Effect Size Assess effect size for a significant t-test
Eta Squared Assess effect size for a significant ANOVA test
 What statistical tests are used for comparing groups?

Statistic name Use

Mann Whitney Test difference among 2 groups, 1
dependent variable that is non-normal
Kruskal–Wallis one-way Test difference among 2 or more groups, 1
Analysis of Variance dependent variable that is non-normal
Odds Ratio Describe likelihood (or odds) that a trait
occurs when an independent/predictor
variable is present relative to the odds that
the trait occurs when the
independent/predictor variable is absent
 What statistical tests are used for relating
variables?
Statistic
name Use
Pearson Test relationship between two variables
Correlation
Multiple Determine degree to which 2 or more independent
Regression variables are related to (or predict) 1 dependent
variable
Beta Provide standardized measure of each independent
variable’s individual contribution to the dependent
variable in a multiple regression analysis
Factor Determine how items on an instrument relate to each
Analysis other to form different subscales through correlational
techniques
 What statistical tests are used for relating
variables?
Statistic
name Use
Structural Test whether a set of variables are related to each
equation other in a specific way suggested by a theoretical
modeling model through correlational techniques
Hierarchical Analyze data that represents multiple levels (such as
linear and students, their teachers, and their schools) for
multilevel relationships (or differences)
modeling
Coefficient of Assess the proportion of variability in one variable
determination accounted for by a second variable
R squared Assess the proportion of variability in the dependent
variable accounted for by the combination of
independent variables in the regression equation
How do you make a decision to reject the null hypothesis?
(and therefore accept the alternate hypothesis)

– If the analysis finds that the two groups are

sufficiently different (or the relationship among
variables is sufficiently large) then the
researcher is able to conclude that it is very
unlikely that the null hypothesis is true
– In other words, the data support the
conclusion that there is a difference (or a
relationship)
How do you make a decision to fail to reject the null hypothesis?
(and therefore reject the alternate hypothesis)

– If the analysis finds that the two groups are not very
different (or the relationship among variables is not
very large), then the researcher is able to conclude
that there is not sufficient evidence to reject the null
hypothesis
– In other words, the data support the conclusion that
there is no difference (or no relationship)
How do you understand the results in a quantitative study?

• Examine tables that summarize statistical

information
• Examine figures that portray variables and
their relationships
• Read the detailed explanations written
about the statistical results
How do you examine tables to learn a
summary of major results?
– A table is a summary of quantitative information
organized into rows and columns
– Read the title to learn the type of information
presented in the table
– Look at the labels for each column and each row to
identify specific variables and computed statistics
– Identify the primary statistics that are included
– Tables may include
• Participant demographic tables
• Group comparison tables
• Variable correlation tables
How do you examine figures to learn how variables are related?

• A figure is a summary of quantitative information

presented as a chart, graph, or picture that
visually shows the relations among scores or
variables in a study
• Read the caption, typically found at the bottom,
that describes the contents of the figure
• Read the labels to learn which specific variables
are displayed in the figure
• Look at the legends and notes for additional
information about what is displayed in the figure
What are the steps for qualitative data analysis?

– Step 1: Preparing the data

– Step 2: Exploring the data
– Step 3: Coding the data
– Step 4: Developing description and themes
– Step 5: Validating the findings
How do your prepare qualitative data for
analysis?
– Create digital copies (such as scanning a
document)
– Transcribe audio recordings and field notes
• Verbatim means that the researcher types all
spoken words as well as unspoken events
– Prepare summaries of the collected data
• For cases where interview was not recorded or
resources for transcription are limited
– Organize the database to facilitate the
analysis process
How do you explore qualitative data?
– Read transcripts and field notes through several
times
– Become familiar with the information
– Form some initial ideas about the data
– Create memos, a record of their thoughts and
ideas about the data
• Short phrases, ideas, concepts, or hunches
• Document interpretations throughout the process
• A record of the researcher’s personal reflections
– Organize the data and consider whether more
data are needed
 What are codes?
– Codes are labels used to describe the
meaning of a segment of text or an image in
relation to the study’s central phenomenon
– Codes represent the researcher’s
interpretations of the data
How do you code qualitative data?
– May start with specific topics of interest as
codes to identify the data that relates to those
topics
– May read the data and create codes based on
their interpretation of the meaning of the data
– Codes may be phrased in standard academic
terms
– Codes may be participant’s actual words,
which are called in vivo codes
 How do you manage the coding
process?
– By hand
• Highlight segments of text with different colors
• Cut and paste text segments onto cards (with
scissors and tape!)
– Use a computer program
• Electroniclly assign labels or codes to the data
• Facilitates searching through the data and locating
text assigned to specific codes
 What are themes?
– Also called categories
– Similar codes aggregated together to form a
major idea about the central phenomenon
– Larger patterns in the data that have emerged
from the analysis
How do you develop themes?
– Refine codes and build findings
– Also called analytic induction, constant
comparison, or thematic development
– Consists of three stages:
• Develop description
• Build themes
• Relate multiple themes
How do you develop description?
– Build a detailed rendering of people, places,
or events that provide the context for a
qualitative research study
– Transport the reader to a research site or help
the reader visualize a person
 How do you build themes?
– Reduce the number of codes to about 20 by
combining redundant ideas
– Merge codes into five to seven broader, yet
related categories which represent the most
important ideas about the central
phenomenon
How do you relate multiple themes?
– Layer themes
• Depict themes using embedded levels (major
themes and sub-themes)
– Interconnect themes
• Larger order of ideas (e.g. chronological)
• Sequence of relationships (e.g. theoretical model)
– Develop a visual model to depict the
connections
 How do you validate qualitative
findings?
– Use strategies used to ensure the accuracy and
credibility of the findings as part of the analysis
process
– Attended to any ideas found in the data that did
not seem to fit the list of codes
– Four forms frequently reported by qualitative
researchers:
• Bracketing
• Triangulation
• Member checking
• Auditing
 How are tables and figures used to report qualitative
findings?

– Provide details beyond what is in the text

– Convey the complexity of the findings in concise visual
– May include:
• Demographic tables: Relate the important characteristics of
the settings and participants
• Map figures: Display important details of a setting and how
those details relate to the study’s description and themes
• Comparison tables: show different perspectives about a
theme based on a grouping characteristic (e.g. gender)
• Hierarchical tree figures: display how themes combine
together to form larger ideas about the central phenomenon
• Thematic figures: identify the major themes and how the
themes relate to each
How do the report of findings differ in
relation to the research design?
– General qualitative approach
• A simple report of the themes that emerged from the study
– Narrative approach
• A chronological retelling of the participant’s story, a description of
the context, and themes that emerged
– Case study approach
• An in-depth description of each case, themes that emerged, and
themes across cases (if multiple cases were studied)
– Ethnographic approach
• A detailed description of how a group behaves, thinks, or talks as
well as the context, or setting of the group
– Grounded theory approach
• The themes (or categories) that emerged, a display of the
interconnections among the themes, and a discussion of the theory
that emerged, and the hypotheses (or propositions) suggested by
these connections
The End