Choose The Appropriate Statistical Test

Choose The Appropriate
Statistical Test
By
Mohammed Nawaiseh
Things to consider before choosing the appropriate test
● Variables
● Study design
● Level of measurements → continuous, interval, ordinal, nominal (dichotomous)
● Assumptions
Variables
● Independent Variable
○ How many? /// how many level for each one?
● Dependent Variable
○ How many? /// Level of measurements
● Covariate?
Study design
● Between-subjects or within-subjects
○ Unrelated groups → Between-subjects (Between different groups)
■ Different participants in each group
■ Ex → comparing the means of two different group (anxiety level between treatment and control group)
○ Related groups → within-subjects (within the same group)
■ Repeated measures
■ Same participant is measured more than once
■ Ex → comparing the means of the same group (treatment group) at separate times (before and after treatment)
Level of measurements
● Nominal → In name only
● Ordinal → Observations can be ordered, Ranks
● Scale
○ Interval → Distance between observations is meaningful
○ Ratio → True zero
How to choose the most appropriate Statistical test?
1. Determine the type and the number of dependent variable (DV; outcome) and independent Variable (IV;
Cause)
2. Level of measurements
3. Do you want to study group differences or the association?
4. Between-subjects or within-subjects
5. Try to start with parametric tests. If assumption not met, go to non parametric tests.
Group Differences
● Determine the DV
○ If Continuous (Scale; interval or ratio) DV
■ Determine the IV
● Dichotomous
○ Comparison between two different groups
■ Independent sample t-test
● Assumption not met → Mann-Whitney U test
○ Comparison within the same group (at two different times)
■ Dependent (paired) sample t-test
● Assumption not met → Wilcoxon signed-rank test
● Ordinal or multinomial
○ Comparison between 3 or more different groups → One-way ANOVA
■ Assumption not met → Kruskal-Wallis test
○ Comparison within the same group (at 3 or more different times) → repeated measure ANOVA
■ Assumption not met → Kruskal-Wallis test
○ If Ordinal DV
● Dichotomous
○ Mann-Whitney U test
○ Kruskal-Wallis test
○ If Dichotomous DV
● Dichotomous
○ Test of proportion (ex;Chi Square)
○ Test of proportion (ex;Chi Square)
Association
● Determine the DV
○ If Continuous (Scale; interval or ratio) DV
● Dichotomous
○ Point Biserial Test
■ Assumption not met → Kendall tau B Test
● Ordinal
○ Spearman's Test
■ Assumption not met → Kendall tau B Test
● Continuous
○ Pearson’s Test
■ Assumption not met →Spearman's Test
○ If Ordinal DV
● Dichotomous
○ Kendall tau B Test
● Ordinal
○ Spearman's Test or
○ If Dichotomous DV
● Dichotomous
○ Test of proportion
● In association, it’s less important to differentiate between IV and DV

Correlation
● Types
○ Pearson’s or spearman’s or Kendall tau B Test
■ And Point Biserial Test
● Two variables → there is no IV and DV “it's less important to differentiate between IV and DV”
● Continuous level of measurement “most commonly” → both variables
● Null hypothesis
○ The population correlation equals zero
● Parametric → Pearson’s test
● Nonparametric alternative
○ Spearman's rank-order correlation
Point Biserial Test

● One dichotomous variable
○ Two levels
○ Nominal level of measurement
● One continuous variable
○ Interval or ratio level of measurement
● Null hypothesis
○ The population correlation equals zero
● Alternative statistic “Parametric”
○ Independent-samples t test
T-Test
1. One-sample t-test
2. Independent-samples t-test
3. Dependent (Paired)-samples t-test
One-sample t-test
● This test is also known as → Single Sample t Test

● The One Sample t Test determines whether the sample mean is statistically different from a known or
hypothesized population mean. The One Sample t Test is a parametric test.
INDEPENDENT-SAMPLES T TEST
○ One
○ Two Unrelated groups
○ One
● Design
○ Between-subjects → comparison between two groups
● Ex
○ Comparing Anxiety scores between two groups (Treatment group and control group /// Treatment with
medications and treatment with CBT)
■ IV → Treatment (two groups) /// DV → anxiety score
● Null hypothesis
○ Population means are equal
● Alternative statistic
○ Point-Biserial Correlation, One-way ANOVA (It's used when the IV has 2 or more levels)
DEPENDENT(paired)-SAMPLES T TEST
○ One
○ Two groups
○ One
● Design
○ Within-subjects → Same group (different time)
● Ex
○ Comparing Anxiety scores between Patients in the same groups before and after Treatment
○ IV → Treatment (one group) /// DV → anxiety score
● Null hypothesis
○ Mean difference between the paired observations equals zero
● Alternative statistic
○ Repeated measures ANOVA (when there are 3 or more related groups)
■ Ex → Comparing Anxiety scores between Patients in the same groups before Treatment, after
Treatment within one month and after treatment within 6 months)
○ Wilcoxon signed-rank test
ANOVA tests
1. One way ANOVA
2. One way ANCOVA
3. Factorial ANOVA
a. 2-way Factorial ANOVAs
b. 3-way Factorial ANOVAs and Higher
4. Factorial ANCOVA
5. MANOVA
a. Factorial MANOVA
6. MANCOVA
a. Factorial MANCOVA
7. Repeated Measures ANOVA
a. Repeated measures ANCOVA, Repeated measures MANOVA , Repeated measures MANCOVA
Between- subjects ANOVA

● ANOVA → Used when there is 3 or more groups
● One way → one IV
● ANCOVA (Co-Variance) → when there is a covariate variable (variables that might influence the dependent variable; so its
controlled in this test)
● Factorial → 2 or more IV
● Multivariate → 2 or more DV
● Those can be Mixed up
Within- subjects
● REPEATED MEASURES ANOVA → same group, compared at 3 or more different times or conditions
● Can be mixed with Between- subjects ANOVA
ONE-WAY ANOVA “ANalysis Of VAriance”
○ One
○ More than two groups
○ One
● Design
○ Between-subjects (between different groups)
● Ex
○ Comparing Anxiety scores between 3 groups (Treatment with medications, treatment with CBT and no
treatment group)
■ IV → Treatment (2 or more groups) /// DV → anxiety score
● Null hypothesis
○ Population means of each level are the same
○ Kruskal-Wallis H test
● Post hoc test is used to tell exactly where is the difference between which groups, as ANOVA will tell only
that there is a difference without saying where is the difference
ONE-WAY ANCOVA “ANalysis Of COVAriance”
○ One
○ More than two groups
○ One
● One or more covariates (variables that might influence the dependent variable; so its controlled here)
● Design
○ Between-subjects
● Null hypothesis
○ Adjusted means of each level are the same
FACTORIAL ANOVA → two way ANOVA or 3 way ANOVA
○ Two or more Variables
○ Each variable might have Two or more groups
○ One
● Ex
○ Comparing Anxiety scores between 3 groups (Treatment with CBT, Treatment with Medications, and
no treatment) and Between males and females
■ This is called 2 by 3 ANOVA
■ DV → anxiety scores /// IV → Treatment and gender
● Design
● Null hypotheses
○ Population means of the factors are equal
○ No interaction
FACTORIAL ANCOVA → Two way ANCOVA or 3 way ANCOVA
○ Two or more
○ Two or more groups
○ One
● One or more covariates
● Design
● Null hypotheses
○ Adjusted means of the factors are equal
○ No interaction
ONE-WAY MANOVA “Multivariate ANalysis of VAriance”
● Is simply an ANOVA with several dependent variables. That is to say, ANOVA tests for the difference in
means between two or more groups, while MANOVA tests for the difference in two or more vectors of
means.
○ One
○ Two or more
● Comparing Anxiety scores and depression score between 3 groups (Treatment with CBT, Treatment with
Medications, and no treatment)
● Design
● Null hypothesis
○ Means of each level are the same
ONE-WAY MANCOVA
○ One
○ Two or more
● Design
● Null hypothesis
FACTORIAL MANOVA
○ Two or more
○ Two or more
● Comparing Anxiety scores and depression scores between 3 groups (Treatment with CBT, Treatment with
Medications, and no treatment) and Between males and females
○ This is called 2 by 3 MANOVA
○ DV → anxiety scores and depression scores /// IV → Treatment and gender
● Design
● Null hypotheses
○ Population means of the factors are equal
● No interaction
FACTORIAL MANCOVA
○ Two or more
○ Two or more
● Design
● Null hypotheses
● No interaction
ONE-WAY REPEATED MEASURES ANOVA
○ One
○ One
● Design
○ Within-subjects (within the same group at 3 or more different times or at 3 or more different
conditions)
● Ex
○ Comparing Anxiety scores between Patients in the same groups before Treatment, after Treatment
within one month and after treatment within 6 months)
○ Comparing Test scores between Patients in the same groups at 3 different conditions (No light level,
moderate light level and very bright light in the test room)
● Null hypothesis
○ Means of the measures are equal
○ Friedman's ANOVA
FACTORIAL REPEATED MEASURES ANOVA
○ Two or more
○ One
● Ex
○ Comparing Anxiety scores between Patients in the same group (before Treatment, after Treatment within one month
and after treatment within 6 months) and between males and females (Gender)
■ DV → Anxiety scores /// IV → Time (Within-subjects), Gender (between-subjects)
○ Comparing test scores in the same group at 2 different conditions
■ Light → Low light, moderate light, high light levels
■ Sound → Low Sound, moderate Sound, high Sound levels
■ DV → test scores /// IV → Light and sound (both Within-subjects)
● Design
○ Within-subjects OR Between- and Within-subjects
■ At least one factor is Within-subject, other factors might be Within-subject or between-subject or both
● Null hypotheses
○ Means of the levels of each factor are equal
● No interaction effect
● Other types
○ Repeated measures ANCOVA, Repeated measures MANOVA , Repeated measures MANCOVA
Chi Square
1. Chi-square goodness of fit test
2. Chi-square test for independence
Chi Square
● Types
○ A chi-square goodness of fit test determines if a sample data matches a population.
○ A chi-square test for independence compares two variables in a contingency table to see if they are related. In a more general
sense, it tests to see whether distributions of categorical variables differ from each other.
■ A very small chi square test statistic means that your observed data fits your expected data extremely well. In other words,
there is a relationship.
■ A very large chi square test statistic means that the data does not fit very well. In other words, there isn’t a relationship.
● A chi-square statistic is one way to show a relationship between two categorical variables. The chi-squared statistic is a single number
that tells you how much difference exists between your observed counts and the counts you would expect if there were no relationship at
all in the population.
Regression
LINEAR REGRESSION
● linear regression is a linear approach to modeling the relationship between a scalar response (or dependent
variable) and one or more explanatory variables (or independent variables).
● The case of one explanatory variable is called simple linear regression. For more than one explanatory
variable, the process is called multiple linear regression.
● This term is distinct from multivariate linear regression, where multiple correlated dependent variables are
predicted, rather than a single scalar variable
● Use IV to predict the DV (outcome)
○ One predictor IV → simple linear regression
○ More than one predictor IV →multiple linear regression
○ One or more
○ Continuous
○ One
● Null hypothesis
○ Slope is equal to zero → IV can’t be used to predict the DV
LOGISTIC REGRESSION
○ One or more
○ Nominal, Continuous
○ One
○ Nominal level of measurement
■ DV with two levels → Binary Logistic regression
■ DV with more than two levels → multinomial Logistic regression
● Null hypothesis
○ Slope is equal to zero → IV can’t be used to predict the DV
Another similar approach
Comparing Groups of Cases
OUTCOME 1 GROUP OF CASES 2 GROUPS OF CASES 3(+) GROUPS OF CASES

VARIABLE
Dichotomous Z-Test for One Proportion or Z-Test for 2 Independent Chi-Square Independence Test
Binomial Test Proportions or
Chi-Square Independence Test
Nominal Chi-Square Goodness-of-Fit Test Chi-Square Independence Test Chi-Square Independence Test
Ordinal Sign Test for One Median Mann-Whitney Test or Kruskal-Wallis test
Median Test for 2(+) Independent
Medians
Metric (“scale”) One Sample T-Test Independent Samples T-Test One-Way ANOVA
Comparing Variables
OUTCOME 1 VARIABLE 2 VARIABLES 3(+) VARIABLES

VARIABLE(S)
Dichotomous Z-Test for One Proportion or McNemar Test Cochran Q test

Binomial Test
Nominal Chi-Square Goodness-of-Fit (None) (None)

Test
Ordinal Sign Test for One Median Wilcoxon Signed-Ranks Test or Friedman Test
Sign Test for 2 Related Medians
Metric (“scale”) One Sample T-Test Paired Samples T-Test Repeated Measures ANOVA
Examples
Case DV, IV + levels + type Test Alternative or nonparametric
● Dichotomous (D)
● Nominal (N)
● Ordinal (O)
● Scale (S)
Comparison (C) or Association (A)
Mean height is significantly different DV → height (S) Independent Samples t Test Mann-Whitney U Test
between a group of males and a IV → gender (two levels,D)
group of females Design → Comparison
Difference in duration of hospital stay DV → length of stay (continuous) Independent Samples t Test Mann-Whitney U Test
between babies with infection and IV → infection (categorical, D) ● Assumption not met
without infection Design → Comparison (skewed,outliers, small
sample size)
● Mann-Whitney U Test
Compare Height of infants at 1 and 3 DV → Height (S) Paired Samples t Test Wilcoxon signed rank test
months of age IV → no IV ● If normality assumption is
Design → Comparison not met or small sample
size
Compare the sample IQ to the DV → IQ Single-sample t-test

average IQ (100) Design → Comparison
● Dichotomous (D), Nominal
(N)
● Ordinal (O)
● Scale (S)
The effect of socioeconomic status, IV → socioeconomic status (3 One way ANOVA Kruskal-Wallis test
which has three groups (low, medium groups, one variable, N, categorical)
and high), on weight DV → weight
Design → Comparison
The effects of gender and IV → socioeconomic status (3 factorial ANOVA

socioeconomic status on weight groups, N, categorical) & Gender (2
groups, D, categorical)
DV → weight
The effects of gender and IV → socioeconomic status (3 ANCOVA

socioeconomic status on weight after groups, N, categorical) & Gender (2
adjusting for height groups, D, categorical)
DV → weight
Covariance → height
● Dichotomous (D), Nominal
(N)
● Ordinal (O)
● Scale (S)
Association between the weight and DV → weight and length (C ) Pearson’s correlation coefficient (r ) ● Spearman’s ρ (rho)
length in certain population Design → Association ● Kendall’s τ (tau)
Test whether the absence or DV → illness (D) Chi Square

presence of an illness is independent IV → immunization (D)
of whether a child was or was not Design → Comparison
immunised
References
● Selecting the Correct Inferential Statistic
○ https://www.youtube.com/watch?v=8Lvx7BXLg44
○ This video demonstrates how to select the correct inferential statistic given specific variables, design,
and levels of measurement. Nonparametric alternatives to parametric statistics are provided. Several
statistics are reviewed including Correlation, Point-Biserial Correlation, Independent-Samples t test,
Dependent-Samples t test, ANOVA, ANCOVA, MANOVA, MANCOVA, Repeated Measures ANOVA,
Factorial ANOVA, Linear Regression, and Logistic Regression.
● A Beginner’s Guide to Using Open Access Data
○ By Saif Aldeen Saleh AlRyalat, Shaher Momani
● https://www.spss-tutorials.com/simple-overview-statistical-comparison-tests/#binomial-test
● Medical Statistics A Guide to Data Analysis and Critical Appraisal → Ch 1
● https://stats.idre.ucla.edu/other/mult-pkg/whatstat/?fbclid=IwAR0hb1GV55_vkvR6t9ps0qEBbj1GBvtGDmqlZao7IZT8qq9O0Urxc-WVI44

Choose The Appropriate Statistical Test

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Choose The Appropriate Statistical Test

Uploaded by

Copyright:

Available Formats

Choose The Appropriate

● In association, it’s less important to differentiate between IV and DV

Point Biserial Test

● This test is also known as → Single Sample t Test

Between- subjects ANOVA

OUTCOME 1 GROUP OF CASES 2 GROUPS OF CASES 3(+) GROUPS OF CASES

OUTCOME 1 VARIABLE 2 VARIABLES 3(+) VARIABLES

Dichotomous Z-Test for One Proportion or McNemar Test Cochran Q test

Nominal Chi-Square Goodness-of-Fit (None) (None)

Compare the sample IQ to the DV → IQ Single-sample t-test

The effects of gender and IV → socioeconomic status (3 factorial ANOVA

The effects of gender and IV → socioeconomic status (3 ANCOVA

Test whether the absence or DV → illness (D) Chi Square

You might also like