Hypothesis

Testing
 Outline

 The Null Hypothesis

 Type I and Type II Error

 Using Statistics to test the Null Hypothesis

 The Logic of Data Analysis

Research Questions and Hypotheses

 Research question:
 Non-directional:
 No stated expectation about outcome
 Example:
 Do men and women differ in terms of conversational memory?

 Hypothesis:
 Statement of expected relationship
 Directionality of relationship
 Example:
 Women will have greater conversational memory than men
 Grounding Hypotheses in Theory

 Hypotheses have an underlying rationale:

 Logical reasoning behind the direction of the hypotheses
(theoretical rationale – explanation)
 Why do we expect women to have better conversational
memory?

 Theoretical rationale based on:

 1. Past research
 2. Existing theory
 3. Logical reasoning
 The Null Hypothesis

 Null Hypothesis - the absence of a relationship

 E..g., There is no difference between men’s and
women’s with regards to conversational memories

 Compare observed results to Null Hypothesis

 How different are the results from the null
hypothesis?

 We do not propose a null hypothesis as research

hypothesis - need very large sample size / power
 Used as point of contrast for testing
 Hypotheses testing

 When we test observed results against null:

 We can make two decisions:
 1. Accept the null
 No significant relationship
 Observed results similar to the Null Hypothesis
 2. Reject the null
 Significant relationship
 Observed results different from the Null Hypothesis
 Whichever decision, we risk making an error
 Type I and Type II Error

 1. Type I Error
 Reality: No relationship
 Decision: Reject the null
 Believe your research hypothesis have received support when in
fact you should have disconfirmed it
 Analogy: Find an innocent man guilty of a crime

 2. Type II Error
 Reality: Relationship
 Decision: Accept the null
 Believe your research hypothesis has not received support when in
fact you should have rejected the null.
 Analogy: Find a guilty man innocent of a crime
 Potential outcomes of testing
Decision
Accept Null Reject Null
R

1 2
E No
A Relationship
L
I Relationship
T
Y 3 4
 Potential outcomes of testing
Decision
Accept Null Reject Null
R

Correct
2
E No
A Relationship decision
L
I Relationship
T
Y 3 4
 Potential outcomes of testing
Decision
Accept Null Reject Null
R

1 2
E No
A Relationship
L
I Relationship
T
Y 3 Correct
decision
 Potential outcomes of testing
Decision
Accept Null Reject Null
R

1
E No
Type I Error
A Relationship
L
I Relationship

3 4
T
Y
 Potential outcomes of testing
Decision
Accept Null Reject Null
R

1 2
E No
A Relationship
L
I Relationship

4
T
Type II Error
Y
 Potential outcomes of testing
Decision
Accept Null Reject Null
R
E No
Correct
Type I Error
A Relationship decision
L
I Relationship
T
Type II Error Correct
Y decision
 Function of Statistical Tests

 Statistical tests determine:

 Accept or Reject the Null Hypothesis

 Based on probability of making a Type I error

 Observed results compared to the results expected by
the Null Hypotheses
 What is the probability of getting observed results if Null
Hypothesis were true?
 If results would occur less than 5% of the time by simple
chance then we reject the Null Hypothesis
 Start by setting level of risk of
making a Type I Error

 How dangerous is it to make a Type I Error:

 What risk is acceptable?:
 5%?
 1%?
 .1%?
 Smaller percentages are more conservative in guarding
against a Type I Error

 Level of acceptable risk is called “Significance level” :

 Usually the cutoff - <.05
 Conventional Significance Levels

 .05 level (5% chance of Type I Error)

 .01 level (1% chance of Type I Error)

 .001 level (.1% chance of Type I Error)

 Rejecting the Null at the .05 level means:

 Taking a 5% risk of making a Type I Error
 Steps in Hypothesis Testing

 1. State research hypothesis

 2. State null hypothesis
 3.Set significance level (e.g., .05 level)
 4. Observe results
 5. Statistics calculate probability of results if null
hypothesis were true
 6. If probability of observed results is less than significance
level, then reject the null
Guarding against Type I Error

 Significance level regulates Type I Error

 Conservative standards reduce Type I Error:

 .01 instead of .05, especially with large sample

 Reducing the probability of Type I Error:

 Increases the probability of Type II Error

 Sample size regulates Type II Error

 The larger the sample, the lower the probability of Type II
Error occurring in conservative testing
 Statistical Power

 The power to detect significant relationships

 The larger the sample size, the more power
 The larger the sample size, the lower the probability of Type
II Error

 Power = 1 – probability of Type II Error

 Statistical Analysis

 Statistical analysis:
 Examines observed data
 Calculates the probability that the results could occur by
chance (I.e., if Null was true)

 Choice of statistical test depends on:

 Level of measurement of the variables in question:
 Nominal, Ordinal, Interval or Ratio
 Logic of data analysis

 Univariate analysis
 One variable at a time (descriptive)

 Bivariate analysis
 Two variables at a time (testing relationships)

 Multivariate analysis
 More than two variables at a time (testing relationships and
controlling for other variables)
 Variables

 Dependent variable:
 What we are trying to predict
 E.g., Candidate preference

 Independent variables:
 What we are using as predictors
 E.g., Gender, Party affiliation
 Testing hypothesis for two nominal
variables

Variables Null hypothesis Procedure

Gender

Passing is not Chi-square

related to gender

Pass/Fail
 Testing hypothesis for one nominal
and one ratio variable

Variables Null hypothesis Procedure

Gender

Score is not T-test

related to gender

Test score
Testing hypothesis for one nominal
and one ratio variable
Variable Null hypothesis Procedure
Year in school
Score is not
related to year in ANOVA
school
Test score

 Can be used when nominal variable has more than two categories and can
include more than one independent variable
 Testing hypothesis for two ratio
variables

Variable Null hypothesis Procedure

Hours spent

studying Score is not

related to hours Correlation

spent studying

Test score
Testing hypothesis for more than two
ratio variables
Variable Null hypothesis Procedure
Hours spent
studying Score is not positively
related to hours
Classes spent studying and Multiple
missed not negatively related regression
to classes missed
Test score
 Commonality across all statistical
analysis procedures

 Set the significance level:

 E.g., .05 level
 Means that we are willing to conclude that there is a
relationship if:
 Chance of Type I error is less than 5%

 Statistical tests tell us whether:

 The observed relationship has less than a 5% chance of
occurring by chance
Summary of Statistical Procedures

Variables Procedure
Nominal IV, Nominal DV Chi-square
Nominal IV, Ratio DV T-test
Multiple Nominal IVs, Ratio ANOVA
DV
Ratio IV, Ratio DV Pearson’s R
Multiple Nominal IVs, Ratio ANCOVA
DV with ratio covariates
Multiple ratio Multiple Regression