Competency-based

Medical Bachelor Program

“CBMBP”
 Type I & Type II Errors

Prof.Dr/Ibrahim Kabbash
Professor of public Health and Community Medicine
 ILOs

•Identify type I and type II errors

•Describe test power

•Make a decision in practical situation

 Fundamental and Important Concept

1. What is the concept of tests of significance?

2. Why we perform tests of significance
3. What is the meaning of presence of a significant
effect?
4. How can we accept or reject presence of
significant effect?
5. What are possible errors when accepting or
rejecting a test of significance?
 Hypothesis Testing…

1. There are two hypotheses, the null and the

alternative hypothesis.
2. First assume that the null hypothesis is true.
3. The goal is to determine whether there is enough
evidence to infer that the alternative hypothesis is
true, or the null is not likely to be true.
4. There are two possible decisions:
• Conclude that there is enough evidence to support the
alternative hypothesis. Reject the null.
• Conclude that there is not enough evidence to support
the alternative hypothesis. Fail to reject the null.
 Hypothesis Testing…

A criminal trial is an example of hypothesis testing

without the statistics.
In a trial a jury must decide between two
hypotheses. The null hypothesis is
H0: The defendant is innocent
The alternative hypothesis is
H1: The defendant is guilty
The jury does not know which hypothesis is true.
They must make a decision on the basis of
evidence presented.
 Hypothesis Testing…

•In the language of statistics convicting the

defendant is called rejecting the null
hypothesis in favor of the alternative
hypothesis.
•That is, the jury is saying that there is
enough evidence to conclude that the
defendant is guilty (i.e., there is enough
evidence to support the alternative
hypothesis).
 Hypothesis Testing…

If the jury acquits it is stating that there is

not enough evidence to support the
alternative hypothesis.
Notice that the jury is not saying that the
defendant is innocent, only that there is not
enough evidence to support the alternative
hypothesis.
That is why we never say that we accept the
null hypothesis.
 Hypothesis Testing…

•There are two possible errors.

Type I error occurs when we reject a true
null hypothesis. That is, a Type I error occurs
when the jury convicts an innocent person.
Type II error occurs when we don’t reject a
false null hypothesis . That occurs when a
guilty defendant is acquitted.
 Hypothesis Testing…

The probability of a Type I error is denoted as α

(Greek letter alpha). [usually 0.05 or 0.01]
The probability of a type II error is β (Greek
letter beta).
The two probabilities are inversely related.
Decreasing one increases the other, for a fixed
sample size.
In other words, you can’t have  and β both
real small for any sample size. You may have to
take a much larger sample size, or in the court
example, you need much more evidence.
•In medical research
 Hypothesis

• A statement about what finding are expected.

• A null hypothesis:
The two groups will not differ
• Alternative hypothesis
Group A and group B will not perform the same
( e.g. group B will perform better than group A)
 A Test of Hypotheses

A test of hypotheses is a method for

using sample data to decide whether
the null hypothesis should be rejected.

Copyright (c) 2004

Brooks/Cole, a division of Thomson
Learning, Inc.
 Recommended Steps in
Hypothesis-Testing Analysis
1. Identify the parameter of interest and
describe it in the context of the problem
situation.
2. Determine the null value and state the null
hypothesis.
3. State the alternative hypothesis.
 Hypothesis-Testing Analysis
4. Give the formula for the computed value of
the test statistic.
5. State the rejection region for the selected
significance level
6. Compute any necessary sample quantities,
substitute into the formula for the test
statistic value, and compute that value.
7. Decide whether H0 should be rejected and
state this conclusion in the problem context.
 data

Are our inferences valid?…Best we can do is to calculate probability

about inferences
 Inferential Statistics: uses sample data
to evaluate the credibility of a hypothesis
about a population

NULL Hypothesis:

NULL (nullus - latin): “not any”  no

differences between means

H0 : m1 = m2

Always testing the null hypothesis “H- Naught”

Inferential statistics: uses sample data to
evaluate the credibility of a hypothesis
about a population

Hypothesis: Scientific or alternative

hypothesis

Predicts that there are differences

between the groups

H1 : m1 = m2
 Possible Outcomes in
Hypothesis Testing (Decision)

Null is True Null is False

Correct
Accept Error
Decision
Type II Error

Correct
Reject Error
Decision
Type I Error

Type I Error: Rejecting a True Hypothesis

Type II Error: Accepting a False Hypothesis
 Remember

When you perform a statistical test, you’re only taking

ONE SAMPLE from a population – and there are tons of
different samples you could potentially be collecting.
You have to think about your sample in the context of
aLL the potential samples you could have collected…
fortunately this is made easy thanks to the sampling
distributions of proportions and means.
ALPHA

the probability of making a type I error  depends on the

criterion you use to accept or reject the null hypothesis =
significance level (smaller you make alpha, the less likely
you are to commit error) 0.05 (5 chances in 100 that the
difference observed was really due to sampling error – 5%
of the time a type I error will occur)

Possible Outcomes in
Hypothesis Testing
Alpha () Null is True
Correct
Null is False
Accept Decision Error
Type II Error
Difference observed is really Correct
Reject Error Decision
just sampling error Type I Error
The prob. of type one error
  When we do statistical analysis… if alpha
(p value- significance level) greater than 0.05

WE ACCEPT THE NULL HYPOTHESIS

 if alpha (p value- significance level)

is equal to or less that 0.05 we

REJECT THE NULL (difference btw means)

Two Tail

2.5% 2.5%

5%region of rejection of null hypothesis

Non directional
One Tail

5%

5%region of rejection of null hypothesis

Directional
 BETA

Probability of making type II error  occurs when we fail

to reject the Null when we should have
Possible Outcomes in
Hypothesis Testing
Null is True Null is False
Beta (b) Accept Correct
Decision Error
Type II Error

Difference observed is real Error Correct

Reject Decision
Failed to reject the Null Type I Error

POWER: ability to reduce type II error

 POWER: ability to reduce type II error
(1-Beta) – Power Analysis

The power to find an effect if an effect is present

1. Increase our n

2. Decrease variability

3. More precise measurements

Effect Size: measure of the size of the difference

between means attributed to the treatment
 The Pregnancy Test Example

H0: woman is not pregnant.

HA: woman is pregnant.

There are four things that can happen when you take the pregnancy test:
The test can be ACCURATE, and say that:
1) Woman is pregnant when she actually is, OR
Reality
2) woman not pregnant when she actually ISN’T
H0 True HH00 False
False
(Really
(Really NOT
Are (Really
(Really ARE
NOT
OR it can be INACCURATE, and say that: Pregnant)
pregnant) pregnant)
pregnant)
3) Woman is pregnant when she is NOT
(a FALSE ALARM), or POS
What the (Reject H0)
4) Woman is not
Pregnancy
pregnant when NEG
Test Said
she actually IS…. (Fail to
(a FAILURE TO DETECT the effect) Reject H0)
 The Pregnancy Test Example

H0: woman is not pregnant.

HA: woman is pregnant.

There are four things that can happen when you take the pregnancy test:
The test can be ACCURATE, and say that:
1) Woman is pregnant when you actually are, OR
Reality
2) Woman is not pregnant when she actually ISN’T
H0 True HH00 False
False
(Really
(Really NOT
Are (Really
(Really ARE
NOT
Or it can be INACCURATE, and say that: Pregnant)
pregnant) pregnant)
pregnant)
3) Woman is pregnant when she ISN’T
= probability of
(a FALSE ALARM), or POS this test raising a
What the (Reject H0)
4) Woman is not FALSE ALARM

pregnant when Pregnancy b = probability of

Test Said NEG this test FAILING
she actually IS…. (Fail to TO DETECT your
(a FAILURE TO DETECT the effect) Reject H0) pregnancy!
 The Pregnancy Test Example

H0: woman is not pregnant.

HA: woman is pregnant.
If Type II Error is the probability that a statistical test performed on this particular
sample FAILS TO DETECT THE EFFECT (pregnancy), then what is the probability
that a test using this sample will SUCCESSFULLY DETECT THE EFFECT?
Reality
P(successfully detecting the effect) = H0 True HH00 False
False
(Really
(Really NOT
Are (Really
(Really ARE
NOT
1 – P(NOT detecting the effect) = 1 - b Pregnant)
pregnant) pregnant)
pregnant)

= probability of
POS
This (1 - b) is called (Reject H0)
this test raising a
What the FALSE ALARM
The POWER OF Pregnancy b = probability of
THE TEST! Test Said NEG this test FAILING
(Fail to TO DETECT your
Reject H0) pregnancy!
 Consequences of Pregnancy Test Errors

• Type I Error – The test raised a false alarm and got the woman
either very worried or excited. She may have already run to the
store to buy baby supplies, incurring unnecessary costs. she
may have spent days or weeks panicking until she realized that
the test was faulty.
• Type II Error - The test failed to detect her pregnancy and she
didn’t stop smoking, therefore potentially harming a life. She
didn’t seek prenatal medical attention.

In a case like this, the test designers need to think about how to
minimize BOTH Type I and Type II Errors. There are
psychological and cost ramifications if either kind happens.
 Power of the Test

Power = 1 - b
• It’s the probability of successfully detecting an effect
The power of the test INCREASES as the effect size increases…
• So , if a woman is just ONE day late, the effect size is small – it
will be difficult for that pregnancy test to give an accurate
result. she’ll have a high Type II Error at this time.
• If she is a week late, the effect size is bigger – the Type II Error
will be lower, because the pregnancy is easier to detect.
• If she is a month late, the effect size is HUGE and the Type II
Error will be even smaller.
 NOW

 now we can:
• Identify type I and type II errors
• Make a decision on practical situation
 reference

1. Peaccock JL. And Peacocok PJ. Oxford handbook of

medical statistics. Oxford university press 2011.
2. Centers for disease control., US department of
health and human services. Principals of
Epidemiology in Public Health Practice: An
introduction to applied Epidemiology and
biostatistics. A self study courses SS 1000.3rd
ed.,Atlanta, GA 3033; 2006
3. Bonita R. Beaglehole R.and Kjellstrom T. Basic
Epidemiology. 2nd ed.,WHO; 2006.

6/27/2020 prof.dr.nadira mansour 33

 References

4. Stephen B.Hulley, Steven R. Cummings, Warren S.

Browner, Deborah G. Grady, Thomas B. Newman.
Designing Clinical Research. 4th ed. Lippincott Williams
& Wilkins .; 2013
5. Jan W. Kuzma. Basic Statistics for Health Sciences. 1st
ed. Mayfield Publishing Company ; 1984
6.Nicole Radziwill, James Madison University.Type I &
Type II Errors,Power of a Statistical Test, &
Effect Size @nicoleradziwill / radziwnm@jmu.edu
free to use & share with citation

6/27/2020 prof.dr.nadira mansour 34