Internal ant;!

External Validi

Internal Validity External Validity

The extent to which The extent to which
alternate explanations, results of an

~~~,~
beyond the variable of experiment can be
interest, can be ruled generalized beyond
the conditions of an
experiment
't;)ecCUJ~e ~~
'\JCU'\~\es o...re
\~\ed Cut ,
;.i .'-'.-.;.' ~t;~. "i ) .~.~ ~ , " -c,

(:{+-\t' . 0" ~\,'C) ~Q\) I\

~. Internal Validity Threats

\) Selection Biases
Initial differences ill groups, based on h~~;. .••••I..' .,
selected •• '1re. ~eero.:trs --. 'ru I..UJ
Attrition
Subjects who leave are different than subjects who
.: - _~?~ ;~Ie W't\o df~cUCo{' '"'e r~
PrC.~Yf\
.... ,t. '",' ,
'1'~ffUS!OIl:rCes.tliie!ljt! f.\\:.';} .~<>_, ;" ~.., •••• ,,. •. ! ,"

First Question External Validity Threats

Administrators at Spokane Mental Health:
\) Generalizability Across Subjects .., I .w',! ",;1.-\
• Have.therapists administer a reliable and
valid depression inventory to clients who How well will these results generalize to people who
are diagnosed with depression at intake are different from those in the study
• This occurs during the first and last therapy Generalizability Across Settings
session (t\a..~\a!s.:<r\') The extent to which results apply in other situations
• About one-third of the clients do not;.
complete a termination testl~\lric::nJ
" "
i:) Generalizability Across Measures
• They compare the pretest and posttest If response was measured differently how would
scores to see if therapy is effective for ~ results generalize
depression ~ Generalizability Across Times
Would response differ across time of day, of week
Identify at least two internal validity threats that Generalizability Across Change Agents
" .are present within,this research method
,... , '~"" r •••• " i. \ ; .i,.,... ,.. '\'.~ '" 1 ; ~ ,1 .' t. ,~..• ' ~ 1 . '
t
c: ; 8,01'\ wel! do ,find,ingsapply to different teachets,
therapists, parents .,., ", '" "--' '. ,

1
 ) I II
: I e,

: , '•...;.,. - I
-. .'

3/30/2011

External Validity Threats Second Question

Reactive Arrangements
The Eastern Region DCFSdirector wants to know if
Influence of being aware that you're in a study
"Hawthorne Effect" family preservation really works, so he
• Identifies all of the cases referred to family
Reactive Measurements preservation, and those whose children were placed
Influence of measurement or assessment on in foster care, during calendar year 2001
response
• Usesthe DCFSdatabase to track time to
Pretest Sensitization permanency for these cases
Pretest may influence behavior in study, set a. _ • Compares the family preservation and foster care
demand characteristic J "'.. •
.f:a~e.0t:ne to ermanency l:. L~er~
Multiple Treatment l,nfer'e'nce -1.1.. '
Were the participants exposed to _rlJyltiple" ' Ide~YJa~two e a~~thr~ts that _" 1'-'---
•••.•
treatments, was it an interaction of treatments • are present within your sites research problem
.
that led to the results ~\. \CUt; e. scme.o \
" . 't • <
0- . ..
u't I 1 ,J f
-."'. · I

€riteria for Inferring Causalityfllo :~' ': ,Pre-Experimental Designs \

Temporal Ordering iThese designs rank low for accepted evidence
based practice research due to the very low •'
The independent (causal) variable precedes the degre~of internal validity
-: ,JI.. ,dependent v~rifP,le in~~m~ ~ ~t •
Unp:>nt[9I1edCase Studies X 0
Empirical Correlation Ii Simply niakes an observation after a stimulus is
presented

~ii~gr:\~:
•

\oe Qi\n
The

er
independent and dependent variables be

ir\)~I I
ere
,i
: .•r •
Address temporal ordering (no threats to internal
, validity are controlled).'
• No correlation addressed
-:S:'CM-c.r,o.JeT\ ~
...'

\\Je.n J
'/ ,.'
., \
~ " ~~
~"\iID,nl'Ii:"",
CO res "" T~ ,,-0
,_L .•.•

«; ..'
"J:tle relationship between the indep,~l1dent;a~.d . , • One Group Repeated Measures 0 X 0 , !.~
. dependent variables cannot be explaine,dby the s- • Makes observations of one group of subjects before
o. fundionlng of 'a ~hir.dyariable:A!:x.JUJe \ r·', and .after a stimulus is presented

D •.,.I'••••
, •• ss ' • '. 'mter~ ': , 'J Address temporal orrtering,and ernpjncat correlation "
'\"\...:, .. '
••

~ '," - ~ -:'1, • .' r, ' L. ." 'i" \ '-\'

:, Ii
~ if:J"
'
..
'.

II~, ''" t ',.~ ~ 'i.' l..A.) S. ' ". ,,' ~ .. H~ , .• :-.

o .•. ' ,',...... , .• '~ I "''': F ~ I :,' , ,~,••.•
, ,II j

" 1 ( •••.•• ~ . j l...... ~t i: ';- ~<.. ,I•• ,..... •• J '~. "~-f(;J

Ii? f·•• ·

~&g~perimental
Posttest Only With Nonequivalent Groups
Designs .. Nonequivalent Control Groups
Designs
(Static-Groupcomparisondesign) X 0 OXO
o .0,0
~ • Use existing·groups,'..:vhich ate hot randomly
.makes observations of two groups, one who assigned to conditions
receives the stimulus and one that does not,
following the stimulus presentation
'Ii Collect prefe~t data on both groups
• Apply the independent variable to one group but
.groups are not necessarily equivalent since not the other
there is no random assignment to groups • Collect posttest data on both groups
.address empirical correlation and temporal .• Address temporal ordering and empirical
ordering correlation but not internal validity ~ -n,.. "~b
e,\~ no \Y'\Cex-naj \lcU\d.''tl o \'(\\-e.rnoj V<u\o.\ ~~
("1 _~-v...",,",11

" ; \ t.

t' ,,~.' '•. ~; : t, ( , ";"J/ -, ,"t tr:: • ~ r" I ••~ I • : .. :'

~ •. 1'1 J.,.'-l<
•

• -""'"' '# _ 'to

. t, .• ,2· ·~i~,,
J

.r. , "
 R -;::
'("Q..ndoO) mssLB hed
IC= LY"\t:ervet'\tion 3/30/2011
vO::.. cb~~o...b.bn

Essenti I Components of Experimental

Designs Posttest Only Control Group Designs
1. Random assignment of participants to experimental and
control groups (not to be confused with random R X 0
sampling)
Randomization refers to participants of a study being R 0
randomly assigned to control a.n4~xperimental g~ps;
done to Increase internal validit)'{jt.eq, U ll"€O.J • Randomly assign participants to conditions
Random Samoling refers to participants 0'1 a study belnq .Apply the independent variable to the
randomly drawn from an populatio~ theoretically being
representative of that population; pone to i[lcrease experimental group but not the control
external validity (generailzability) l- ~. ~\'\Q\\e.PO\\5
2. Applying the independent variable to The experimentlll
group
group but not the control group • Collect posttest data on both groups
3. Comparing the experimental and control groups
Experimental designs allow us to examine the
direct, causal effect of one variable on another
and to make unamtiiguo s inferences. ~ eQJ1l
e.

W l.U\\\ see e lY\C5

Repeated Measures Control Group Other Experimental Designs
Designs (pretest-P9$~~st) ~ .L-~~c.Qn
s~CU'O. ~Y" I • or ss Solomon four-group design
R 0 X 0 .~ '-
R X fO)
0 C~o~e \'m\ ROO
R X 0
~ ("

R \!V 0 ~Iternative Treatmen~eSign L.r~E'Q..SU""~d\ off

• Randomly assign participants to conditions R 0 Xl 0 I;;\-' ,.~. r I.. '~p~lPn
R 0 X2 0 - ~o O\~1e.re.n ••• I
• Collect pretest data on both groups ROO
es Dismantling Studies
• Apply the independent variable to the R 0 XI,2 0
experimental group but not the control R 0 Xl 0
R 0 X2 0
group ROO
• Collect posttest data on both groups

Third Question
A researcher at Spokane Mental Health
Has parents of autistic children complete the
Parental Stress Index (PSI)
H. Randomly assigns the parents into a behavior
management course
iii. Collects posttest PSI from all parents after the
course is completed
lv. Compares the groups on parental stress.

What Design was used to answer your sites

research question?