Newsom 1

USP 656 Multilevel Regression

Winter 2006
Jkj..khh.kh.khk.kjg;uggk..khgg.ggkg.kg.k to e me!sure" without error. #t is !lso !ssume" th!t the
v!lues o$ ! $i%e" v!ri!le in one stu"& !re the s!me !s the v!lues o$ the $i%e" v!ri!le in !nother
stu"&. 'R!n"om v!ri!les( !re !ssume" to e v!lues th!t !re "r!wn $rom ! l!rger )o)ul!tion o$ v!lues
!n" thus will re)resent them. *ou +!n think o$ the v!lues o$ r!n"om v!ri!les !s re)resenting !
r!n"om s!m)le o$ !ll )ossile v!lues o$ th!t v!ri!le. ,hus- we e%)e+t to gener!li.e the results
ot!ine" with ! r!n"om v!ri!le to !ll other )ossile v!lues o$ th!t r!n"om v!ri!le. Most o$ the time
in /N01/ !n" regression !n!l&sis we !ssume the in"e)en"ent v!ri!les !re $i%e".
Random and Fixed Effects
,he terms 'r!n"om( !n" '$i%e"( !re use" in the +onte%t o$ /N01/ !n" regression mo"els- !n" re$er to
! +ert!in t&)e o$ st!tisti+!l mo"el. /lmost !lw!&s- rese!r+hers use $i%e" e$$e+ts regression or /N01/
!n" the& !re r!rel& $!+e" with ! situ!tion involving r!n"om e$$e+ts !n!l&ses. / $i%e" e$$e+ts /N01/
re$ers to !ssum)tions !out the in"e)en"ent v!ri!le !n" the error "istriution $or the v!ri!le. /n
e%)eriment!l "esign is the e!siest e%!m)le $or illustr!ting the )rin+i)!l. Usu!ll&- the rese!r+her is
intereste" in onl& gener!li.ing the results to e%)eriment!l v!lues use" in the stu"&. 2or inst!n+e- !
"rug stu"& using 0 mg- 5 mg- or 30 mg o$ !n e%)eriment!l "rug. ,his is when ! $i%e" e$$e+ts /N01/
woul" e !))ro)ri!te. #n this +!se- the e%tr!)ol!tion is to other stu"ies or tre!tments th!t might use
the s!me v!lues o$ the "rug 4i.e.- 0 mg- 5 mg- !n" 30 mg5. 6owever- i$ the rese!r+her w!nts to m!ke
in$eren+es e&on" the )!rti+ul!r v!lues o$ the in"e)en"ent v!ri!le use" in the stu"&- ! r!n"om
e$$e+ts mo"el is use". / +ommon e%!m)le woul" e the use o$ )uli+ !rt works re)resenting low-
mo"er!te- !n" high !str!+tness 4e.g.- st!tue o$ ! w!r hero vs. ! )ivoting geometri+ "esign5. ,he
rese!r+her woul" like to m!ke in$eren+es e&on" the three )ie+es use"- so the !rt )ie+es !re
+on+e)tu!li.e" !s )ie+es r!n"oml& "r!wn $rom ! l!rger universe o$ )ossile )ie+es !n" the
in$eren+es !re m!"e to ! l!rger universe o$ !rt work !n" r!nge o$ !str!+tness v!lues. Su+h !
gener!li.!tion is more o$ !n in$erenti!l le!)- !n"- +onse7uentl&- the r!n"om e$$e+ts mo"el is less
)ower$ul. R!n"om e$$e+ts mo"els !re sometimes re$erre" to !s 'Mo"el ##( or 'v!ri!n+e +om)onent
mo"els.( /n!l&ses using oth $i%e" !n" r!n"om e$$e+ts !re +!lle" 'mi%e" mo"els.(
Fixed and Random Coefficients in Multilevel Regression
,he r!n"om vs. $i%e" "istin+tion $or v!ri!les !n" e$$e+ts is im)ort!nt in multilevel regression. #n
multilevel regression mo"els- oth level83 !n" level82 predictors !re !ssume" to e $i%e". 6owever-
level83 intercepts and slopes !re t&)i+!ll& !ssume" to v!r& r!n"oml& !+ross grou)s. 9e+!use o$ the
!ssum)tions !out their error "istriutions- we +!ll their v!ri!n+es- oo=var(U0j) !n" 11=var(U1j),
'r!n"om +oe$$i+ients.(
,his me!ns th!t we +!n think !out 0j !n" 1j !s !kin to the r!n"om variables # "es+rie"
!ove. #nste!" o$ !ttem)ting to gener!li.e e&on" the )!rti+ul!r v!lues o$ the in"e)en"ent v!ri!le-
we !re !ttem)ting to gener!li.e e&on" the )!rti+ul!r grou)s in the stu"&. 2or inst!n+e- we m!& h!ve
50 +om)!nies- ut we wish to gener!li.e to ! l!rger universe o$ +om)!nies when we e%!mine the
me!ns 4inter+e)ts5 or the %8& rel!tionshi) 4slo)es5.
#n 6:M- v!ri!n+es $or the inter+e)ts !n" slo)es !re estim!te" & "e$!ult. ,h!t is- when &ou
$irst +onstru+t ! mo"el- U0j !n" U1j !re estim!te" & "e$!ult. 6owever- the rese!r+her h!s the +hoi+e
o$ setting U0j !n" U1j to e .ero. 9& setting them to .ero- we !re testing ! mo"el in whi+h we !ssume
0j !n" 1j "o not v!r& r!n"oml& !+ross grou)s. #n $!+t- their v!ri!n+e is !ssume" to e .ero- so the&
!re !ssume" to e +onst!nt or 'nonv!r&ing( !+ross grou)s. 2or e%!m)le- $i%e"- nonv!r&ing inter+e)ts
woul" im)l& the grou) !ver!ge $or the "e)en"ent v!ri!le is !ssume" to e e7u!l in e!+h grou).
/lthough we t&)i+!ll& re$er to this +onstr!int !s '$i%ing the inter+e)ts( or '$i%ing the slo)es-( the term is
somewh!t loosel& !))lie"- e+!use we !re re!ll& !ssuming the& !re $i%e" and nonvarying.
Intercept Model and Random Effects ANOVA
#n the sim)lest 6:M mo"el- there !re no )re"i+tors. We h!ve one level83 e7u!tion !n" one level82
e7u!tion;
0 ij j ij
Y R = +
Newsom 2
USP 656 Multilevel Regression
Winter 2006
where Y is the "e)en"ent v!ri!le- 0j is the inter+e)t- !n" Rij is the resi"u!l or error. 0j +!n e
thought o$ !s the me!n o$ e!+h grou). ,he level82 e7u!tion !lso h!s no )re"i+tors in it<s sim)lest
$orm;
0 00 0 j j
U = +
where 0j is the "e)en"ent v!ri!le- 00 is the level82 inter+e)t- !n" U0j is the level82 error. #n this
e7u!tion- 00 re)resents the gr!n" me!n or the me!n o$ the inter+e)ts. U0j re)resents the "evi!tion o$
e!+h me!n $rom the gr!n" me!n. When the !ver!ge "evi!tion is l!rge- there !re l!rge grou)
"i$$eren+es.
#t +!n e shown th!t this is re!ll& the /N01/ mo"el. /N01/ e%!mines the "evi!tions o$ grou) me!ns
$rom the gr!n" me!n. 9e+!use we !ssume th!t the grou) me!ns- re)resente" & 0j !n"- thus- their
"evi!tions !re r!n"oml& v!r&ing- this mo"el is e7uiv!lent to the random effects /N01/ mo"el. We
+!n rewrite our two 6:M e7u!tions !s ! single e7u!tion- i$ we )lug in the level82 terms into level83
e7u!tion.
00 0 ij j ij
Y U R = + +
#n this e7u!tion- 00 re)resents the gr!n" me!n- the U0j term re)resents the "evi!tions $rom the me!n
4or me!n "i$$eren+es or tre!tment e$$e+t- i$ &ou woul" like5- !n" Rij re)resents the within8grou) error or
v!ri!tion. #n more m!them!ti+!ll& oriente" /N01/ te%tooks- &ou will see !n /N01/ mo"el written
something like;
ij j ij
Y e = + +
#$ the mo"el is !n r!n"om e$$e+ts /N01/- j is !ssume" to h!ve ! r!n"om "istriution. #n multilevel
regression- e+!use we !ssume U0j to v!r& r!n"oml&- the sim)le 6:M mo"el with no level83 or level82
)re"i+tors is e7uiv!lent to the r!n"om e$$e+ts /N01/ mo"el.
When we !"" )re"i+tors to the level83 e7u!tion- the& !re +ov!ri!tes !n" the mo"el e+omes !
r!n"om e$$e+ts /N=01/ in whi+h the me!ns !re !"juste" $or the +ov!ri!te.
Newsom 3
USP 656 Multilevel Regression
Winter 2006
Summary a!le
Random vs"
Fixed
#efinition Example $se in Multilevel
Regression
Varia!les Random variable; 435 is !ssume" to e me!sure" with
me!surement error. ,he s+ores !re ! $un+tion o$ ! true
s+ore !n" r!n"om error; 425 the v!lues +ome $rom !n"
!re inten"e" to gener!li.e to ! mu+h l!rger )o)ul!tion
o$ )ossile v!lues with ! +ert!in )ro!ilit& "istriution
4e.g.- norm!l "istriution5; 4>5 the numer o$ v!lues in
the stu"& is sm!ll rel!tive to the v!lues o$ the v!ri!le !s
it !))e!rs in the )o)ul!tion it is "r!wn $rom. Fixed
variable; 435 !ssume" to e me!sure" without
me!surement error; 425 "esire" gener!li.!tion to
)o)ul!tion or other stu"ies is to the s!me v!lues; 4>5
the v!ri!le use" in the stu"& +ont!ins !ll or most o$ the
v!ri!le<s v!lues in the )o)ul!tion.
#t is im)ort!nt to "istinguish etween ! v!ri!le th!t is
varying !n" ! v!ri!le th!t is random. / $i%e" v!ri!le
+!n h!ve "i$$erent v!lues- it is not ne+ess!ril& inv!ri!nt
4e7u!l5 !+ross grou)s.
Random variable:
)hotogr!)hs
re)resenting in"ivi"u!ls
with "i$$ering levels o$
!ttr!+tiveness
m!ni)ul!te" in !n
e%)eriment- +ensus
tr!+ks
Fixed variable:
gen"er- r!+e- or
intervention vs. +ontrol
grou).
Pre"i+tor
v!ri!les in M:R
gener!ll&
!ssume" to e
$i%e"
Effects Random effect; 435 "i$$erent st!tisti+!l mo"el o$
regression or /N01/ mo"el whi+h !ssumes th!t !n
in"e)en"ent v!ri!le is r!n"om; 425 gener!ll& use" i$
the levels o$ the in"e)en"ent v!ri!le !re thought to e
! sm!ll suset o$ the )ossile v!lues whi+h one wishes
to gener!li.e to; 4>5 will )ro!l& )ro"u+e l!rger
st!n"!r" errors 4less )ower$ul5. Fixed effect; 435
st!tisti+!l mo"el t&)i+!ll& use" in regression !n" /N01/
!ssuming in"e)en"ent v!ri!le is $i%e"; 425
gener!li.!tion o$ the results !))l& to simil!r v!lues o$
in"e)en"ent v!ri!le in the )o)ul!tion or in other
stu"ies; 4>5 will )ro!l& )ro"u+e sm!ller st!n"!r"
errors 4more )ower$ul5.
Random effect;
r!n"om e$$e+ts /N01/-
r!n"om e$$e+ts
regression
Fixed effect; $i%e"
e$$e+ts /N01/- $i%e"
e$$e+ts regression
#nter+e)t onl&
mo"els in M:R
!re e7uiv!lent to
r!n"om e$$e+ts
/N01/ or
/N=01/.
Coefficients Random coefficient: term !))lies onl& to M:R
!n!l&ses in whi+h inter+e)ts- slo)es- !n" v!ri!n+es +!n
e !ssume" to e r!n"om. M:R !n!l&ses most
t&)i+!ll& !ssume r!n"om +oe$$i+ients. 0ne +!n
+on+e)tu!li.e the +oe$$i+ients ot!ine" $rom the level83
regressions !s ! t&)e o$ r!n"om v!ri!le whi+h +omes
$rom !n" gener!li.es to ! "istriution o$ )ossile v!lues.
?rou)s !re +on+eive" o$ !s ! suset o$ the )ossile
grou)s.
Fixed coefficient: ! +oe$$i+ient +!n e $i%e" to e non8
v!r&ing 4inv!ri!nt5 !+ross grou)s & setting its etween
grou) v!ri!n+e to .ero.
R!n"om +oe$$i+ients must e v!ri!le !+ross grou)s.
=on+e)tu!ll&- $i%e" +oe$$i+ients m!& e inv!ri!nt or
v!r&ing !+ross grou)s.
Random coefficient;
the level82 )re"i+tor-
!ver!ge in+ome- is
use" to )re"i+t s+hool
)er$orm!n+e in e!+h
s+hool. #nter+e)t
v!lues $or s+hool
)er$orm!n+e !re
!ssume" to e !
s!m)le o$ the inter+e)ts
$rom ! l!rger )o)ul!tion
o$ s+hools.
Fixed coefficient;
slo)es or inter+e)ts
+onstr!ine" to e e7u!l
over "i$$erent s+hools.
9oth use" in
M:R. Slo)es !n"
inter+e)t v!lues
+!n e
+onsi"ere" to e
$i%e" or r!n"om-
"e)en"ing on
rese!r+hers
!ssum)tions !n"
how the mo"el is
s)e+i$ie". ,he
v!ri!n+e o$ the
slo)es or
inter+e)ts !re
+onsi"ere"
r!n"om
+oe$$i+ients.