306 PART 5 / ANALYSI fies Foundations of Ani ¢ AND REPORTING alysis for Research Design, re you answer the major research rt whet dar anal ch desig. The di frame the ar seoires and participant ae bough ves scifies how me eeause it secs Po research design also frames dag “The heart of the questions—is ie research endeavor together. So, it shoul analysis (cee es Th de statonship between design and quantitative ang Fe ge ect se a ysis We wl deine kre cred to address a research queen of hypothe re ee ea ‘General Linear Model (GLM)—the GLM underlie al of tie ethene od the aero aii og anal Peh,2 if you get a good understanding ‘of what that’s all about, the soca roid bea Tele easier to handle. For this reason, even though each pce “Tein bas its own unique quirks and idiosynrasiem, things aren’t as confusing or ve have emphasized the concepts and complicated as they may seem at first. Als, taste formulas rather than specific statistical program ourP is because this way you will have some understanding of what is go'n St “under the hood” of the 20¥ sical programs that all use some variation of this model. Troe ererove on to specifcs—frst by considering the basic randomized ‘experimental designs, starting with the simplest—the two-group posttestonly eriment--and then moving on to more complex designs. Finally, we briefy d- ‘gus the world of quasi-experimental analysis where the quasi nature ofthe design Ieads to all pes of analytic problems (some of which may even make you ques). By the tine you're through with all ofthis, we hope that you'll have a pretty fm ‘grasp on a least two things: 1) how analysis is linked to your research design, and 2) the perils of applying a seemingly obvious analysis to the wrong design structure I 12.2 Inferential Statistics Inforontial statistics Stalistcal analyses used foreach conclusions that extend beyond the imme lato data alone. general linear model {GLM) A sysiom of oqua- tis tha is used as the mathematical ramework for many of tho statistical analyses used in applied social rosoaic ona eats the proces fying to each conclusions that extend beyond the immediate data Such approaches ae undertaken when you ae yng" ue are ae asis for drawing broader inferences (thus, the name) or genera avons Thats you us inferential statistics o try to infer from the sample ‘ise robe lation. Or, you use inferential statistics to make judgments t0 Areas tea observed difference between groups is a dependable ont. Ahi ge you ara able to extimate the size of the efecto elation in thendata ken ear pce statistics simply to describe what's going 0” fom yor das ce Ps You se inferential ais vo draw conlsot Tn this chapter, we conc perinatal and qua-enperimental rseeeh ese various esearch design ‘itst, you need to unders ‘model (GLM), Vietaly all a sand what is meant by the term genera Ie Of sail model age maior inferential statistics come from the fl Seed a et Doras the GLM. Given the importance of the GLM idea of i. The discussion of eins feseaFcher to become familiar with the hve of the GLM here is elementary and considers onlychapter 12/ INFERENTIAL ANALYSIS 307 ‘ofthese models; but it will familiarize you with the idea of the lin. aoe jris adapted f0 meet the analysis needs of different research ieee sat ond one ofthe keys to understanding how the GLM can be adapted forthe sii specific research ieee is to learn what a dummy variable is and how perhaps these variables would be better named as stand-in variables, Es we Syadunmy variables one that uses discrete numbers, usally O and 1, to rep. fa diferent roups in your study inthe equations ofthe GLM., The concep of ss vaabls i a simple one that enables some complicated things to happen, fornsance by including a simple dummy variable in a model, you can model two ‘rt goups (eg teatment and comparison group) withina single equation, “Tid, we need to return to the idea of conclusion validity in the context of ‘analysis. Despite what you may have heard, making a valid inference in- ‘esmore than just a statistical rest and its associated probability (p value). This Secausea statistical test by itself addresses only one aspect ofthe study outcome, veda very limited one at that. The p value is the probability of a result given that foul bypothesis is true. And that is all itis. To really answer a research question til we also need to know about the size ofthe effect (the effect size, ES), and the ‘dae precision of our estimate ofthe effect (with a confidence interval, Cl). dummy variable. A van Wehave discussed effect sizes as a signal-to-noise ratio, in which the signal is able at uses discrete teefet difference in means, strength of correlation, etc.) and the noise is error "UMD, wvaly 0 and ‘mance in our measurements. Effect Sizes provide a common metric for compar-_{nf0fresnt fron ingand integrating results across studies. Effect Sizes are the basis of meta-analysis yuh yacctice ne andthey enable us to have a cumulative kind of science, as discussed in Chapter 1. gon usedin equations Thee are many books, statistical programs, and online calculators to help inthe Genoa inear cseachers translate results into appropriate Effect Sizes. Mosel (GL We have also discussed confi dace intervals earlie Way back in Gapter 4 we discussed sampling «roy the standard error of an es- | timate, and the “68, 95, 99 rule” | that can be applied when looking atthe standard error in normally datibuted data. Not surprising 'y, searchers have adopted the 9% percent confidence interval 3 (vo standard errors) as an accept- ‘le boundary for an estimate. So {separ an effet size with a Fercent confidence interval, # Wate providing readers with not cilya standardized estimate of our {hit ucome, but also an idea z 4 the range of likely outcomes Sige fest of possible replica- 2 of our study. i “Ob, ifenly it were so simple ‘aly, there is the matter of © Figure 12.1 To realy un- the statistical result translates to real life. Even if the result is statistically gerstand complex study Sttfcan, is it of any practical significance in real life? Just because we C2 rasuits, we have to Know nest program or intervention leads to statistically significant improte- someting abou pect ‘i €°esn't mean the amount of improvement is meaningful or all that valu- cal as well as ™ (Fgure 12.1), For instance, if we can conduct a study that shows that a signifeance snl ae creeTable 12.1 Possible out Romeo significa cera fie 308 PART 5 / ANALYSIS AND REPORTING 3 General Linea dio statistical and practical significance. ‘ames of a study with reg te tee be jing board ‘A definite may! Beokto the crewing PO, | (lay De Lassie! res, etc.) consider more participants, better aa measures?) ‘Juice not worth the squeeze ‘What we want! tease the 25608" beter mplomenlation?) 1m into jail from 26 percent istically significant amount, lly valuable to us of that it portance means the degree to ingful difference in daily life. post-incarceration program can reduce recidivis Pe24 percent and we determine that this is a stat jt doesn’t mean that 2 percent is all that pra‘ js worth the cost of the program. Practical imy which a statistical result translates to a meani at ve example ofthis comes from the classic meta-analysis by Smith one Fae (1980) on the benefits of psychotherapy. They combined results of 395 suadies and determined thatthe overall effect size was .68 (more than two > fe of standard deviation improvement). In terms of practical importance, ‘his meant that the median treated client (at the SOth percentile at che beginning of the study) ended up better off ehan 75 percent of those untreated. Table 12.1 provides a summary of the possible combinations of statistical and practical significance. ‘The GLM that you'll read about in the balance of this chapter does the main work of providing a model to test your hypothesis. As you can sec, put- ting statistically significant results from the model in the context of Effect Sizes bounded by confidence intervals and translated into practical impor- tance provides a much more complete understanding of your data than the statistical test alone. r Model The GLM is the foundation for the t-test, Analysis of Variance (ANOVA) Aaa Coney (ANCOVA), regression analysis, and oa ie mulvariat mahods inldig factor analysis, caster analysis, males: sional sealing, discriminant fanction analysis; canonical correlation, and ott crs That i all of he analyses we jst mentioned canbe deseribed using the See eguaions known as the GLM. Who cares? Well, have you ever fee arte. seis seem to know so much abot See 2 Sure, the ears sloggin advance rears ra, paar sel But the seal reason ner Ty online cae ttons with some authority statistically i that they are fe Learn the: general Doe the GLM, which has lots of v sn sms ma dace ae an ae i