Applied Univariate, Bivariate, and

Multivariate Statistics: Understanding

Statistics for Social and Natural
Scientists, With Applications in SPSS
and R 2nd Edition Daniel J. Denis
Visit to download the full and correct content document:
https://ebookmass.com/product/applied-univariate-bivariate-and-multivariate-statistics
-understanding-statistics-for-social-and-natural-scientists-with-applications-in-spss-an
d-r-2nd-edition-daniel-j-denis/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Applied Statistics: From Bivariate Through Multivariate

Techniques Second

https://ebookmass.com/product/applied-statistics-from-bivariate-
through-multivariate-techniques-second/

Applied Statistics: From Bivariate Through Multivariate

Techniques Second Edition – Ebook PDF Version

https://ebookmass.com/product/applied-statistics-from-bivariate-
through-multivariate-techniques-second-edition-ebook-pdf-version/

Using Statistics in the Social and Health Sciences with

SPSS Excel 1st…

https://ebookmass.com/product/using-statistics-in-the-social-and-
health-sciences-with-spss-excel-1st/

Statistics and Probability with Applications for

Engineers and Scientists using MINITAB, R and JMP,
Second Edition Bhisham C. Gupta

https://ebookmass.com/product/statistics-and-probability-with-
applications-for-engineers-and-scientists-using-minitab-r-and-
jmp-second-edition-bhisham-c-gupta/
Applied Statistics: Theory and Problem Solutions with R
Dieter Rasch Rostock

https://ebookmass.com/product/applied-statistics-theory-and-
problem-solutions-with-r-dieter-rasch-rostock/

Applied Statistics with R: A Practical Guide for the

Life Sciences Justin C. Touchon

https://ebookmass.com/product/applied-statistics-with-r-a-
practical-guide-for-the-life-sciences-justin-c-touchon/

Statistics for Engineers and Scientists 5th Edition

William Navidi

https://ebookmass.com/product/statistics-for-engineers-and-
scientists-5th-edition-william-navidi-2/

Statistics for Engineers and Scientists, 6th Edition

William Navidi

https://ebookmass.com/product/statistics-for-engineers-and-
scientists-6th-edition-william-navidi-2/

Statistics For Engineers and Scientists 6th Edition

William Navidi

https://ebookmass.com/product/statistics-for-engineers-and-
scientists-6th-edition-william-navidi/
APPLIED UNIVARIATE,
BIVARIATE, AND
MULTIVARIATE
STATISTICS
APPLIED UNIVARIATE,
BIVARIATE, AND
MULTIVARIATE
STATISTICS:
UNDERSTANDING
STATISTICS FOR SOCIAL
AND NATURAL SCIENTISTS,
WITH APPLICATIONS
IN SPSS AND R
Second Edition

DANIEL J. DENIS
This second edition first published 2021
© 2021 John Wiley & Sons, Inc.

Edition History
John Wiley and Sons, Inc. (1e. 2016)

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or
by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to
obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Daniel J. Denis to be identified as the author of this work has been asserted in accordance with law.

Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

Editorial Office
111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at
www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard
print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties
with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including
without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or
extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization,
website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the
publisher and authors endorse the information or services the organization, website, or product may provide or recommendations
it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The
advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where
appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when
this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data applied for:

ISBN 978-1-119-58304-2

Cover Design: Wiley

Cover Image: © tatianazaets/Getty Images

Set in 10/12pt TimesLTStd by SPi Global, Pondicherry, India

10 9 8 7 6 5 4 3 2 1
To Kaiser
CONTENTS

Preface xviii
About the Companion Website xxi

1 Preliminary Considerations 1
1.1 The Philosophical Bases of Knowledge: Rationalistic Versus Empiricist Pursuits, 1
1.2 What is a “Model”?, 3
1.3 Social Sciences Versus Hard Sciences, 5
1.4 Is Complexity a Good Depiction of Reality? Are Multivariate Methods Useful?, 7
1.5 Causality, 8
1.6 The Nature of Mathematics: Mathematics as a Representation of Concepts, 8
1.7 As a Scientist, How Much Mathematics Do You Need to Know?, 10
1.8 Statistics and Relativity, 11
1.9 Experimental Versus Statistical Control, 12
1.10 Statistical Versus Physical Effects, 12
1.11 Understanding What “Applied Statistics” Means, 13
Review Exercises, 14
Further Discussion and Activities, 14

2 Introductory Statistics 16
2.1 Densities and Distributions, 17
2.1.1 Plotting Normal Distributions, 19
2.1.2 Binomial Distributions, 21
2.1.3 Normal Approximation, 23
2.1.4 Joint Probability Densities: Bivariate and Multivariate Distributions, 24
2.2 Chi-Square Distributions and Goodness-of-Fit Test, 27
2.2.1 Power for Chi-Square Test of Independence, 30
2.3 Sensitivity and Specificity, 31
2.4 Scales of Measurement: Nominal, Ordinal, Interval, Ratio, 31

vii
viii CONTENTS

2.4.1 Nominal Scale, 32

2.4.2 Ordinal Scale, 32
2.4.3 Interval Scale, 33
2.4.4 Ratio Scale, 33
2.5 Mathematical Variables Versus Random Variables, 34
2.6 Moments and Expectations, 35
2.6.1 Sample and Population Mean Vectors, 36
2.7 Estimation and Estimators, 38
2.8 Variance, 39
2.9 Degrees of Freedom, 41
2.10 Skewness and Kurtosis, 42
2.11 Sampling Distributions, 44
2.11.1 Sampling Distribution of the Mean, 44
2.12 Central Limit Theorem, 47
2.13 Confidence Intervals, 47
2.14 Maximum Likelihood, 49
2.15 Akaike’s Information Criteria, 50
2.16 Covariance and Correlation, 50
2.17 Psychometric Validity, Reliability: A Common Use of Correlation Coefficients, 54
2.18 Covariance and Correlation Matrices, 57
2.19 Other Correlation Coefficients, 58
2.20 Student’s t Distribution, 61
2.20.1 t-Tests for One Sample, 61
2.20.2 t-Tests for Two Samples, 65
2.20.3 Two-Sample t-Tests in R, 65
2.21 Statistical Power, 67
2.21.1 Visualizing Power, 69
2.22 Power Estimation Using R and G∗Power, 69
2.22.1 Estimating Sample Size and Power for Independent Samples t-Test, 71
2.23 Paired-Samples t-Test: Statistical Test for Matched-Pairs (Elementary Blocking)
Designs, 73
2.24 Blocking With Several Conditions, 76
2.25 Composite Variables: Linear Combinations, 76
2.26 Models in Matrix Form, 77
2.27 Graphical Approaches, 79
2.27.1 Box-and-Whisker Plots, 79
2.28 What Makes a p-Value Small? A Critical Overview and Practical Demonstration
of Null Hypothesis Significance Testing, 82
2.28.1 Null Hypothesis Significance Testing (NHST): A Legacy of Criticism, 82
2.28.2 The Make-Up of a p-Value: A Brief Recap and Summary, 85
2.28.3 The Issue of Standardized Testing: Are Students in Your School Achieving
More than the National Average?, 85
2.28.4 Other Test Statistics, 86
2.28.5 The Solution, 87
2.28.6 Statistical Distance: Cohen’s d, 87
2.28.7 What Does Cohen’s d Actually Tell Us?, 88
2.28.8 Why and Where the Significance Test Still Makes Sense, 89
2.29 Chapter Summary and Highlights, 89
Review Exercises, 92
Further Discussion and Activities, 95
CONTENTS ix

3 Analysis of Variance: Fixed Effects Models 97

3.1 What is Analysis of Variance? Fixed Versus Random Effects, 98
3.1.1 Small Sample Example: Achievement as a Function of Teacher, 99
3.1.2 Is Achievement a Function of Teacher?, 100
3.2 How Analysis of Variance Works: A Big Picture Overview, 101
3.2.1 Is the Observed Difference Likely? ANOVA as a Comparison (Ratio) of
Variances, 102
3.3 Logic and Theory of ANOVA: A Deeper Look, 103
3.3.1 Independent-Samples t-Tests Versus Analysis of Variance, 104
3.3.2 The ANOVA Model: Explaining Variation, 105
3.3.3 Breaking Down a Deviation, 106
3.3.4 Naming the Deviations, 107
3.3.5 The Sums of Squares of ANOVA, 108
3.4 From Sums of Squares to Unbiased Variance Estimators: Dividing by Degrees of
Freedom, 109
3.5 Expected Mean Squares for One-Way Fixed Effects Model: Deriving the F-ratio, 110
3.6 The Null Hypothesis in ANOVA, 112
3.7 Fixed Effects ANOVA: Model Assumptions, 113
3.8 A Word on Experimental Design and Randomization, 115
3.9 A Preview of the Concept of Nesting, 116
3.10 Balanced Versus Unbalanced Data in ANOVA Models, 116
3.11 Measures of Association and Effect Size in ANOVA: Measures of Variance Explained, 117
3.11.1 η2 Eta-Squared, 117
3.11.2 Omega-Squared, 118
3.12 The F-Test and the Independent Samples t-Test, 118
3.13 Contrasts and Post-Hocs, 119
3.13.1 Independence of Contrasts, 122
3.13.2 Independent Samples t-Test as a Linear Contrast, 123
3.14 Post-Hoc Tests, 124
3.14.1 Newman–Keuls and Tukey HSD, 126
3.14.2 Tukey HSD, 127
3.14.3 Scheffé Test, 128
3.14.4 Other Post-Hoc Tests, 129
3.14.5 Contrast Versus Post-Hoc? Which Should I be Doing?, 129
3.15 Sample Size and Power for ANOVA: Estimation With R and G∗Power, 130
3.15.1 Power for ANOVA in R and G∗Power, 130
3.15.2 Computing f, 130
3.16 Fixed effects One-Way Analysis of Variance in R: Mathematics Achievement
as a Function of Teacher, 133
3.16.1 Evaluating Assumptions, 134
3.16.2 Post-Hoc Tests on Teacher, 137
3.17 Analysis of Variance Via R’s lm, 138
3.18 Kruskal-Wallis Test in R and the Motivation Behind Nonparametric Tests, 138
3.19 ANOVA in SPSS: Achievement as a Function of Teacher, 140
3.20 Chapter Summary and Highlights, 142
Review Exercises, 143
Further Discussion and Activities, 145
x CONTENTS

4 Factorial Analysis of Variance: Modeling Interactions 146

4.1 What is Factorial Analysis of Variance?, 146
4.2 Theory of Factorial ANOVA: A Deeper Look, 148
4.2.1 Deriving the Model for Two-Way Factorial ANOVA, 149
4.2.2 Cell Effects, 150
4.2.3 Interaction Effects, 151
4.2.4 Cell Effects Versus Interaction Effects, 152
4.2.5 A Model for the Two-Way Fixed Effects ANOVA, 152
4.3 Comparing One-Way ANOVA to Two-Way ANOVA: Cell Effects in Factorial ANOVA
Versus Sample Effects in One-Way ANOVA, 153
4.4 Partitioning the Sums of Squares for Factorial ANOVA: The Case of Two Factors, 153
4.4.1 SS Total: A Measure of Total Variation, 154
4.4.2 Model Assumptions: Two-Way Factorial Model, 155
4.4.3 Expected Mean Squares for Factorial Design, 156
4.4.4 Recap of Expected Mean Squares, 159
4.5 Interpreting Main Effects in the Presence of Interactions, 159
4.6 Effect Size Measures, 160
4.7 Three-Way, Four-Way, and Higher Models, 161
4.8 Simple Main Effects, 161
4.9 Nested Designs, 162
4.9.1 Varieties of Nesting: Nesting of Levels Versus Subjects, 163
4.10 Achievement as a Function of Teacher and Textbook: Example of Factorial
ANOVA in R, 164
4.10.1 Comparing Models Through AIC, 167
4.10.2 Visualizing Main Effects and Interaction Effects Simultaneously, 169
4.10.3 Simple Main Effects for Achievement Data: Breaking Down
Interaction Effects, 170
4.11 Interaction Contrasts, 171
4.12 Chapter Summary and Highlights, 172
Review Exercises, 173

5 Introduction to Random Effects and Mixed Models 175

5.1 What is Random Effects Analysis of Variance?, 176
5.2 Theory of Random Effects Models, 177
5.3 Estimation in Random Effects Models, 178
5.3.1 Transitioning from Fixed Effects to Random Effects, 178
5.3.2 Expected Mean Squares for MS Between and MS Within, 179
5.4 Defining Null Hypotheses in Random Effects Models, 180
5.4.1 F-Ratio for Testing H0, 181
5.5 Comparing Null Hypotheses in Fixed Versus Random Effects Models:
The Importance of Assumptions, 182
5.6 Estimating Variance Components in Random Effects Models: ANOVA,
ML, REML Estimators, 183
5.6.1 ANOVA Estimators of Variance Components, 183
5.6.2 Maximum Likelihood and Restricted Maximum Likelihood, 184
5.7 Is Achievement a Function of Teacher? One-Way Random Effects Model in R, 185
5.7.1 Proportion of Variance Accounted for by Teacher, 187
5.8 R Analysis Using REML, 188
CONTENTS xi

5.9 Analysis in SPSS: Obtaining Variance Components, 188

5.10 Factorial Random Effects: A Two-Way Model, 190
5.11 Fixed Effects Versus Random Effects: A Way of Conceptualizing Their Differences, 191
5.12 Conceptualizing the Two-Way Random Effects Model: The Make-Up of a
Randomly Chosen Observation, 192
5.13 Sums of Squares and Expected Mean Squares for Random Effects: The
Contaminating Influence of Interaction Effects, 193
5.13.1 Testing Null Hypotheses, 194
5.14 You Get What You Go In With: The Importance of Model Assumptions and
Model Selection, 195
5.15 Mixed Model Analysis of Variance: Incorporating Fixed and Random Effects, 196
5.15.1 Mixed Model in R, 199
5.16 Mixed Models in Matrices, 199
5.17 Multilevel Modeling as a Special Case of the Mixed Model: Incorporating Nesting
and Clustering, 200
5.18 Chapter Summary and Highlights, 201
Review Exercises, 202

6 Randomized Blocks and Repeated Measures 204

6.1 What is a Randomized Block Design?, 205
6.2 Randomized Block Designs: Subjects Nested Within Blocks, 205
6.3 Theory of Randomized Block Designs, 207
6.3.1 Nonadditive Randomized Block Design, 208
6.3.2 Additive Randomized Block Design, 209
6.4 Tukey Test for Nonadditivity, 211
6.5 Assumptions for the Covariance Matrix, 212
6.6 Intraclass Correlation, 213
6.7 Repeated Measures Models: A Special Case of Randomized Block Designs, 215
6.8 Independent Versus Paired-Samples t-Test, 215
6.9 The Subject Factor: Fixed or Random Effect?, 216
6.10 Model for One-Way Repeated Measures Design, 217
6.10.1 Expected Mean Squares for Repeated Measures Models, 217
6.11 Analysis Using R: One-Way Repeated Measures: Learning as a Function of Trial, 218
6.12 Analysis Using SPSS: One-Way Repeated Measures: Learning as a Function of Trial, 222
6.12.1 Which Results Should Be Interpreted?, 224
6.13 SPSS Two-Way Repeated Measures Analysis of Variance Mixed Design:
One Between Factor, One Within Factor, 226
6.13.1 Another Look at the Between-Subjects Factor, 229
6.14 Chapter Summary and Highlights, 230
Review Exercises, 231

7 Linear Regression 232

7.1 Brief History of Regression, 233
7.2 Regression Analysis and Science: Experimental Versus Correlational Distinctions, 235
7.3 A Motivating Example: Can Offspring Height Be Predicted?, 236
7.4 Theory of Regression Analysis: A Deeper Look, 238
7.5 Multilevel Yearnings, 240
7.6 The Least-Squares Line, 240
xii CONTENTS

7.7 Making Predictions Without Regression, 241

7.8 More about εi, 243
7.9 Model Assumptions for Linear Regression, 243
7.9.1 Model Specification, 245
7.9.2 Measurement Error, 245
7.10 Estimation of Model Parameters in Regression, 246
7.10.1 Ordinary Least-Squares (OLS), 247
7.11 Null Hypotheses for Regression, 248
7.12 Significance Tests and Confidence Intervals for Model Parameters, 250
7.13 Other Formulations of the Regression Model, 251
7.14 The Regression Model in Matrices: Allowing for More Complex Multivariable Models, 252
7.15 Ordinary Least-Squares in Matrices, 255
7.16 Analysis of Variance for Regression, 256
7.17 Measures of Model Fit for Regression: How Well Does the Linear Equation Fit?, 259
7.18 Adjusted R2, 260
7.19 What “Explained Variance” Means and More Importantly, What It Does Not Mean, 260
7.20 Values Fit by Regression, 261
7.21 Least-Squares Regression in R: Using Matrix Operations, 262
7.22 Linear Regression Using R, 265
7.23 Regression Diagnostics: A Check on Model Assumptions, 267
7.23.1 Understanding How Outliers Influence a Regression Model, 268
7.23.2 Examining Outliers and Residuals, 269
7.23.3 Detecting Outliers, 272
7.23.4 Normality of Residuals, 274
7.24 Regression in SPSS: Predicting Quantitative from Verbal, 275
7.25 Power Analysis for Linear Regression in R, 279
7.26 Chapter Summary and Highlights, 281
Review Exercises, 283
Further Discussion and Activities, 285

8 Multiple Linear Regression 286

8.1 Theory of Partial Correlation, 287
8.2 Semipartial Correlations, 288
8.3 Multiple Regression, 289
8.4 Some Perspective on Regression Coefficients: “Experimental Coefficients”?, 290
8.5 Multiple Regression Model in Matrices, 291
8.6 Estimation of Parameters, 292
8.7 Conceptualizing Multiple R, 292
8.8 Interpreting Regression Coefficients: Correlated Versus Uncorrelated Predictors, 293
8.9 Anderson’s Iris Data: Predicting Sepal Length From Petal Length and Petal Width, 293
8.10 Fitting Other Functional Forms: A Brief Look at Polynomial Regression, 297
8.11 Measures of Collinearity in Regression: Variance Inflation Factor and Tolerance, 298
8.12 R-squared as a Function of Partial and Semipartial Correlations:
The Stepping Stones to Forward and Stepwise Regression, 300
8.13 Model-Building Strategies: Simultaneous, Hierarchical, Forward, Stepwise, 301
8.13.1 Simultaneous, Hierarchical, Forward, 303
8.13.2 Stepwise Regression, 305
8.13.3 Selection Procedures in R, 306
CONTENTS xiii

8.13.4 Which Regression Procedure Should Be Used? Concluding Comments and

Recommendations Regarding Model-Building, 306
8.14 Power Analysis for Multiple Regression, 307
8.15 Introduction to Statistical Mediation: Concepts and Controversy, 307
8.15.1 Statistical Versus True Mediation: Some Philosophical Pitfalls in the
Interpretation of Mediation Analysis, 309
8.16 Brief Survey of Ridge and Lasso Regression: Penalized Regression Models
and the Concept of Shrinkage, 311
8.17 Chapter Summary and Highlights, 313
Review Exercises, 314
Further Discussion and Activities, 315

9 Interactions in Multiple Linear Regression 316

9.1 The Additive Regression Model With Two Predictors, 317
9.2 Why the Interaction is the Product Term xizi: Drawing an Analogy to Factorial
ANOVA, 318
9.3 A Motivating Example of Interaction in Regression: Crossing a Continuous Predictor
With a Dichotomous Predictor, 319
9.4 Analysis of Covariance, 323
9.4.1 Is ANCOVA “Controlling” for Anything?, 325
9.5 Continuous Moderators, 326
9.6 Summing Up the Idea of Interactions in Regression, 326
9.7 Do Moderators Really “Moderate” Anything?, 326
9.7.1 Some Philosophical Considerations, 326
9.8 Interpreting Model Coefficients in the Context of Moderators, 327
9.9 Mean-Centering Predictors: Improving the Interpretability of Simple Slopes, 328
9.10 Multilevel Regression: Another Special Case of the Mixed Model, 330
9.11 Chapter Summary and Highlights, 331
Review Exercises, 331

10 Logistic Regression and the Generalized Linear Model 333

10.1 Nonlinear Models, 335
10.2 Generalized Linear Models, 336
10.2.1 The Logic of the Generalized Linear Model: How the Link
Function Transforms Nonlinear Response Variables, 337
10.3 Canonical Links, 338
10.3.1 Canonical Link for Gaussian Variable, 339
10.4 Distributions and Generalized Linear Models, 339
10.4.1 Logistic Models, 339
10.4.2 Poisson Models, 340
10.5 Dispersion Parameters and Deviance, 340
10.6 Logistic Regression, 341
10.6.1 A Generalized Linear Model for Binary Responses, 341
10.6.2 Model for Single Predictor, 342
10.7 Exponential and Logarithmic Functions, 343
10.7.1 Logarithms, 345
10.7.2 The Natural Logarithm, 346
10.8 Odds and the Logit, 347
xiv CONTENTS

10.9 Putting It All Together: Logistic Regression, 348

10.9.1 The Logistic Regression Model, 348
10.9.2 Interpreting the Logit: A Survey of Logistic Regression Output, 348
10.10 Logistic Regression in R, 351
10.10.1 Challenger O-ring Data, 351
10.11 Challenger Analysis in SPSS, 354
10.11.1 Predictions of New Cases, 356
10.12 Sample Size, Effect Size, and Power, 358
10.13 Further Directions, 358
10.14 Chapter Summary and Highlights, 359
Review Exercises, 360

11 Multivariate Analysis of Variance 361

11.1 A Motivating Example: Quantitative and Verbal Ability as a Variate, 362
11.2 Constructing the Composite, 363
11.3 Theory of MANOVA, 364
11.4 Is the Linear Combination Meaningful?, 365
11.4.1 Control Over Type I Error Rate, 365
11.4.2 Covariance Among Dependent Variables, 366
11.4.3 Rao’s Paradox, 367
11.5 Multivariate Hypotheses, 368
11.6 Assumptions of MANOVA, 368
11.7 Hotelling’s T2: The Case of Generalizing From Univariate to Multivariate, 369
11.8 The Covariance Matrix S, 373
11.9 From Sums of Squares and Cross-Products to Variances and Covariances, 375
11.10 Hypothesis and Error Matrices of MANOVA, 376
11.11 Multivariate Test Statistics, 376
11.11.1 Pillai’s Trace, 378
11.11.2 Lawley–Hotelling’s Trace, 379
11.12 Equality of Covariance Matrices, 379
11.13 Multivariate Contrasts, 381
11.14 MANOVA in R and SPSS, 382
11.14.1 Univariate Analyses, 386
11.15 MANOVA of Fisher’s Iris Data, 387
11.16 Power Analysis and Sample Size for MANOVA, 388
11.17 Multivariate Analysis of Covariance and Multivariate Models:
A Bird’s Eye View of Linear Models, 389
11.18 Chapter Summary and Highlights, 389
Review Exercises, 391
Further Discussion and Activities, 393

12 Discriminant Analysis 394

12.1 What is Discriminant Analysis? The Big Picture on the Iris Data, 395
12.2 Theory of Discriminant Analysis, 396
12.2.1 Discriminant Analysis for Two Populations, 397
12.2.2 Substituting the Maximizing Vector into Squared Standardized
Difference, 398
12.3 LDA in R and SPSS, 399
12.4 Discriminant Analysis for Several Populations, 405
CONTENTS xv

12.4.1 Theory for Several Populations, 405

12.5 Discriminating Species of Iris: Discriminant Analyses for Three Populations, 408
12.6 A Note on Classification and Error Rates, 410
12.6.1 Statistical Lives, 412
12.7 Discriminant Analysis and Beyond, 412
12.8 Canonical Correlation, 413
12.9 Motivating Example for Canonical Correlation: Hotelling’s 1936 Data, 414
12.10 Canonical Correlation as a General Linear Model, 415
12.11 Theory of Canonical Correlation, 416
12.12 Canonical Correlation of Hotelling’s Data, 418
12.13 Canonical Correlation on the Iris Data: Extracting Canonical Correlation
From Regression, MANOVA, LDA, 419
12.14 Chapter Summary and Highlights, 420
Review Exercises, 421
Further Discussion and Activities, 422

13 Principal Components Analysis 423

13.1 History of Principal Components Analysis, 424
13.2 Hotelling 1933, 426
13.3 Theory of Principal Components Analysis, 428
13.3.1 The Theorem of Principal Components Analysis, 428
13.4 Eigenvalues as Variance, 429
13.5 Principal Components as Linear Combinations, 429
13.6 Extracting the First Component, 430
13.6.1 Sample Variance of a Linear Combination, 430
13.7 Extracting the Second Component, 431
13.8 Extracting Third and Remaining Components, 432
13.9 The Eigenvalue as the Variance of a Linear Combination Relative to its Length, 432
13.10 Demonstrating Principal Components Analysis: Pearson’s 1901 Illustration, 433
13.11 Scree Plots, 436
13.12 Principal Components Versus Least-Squares Regression Lines, 439
13.13 Covariance Versus Correlation Matrices: Principal Components and Scaling, 441
13.14 Principal Components Analysis Using SPSS, 441
13.15 Chapter Summary and Highlights, 445
Review Exercises, 446
Further Discussion and Activities, 448

14 Factor Analysis 449

14.1 History of Factor Analysis, 450
14.2 Factor Analysis at a Glance, 450
14.3 Exploratory Versus Confirmatory Factor Analysis, 451
14.4 Theory of Factor Analysis: The Exploratory Factor-Analytic Model, 451
14.5 The Common Factor-Analytic Model, 452
14.6 Assumptions of the Factor-Analytic Model, 454
14.7 Why Model Assumptions are Important, 455
14.8 The Factor Model as an Implication for the Covariance Matrix Σ, 456
14.9 Again, Why is Σ = ΛΛ + ψ So Important a Result?, 457
xvi CONTENTS

14.10 The Major Critique Against Factor Analysis: Indeterminacy and the
Nonuniqueness of Solutions, 457
14.11 Has Your Factor Analysis Been Successful?, 459
14.12 Estimation of Parameters in Exploratory Factor Analysis, 460
14.13 Principal Factor, 460
14.14 Maximum Likelihood, 461
14.15 The Concepts (and Criticisms) of Factor Rotation, 462
14.16 Varimax and Quartimax Rotation, 464
14.17 Should Factors Be Rotated? Is That Not Cheating?, 465
14.18 Sample Size for Factor Analysis, 466
14.19 Principal Components Analysis Versus Factor Analysis: Two Key Differences, 466
14.19.1 Hypothesized Model and Underlying Theoretical Assumptions, 466
14.19.2 Solutions are Not Invariant in Factor Analysis, 467
14.20 Principal Factor in SPSS: Principal Axis Factoring, 468
14.21 Bartlett Test of Sphericity and Kaiser-Meyer-Olkin Measure of
Sampling Adequacy (MSA), 474
14.22 Factor Analysis in R: Holzinger and Swineford (1939), 476
14.23 Cluster Analysis, 477
14.24 What is Cluster Analysis? The Big Picture, 478
14.25 Measuring Proximity, 480
14.26 Hierarchical Clustering Approaches, 483
14.27 Nonhierarchical Clustering Approaches, 485
14.28 K-Means Cluster Analysis in R, 486
14.29 Guidelines and Warnings About Cluster Analysis, 489
14.30 A Brief Look at Multidimensional Scaling, 489
14.31 Chapter Summary and Highlights, 492
Review Exercises, 493
Further Discussion and Activities, 496

15 Path Analysis and Structural Equation Modeling 497

15.1 Path Analysis: A Motivating Example—Predicting IQ Across Generations, 498
15.2 Path Analysis and “Causal Modeling”, 500
15.3 Early Post-Wright Path Analysis: Predicting Child’s IQ (Burks, 1928), 502
15.4 Decomposing Path Coefficients, 503
15.5 Path Coefficients and Wright’s Contribution, 504
15.6 Path Analysis in R—A Quick Overview: Modeling Galton’s Data, 505
15.6.1 Path Model in AMOS, 508
15.7 Confirmatory Factor Analysis: The Measurement Model, 510
15.7.1 Confirmatory Factor Analysis as a Means of Evaluating Construct
Validity and Assessing Psychometric Qualities, 512
15.8 Structural Equation Models, 514
15.9 Direct, Indirect, and Total Effects, 515
15.10 Theory of Statistical Modeling: A Deeper Look Into Covariance Structures
and General Modeling, 516
15.11 The Discrepancy Function and Chi-Square, 518
15.12 Identification, 519
15.13 Disturbance Variables, 520
15.14 Measures and Indicators of Model Fit, 521
CONTENTS xvii

15.15 Overall Measures of Model Fit, 522

15.15.1 Root Mean Square Residual and Standardized Root Mean Square
Residual, 522
15.15.2 Root Mean Square Error of Approximation, 523
15.16 Model Comparison Measures: Incremental Fit Indices, 523
15.17 Which Indicator of Model Fit is Best?, 525
15.18 Structural Equation Model in R, 526
15.19 How All Variables Are Latent: A Suggestion for Resolving the Manifest-Latent
Distinction, 528
15.20 The Structural Equation Model as a General Model: Some Concluding
Thoughts on Statistics and Science, 529
15.21 Chapter Summary and Highlights, 530
Review Exercises, 531
Further Discussion and Activities, 533

References 534

Index 548
PREFACE

Technology is not progress. Empathy is. The dogs are watching us.

Now in its second edition, this book provides a general introduction and overview of univariate
through to multivariate statistical modeling techniques typically used in the social, behavioral, and
related sciences. Students reading this book will come from a variety of fields, including psychology,
sociology, education, political science, biology, medicine, economics, business, forestry, nursing,
chemistry, law, among others. The book should be of interest to anyone who desires a relatively com-
pact and succinct survey and overview of statistical techniques useful for analyzing data in these
fields, while also wanting to understand and appreciate some of the theory behind these tools. Spanning
several statistical methods, the focus of the book is naturally one of breadth than of depth into any one
particular technique, focusing on the unifying principles as well as what substantively (scientifically)
can or cannot be concluded from a method when applied to real data. These are topics usually encoun-
tered by upper division undergraduate or beginning graduate students in the aforementioned fields.
The first edition has also been used widely as a reference resource for both students and researchers
working on dissertations, manuscripts, and other publications. It is hoped to provide the student with a
“big picture” overview of how applied statistical modeling works, while at the same time providing him
or her the opportunity in many places to implement, to some extent at least, many of these models using
SPSS and/or R software. References and recommendations for further reading are provided throughout
the text for readers who wish to pursue these topics further. Each topic and software demonstration can
literally be “unpacked” into a deeper discussion, and so long as the reader is aware of this, they will appre-
ciate this book for what it is—a bird’s eye view of applied statistics, and not the “one and only” source
they should refer to when conducting analyses. The book does not pretend to be a complete compendium
of each statistical method it discusses, but rather is a survey of each method in hopes of conveying how
these methods generally “work,” what technical elements unites virtually all of them, and the benefits and
limitations of how they may be used in addressing scientific questions.
This second edition has been revised to make the book clearer and more accessible compared to
the first edition. The book also contains a gentle introduction (“foot in the door”) to a variety of
new topics that did not appear in the first edition. All chapters have been edited to varying degrees

xviii
Another random document with
no related content on Scribd:
1.F.6. INDEMNITY - You agree to indemnify and hold the
Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and distribution
of Project Gutenberg™ electronic works, harmless from all liability,
costs and expenses, including legal fees, that arise directly or
indirectly from any of the following which you do or cause to occur:
(a) distribution of this or any Project Gutenberg™ work, (b)
alteration, modification, or additions or deletions to any Project
Gutenberg™ work, and (c) any Defect you cause.

Section 2. Information about the Mission of

Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,

Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many small
donations ($1 to $5,000) are particularly important to maintaining tax
exempt status with the IRS.

The Foundation is committed to complying with the laws regulating

charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.

Project Gutenberg™ eBooks are often created from several printed

editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.