Ch1Multivariate Analysis-All statistical methods that simultaneously analyze 3Multidimensional Scaling (MDS) identifies “unrecognized” dimensions that data
ns that data estimation and model estimation- No imputation for individual cases, rather
multiple measurements on each individual or object under investigation. Why use
it?• Methodological Benefits• Measurement• Explanation & Prediction•
Hypothesis Testing• Improved decision making. THREE CONVERGING
TRENDS:1 The Rise of Big Data-Unique elements of Big Data focused in five
areas:• Volume(it is the sheer magnitude of information being collected that
initiated the term.)• Variety• Velocity• Veracity• Variability and Value. Impacts
on:• Organizational Decisions and Academic Research – improved decision-
making capabilities as well as the explosion of data available to characterize
situations on dimensions never before available• Analytics and the Analyst –
expansion of the domains of study embracing analytics as well as methodological
challenges due to seemingly “unlimited” data Problems – emerging on
technological, ethical and sociological fronts. 2: Statistical Versus Data Mining
Models: Two approaches to data analysis-• Statistical/data models( – analysis
where a specific model is proposed (e.g., dependent and independent variables to
be analyzed by the general linear model), the model is then estimated and a
statistical inference is made as to its generalizability to the population through
statistical tests. • Data mining/algorithmic models( – models based on
algorithms (e.g., neural networks, decision trees, support vector machine) that are
widely used in many Big Data applications. Their emphasis is on predictive
accuracy rather than statistical inference and explanation. 3: Causal Inference
(Causal inference is the movement beyond statistical inference to the stronger
statement of “cause and effect” in non-experimental situations):- While causal
statements have been primarily conceived as the domain of randomized controlled
experiments, recent developments have provided researchers with .• the
theoretical frameworks for understanding the requirements for causal inferences
in non-experimental settings.• some techniques applicable to data not gathered in
an experimental setting that still allow some causal inferences to be drawn.
Multivariate analysis: statistical techniques that simultaneously analyze
multiple measurements on individuals or objects under investigation.• 1The
variate is a linear combination of variables with empirically determined weights.
2 Measurement Scales : Nonmetric• Nominal – size of number is not related to
the amount of the characteristic being measured.• Ordinal – larger numbers
indicate more (or less) of the characteristic measured, but not how much more (or
less). Metric• Interval – contains ordinal properties, and in addition, there are
equal differences between scale points• Ratio – contains interval scale properties,
and in addition, there is a natural zero point. The Impact of Choice of
Measurement Scale : A. The researcher must identify the measurement scale of
each variable used, so that nonmetric data are not incorrectly used as metric data,
and vice versa. B) Themeasurement scale is also critical in determining which
multivariate techniques are the most applicable to the data, with considerations
made for both independent and dependent variables. 3Measurement Error (
distorts observed relationships and makes multivariate techniques less powerful)•
(Researchers use summated scales). Assessing Measurement Error: •
1Validity&Reliabilitythe degree to which a measure accurately represents what
it is supposed to.• Reliability the degree to which the observed variable measures
the “true” value and is thus error free.2Employing Multivariate Measurement
:to reduce measurement error by improving individual variables, the researcher
may choose to develop multivariate measurements, known as summated scales,
.MANAGING THE MULTIVARIATE MODEL :1Managing the Variate-
A)Variable Specification – preparing the variables for analysis.• Use original
variables – retain most detailed attributes, but may suffer due to
multicollinearity.• Dimensional reduction – develop some form of composites of
original variables. A)User controlled – Exploratory factor analysis (principal
components or common factor analysis).B) Software controlled – Principal
components regression (PCR).B)Variable Selection – identifying the variables
included in the analysis• User controlled – researcher explicitly defines variables
in analysis.A)Confirmatory or combinatorial• Software controlled. A) Sequential
– subset of variables included based on algorithm (e.g., stepwise). B) Constrained
– regression weight constrained to force some to zero or very low, effectively
eliminating those variables from the analysis (e.g., ridge or LASSO).2) Managing
the Dependence Model :A)Single Equation versus Multiple Equation Models•
Single equation – most widely used methods in the past• Multiple equation –
represents interrelated relationships.B)GLM versus GLZ/GLIM• GLM –
classical OLS-based system of techniques• GLZ/GLIM – developed for non
normal response variables. 3) Statistical Significance and Power: Type I error
(α) – probability of rejecting the null hypothesis when it is true. Type II error (β)
– probability of failing to reject the null hypothesis when it is false. Power (1 – β)
– probability of rejecting the null hypothesis when it is false. Power is
Determined by Three Factors :Effect size: the actual magnitude of the effect of
interest (e.g., the difference between means or the correlation between
variables).Alpha (α): as α is set at smaller levels, power decreases. Typically, α
= .05. .Sample size: as sample size increases, power increases. With very large
sample sizes (1,000+), even very small effects can be statistically significant,
raising the issue of practical significance vs. statistical significance. Rules of
Thumb: Statistical Power Analysis• Researchers should always design the study
to achieve a power level of .80 at the desired significance level. • More stringent
significance levels (e.g., .01 instead of .05) require larger samples to achieve the
desired power level. • Conversely, power can be increased by choosing a less
stringent alpha level (e.g., .10 instead of .05).• Smaller effect sizes always require
larger sample sizes to achieve the desired power.• Any increase in power is most
likely achieved by increased sample size. CLASSIFICATION OF
MULTIVARIATE TECHNIQUES :1. Dependence techniques: a variable or
set of variables is identified as the dependent variable to be predicted or explained
by other variables known as independent variables..• Multiple Regression•
Multiple Discriminant Analysis• Logit/Logistic Regression• Multivariate Analysis
of Variance (MANOVA) and Covariance• Conjoint Analysis• Canonical
Correlation• Structural Equations Modeling (SEM)• Partial Least Squares (PLS)
Modeling. 2Interdependence techniques: involve the simultaneous analysis of
all variables in the set, without distinction between dependent variables and
independent variables.• Principal Components and Common Factor Analysis•
Cluster Analysis• Multidimensional Scaling (perceptual mapping)•
Correspondence Analysis. Selecting a Multivariate Technique: • Dependence
relationship:
• How many variables are being predicted?• What is the measurement scale of the
dependent variable?• What is the measurement scale of the predictor variable (s)
Interdependence relationship: • Are you examining relationships between
variables, respondents, or objects? TYPES OF MULTIVARIATE
TECHNIQUES: 1 Dependence Techniques- 1Multiple Regression (a single
metric dependent variable is predicted by several metric independent variables.)-
2MANOVA(several metric dependent variables are predicted by a set of
nonmetric (categorical) independent variables.)-3 Discriminant Analysis( single,
non-metric (categorical) dependent variable is predicted by several metric
independent variables)-4LogisticRegression(single nonmetric dependent variable
is predicted by several metric independent variables)-5 Canonical
Analysis( several metric dependent variables are predicted by several metric
independent variables.) 6Conjoint Analysis (quasi-experimental design based on
attributes and levels of attributes which develops combinations of attributes/levels
which are then evaluated by respondents)- 7Structural Equation Modeling
(SEM)( estimates multiple, interrelated dependence relationships based on two
components: a. Measurement Model b. Structural Model. Two basic
methodologies CB-SEM – covariance-based SEM PLS-SEM – Partial Least
Squares Modeling (variance-based). Interdependence Techniques:
1Exploratory Factor Analysis (EFA)- analyzes the structure of the
interrelationships among a large number of variables to determine a set of
common underlying dimensions (factors).2Cluster Analysis- groups objects
(respondents, products, firms, variables, etc.) so that each object is similar to the
other objects in the cluster and different from objects in all the other clusters.
affect purchase behavior based on customer judgments ofsimilarities or
preferencesand transforms these into distances represented as perceptual maps.
4Correspondence Analysis uses non-metric data and evaluates either linear or
nonlinear relationships in an effort to develop a perceptual map representing the
association between objects (firms, products,etc.) and a set of descriptive
characteristics of the objects. GUIDELINES FOR MULTIVARIATE
ANALYSES ANDINTERPRETATION:1Establish Practical Significance as
well as Statistical Significance.: Practical significance asks the question, “So
what?”• Applicable to both managerial and academic contexts.2 Sample Size
Affects All Results: Small samples – too little statistical power or too easily
“overfitting” the data.• Large Samples – make the statistical tests overly
sensitive.3 Know Your Data:• Multivariate analyses require an even more
rigorous examination of the data because the influence of outliers, violations of
assumptions, and missing data can be compounded across several variables to
create substantial effects4Strive for Model Parsimony:• Irrelevant variables
usually increase a technique’s ability to fit the sample data, but at the expense of
overfitting the sample data and making the results less generalizable to the
population.• Even though irrelevant variables typically do not bias the estimates
of the relevant variables, they can mask the true effects due to an increase in
multicollinearity 5 Look at Your Errors:• starting point for diagnosing the
validity of the obtained results. • indication of the remaining unexplained
relationships6 Simplify Your Models By Separation:Estimate separate models
when possible (e.g., in presence of moderators)7 Validate Your Results.:Use
split-sample or cross-validation to assess generalizability of any model.A
STRUCTURED APPROACH TO MULTIVARIATE MODEL:Stage 1:
Define the Research Problem, Objectives, and Multivariate Technique(s) to be
Used.Stage 2: Develop the Analysis Plan.Stage 3: Evaluate the Assumptions
Underlying the Multivariate Technique(s)Stage 4: Estimate the Multivariate
Model and Assess Overall Model Fit.Stage 5: Interpret the Variate(s)Stage 6:
Validate the Multivariate ModelBUILDING CH2 THE CHALLENGE OF BIG
DATA RESEARCH EFFORT:1Data Management• Many consider most
daunting challenge• Many times majority of research effort expended in this task•
Complexity arises from A) Merging disparate sources of data B)Use of
unstructured data. 2Data Quality• True “value” of analysis may rest in data
quality• Conceptualized in eight dimensions• Many times “hidden” in basic nature
of the data (e.g., binary measures). Graphical Examination:1Fundamental tool
in data examination is graphical examination.types of graphical tools:• Shape:
(Histogram, Bar Chart, Box & Whisker plot, Stem and Leaf plot)• Relationships(
Scatterplot, Outliers).2PRELIMINARY EXAMINATION OF THE DATa:
Univariate Profiling: Examining the Shape of the Distribution(Histograms and
The Normal Curve,Stem & Leaf Diagram, Frequency Distribution)-3Bivariate
Profiling: Examining the Relationship Between Variables(Box & Whiskers Plots,
HBAT Scatterplot)- Bivariate Profiling: Examining Group Differences-
4Multivariate Profile- New Measures of Association. Wide range of existing
measures beyond Pearson correlation (parametric and non-parametric).5New
measures from data mining:• Hoeffding’s D • nonparametric measure of
association based on departures from independence.• dCor (the distance
correlation)• distance-based measure of association which also is more sensitive
to nonlinear patterns in the data.• MIC (mutual information correlation)• Pattern-
matching approach amenable to identifying both non-linear relationships as well
as a range of distinct patterns.MISSING DATA:Missing Data: information not
available for a subject (or case) about whom other information is available.
Typically occurs when respondent fails to answer one or more questions in a
survey. Systematic? Random?Impact: Reduces sample size available for
analysis, Can distort results.Researcher’s Concern: to identify the patterns and
relationships underlying the missing data in order to maintain as close as possible
to the original distribution of values when any remedy is applied. Two major
issues experiencing a resurgence of interest.• Wide range of data sources now
being used in analysis.• Expanded availability and improved usability of model-
based methods of imputation.Corresponding increase in:• Study and
applications across wide range of disciplines.• Model-based methods availability
in all major software.Four-Step Process for Identifying Missing Data 1:
Determine the Type of Missing Data:Ignorable Missing Data – expected and
part of research design.• Sample – form of missing data, with excluded data the
remaining population.• Part of data collection – e.g., skip patterns.• Censored data
– some data not yet observed (e.g., survival data).Not Ignorable Missing Data –
data which must be addressed in the analysis.• Known process – identified due to
procedural factors (e.g., data entry or data management).• Unknown process –
primarily related to respondent, but important characteristic is level of
randomness (e.g., straight lining; lack of attention).Levels of Missingness:Three
levels:• Item-level – missing data for individual variable.• Construct-level –
missing data for entire set of questions about a specific construct.• Person-level –
missing data related to individual’s willingness or ability to provide responses. 2:
Determine the Extent of Missing Data:Basic question: Is the extent or amount
of missing data is low enough to not affect the results, even if it operates in a
nonrandom manner.Levels of analysis: percentage of data missing by • Variable
– common form of assessment.• Case – amount of missing data across all
variables by case.Guidelines for deleting variables and/or cases:• 10 percent or
less generally acceptable – cases or observations with 10% or less are amenable to
any imputation strategy.• Sufficient sample size – be sure missing data remedy
provides adequate sample size.• Cases with missing data for dependent variable(s)
typically are deleted.• When deleting a variable, ensure that alternative variables,
hopefully highly correlated, are available to represent the intent of the original
variable.• Perform the analysis both with and without the deleted cases or
variables.3: Diagnose the Randomness of the Missing Data Processes : Levels
of Randomness of the Missing Data Process• Missing Data at Random
(MAR)-missing values of Y depend on X, but not on Y- Example – observed Y
values represent a random sample of the actual Y values for each value of X, but
the observed data for Y do not necessarily represent a truly random sample of all
Y values.• Missing Completely at Random (MCAR):• observed values of Y are
truly a random sample of all Y values.• no underlying association to the other
observed variables, characterized as “purely haphazard missingness”.• Not
Missing at Random (NMAR)-Distinct non-random pattern of missing data-
Non-random pattern not related to any other variables-Example: all individuals
with high income had missing data.Diagnostic Tests for Levels of
Randomness:t test of Missingness• Test of differences between cases with
missing data versus not missing data on other variables:1. For specific variable
(e.g., X1), create two groups of cases – cases with missing values on X1 and those
with valid values on X12. Compare these two groups with a t test for differences
on other variables in the analysis (e.g., X2, X3 ….)3. Differences indicate MAR
processes, no differences indicate MCAR processes.Little’s MCAR Test•
analyzes the pattern of missing data on all variables and compares it with the
pattern expected for a random missing data process. • If no significant differences
are found, the missing data can be classified as MCAR.MAR or MCAR?• Useful
for selecting remedy, but less impactful when using model-based methods. 4:
Select the Imputation Method- Imputation of MCAR Using Only Valid Data:
IF MCAR, several approaches available:1. Using only valid data• Complete
case approachA) Use only cases with no missing data.• Using all-available
dataB)Calculate imputed values based on all valid pairwise information.2. Using
known replacement data• Hot or Cold Deck imputation• Case substitution3.
Calculating replacement values• Mean substitutionA) replaces the missing
values with the mean value of that variable calculated from all valid responses.•
Regression imputationB) predict the missing values of a variable based on its
relationship to other variables in the dataset. Imputation of a MAR Missing
Data Process: Best remedy is some form of model-based approach:Two forms of
model-based imputation which rely upon MAR relationships to estimate
missing data:• Maximum likelihood and EM: Single step process of missing
direct estimation of means and covariance matrix.• Multiple imputation:
Estimation of imputed values for missing data of individual cases by specified
model- Calculates multiple sets of imputed values, each set varying by adding a
random element to imputed values and then forming a separate dataset for
estimation-Model estimates made for each imputed dataset and then combined for
final model estimates.Choosing between maximum likelihood and multiple
imputation:Multiple imputation uses conventional techniques for model
estimation while maximum likelihood limited in applicable methods.Choosing
Imputation Based on:Extent of missing data:• Under 10% – Any of the
imputation methods can be applied, complete case method has been shown to be
the least preferred.• 10% to 20% – all-available, hot deck case substitution, and
regression methods most preferred for MCAR data, whereas model-based
methods are necessary with MAR missing data processes.• Over 20% – if
necessary, the preferred methods are:A) The regression method for MCAR
situations.B) Model-based methods when MAR missing data occur.Type of
missing data process:• MCAR – any imputation method can provide unbiased
estimates if MCAR conditions met, but the model-based methods are preferred.•
MAR – only model-based methods.Outlier:An observation/response with a
unique combination of characteristics and identifiable as distinctly different from
the other observations/responses.Contexts for defining outliers: Pre-analysis
Context: A Member of a Population.• focus is on each case as compared to the
other observations under study.Post-analysis Context: Meeting Analysis
Expectations.• defines “normal” as the expectations (e.g., predicted values, group
membership predictions, etc.) generated by the analysis of interest .Impacts of
Outliers: Practical Impacts• Can have substantial impact on the results of any
analysis.Substantive Impacts• Non-representative outliers can distort results and
lame them less generalizable to the population.Outliers– • Good – identify
perhaps, small, but unique, portions of the sample that should be included.• Bad –
distort results and impact generalizability.Classifying Outliers: Types of impacts
of outliers:• Error outliers – differ from expected values generated by the
analysis.• Interesting outliers – different enough to generate insight into the
analysis.• Influential outliers – different enough to substantively impact the
results.Reasons for Outlier Designation:• Procedural Error.• Extraordinary
Event.• Extraordinary Observations.• Observations unique in their combination of
values. Detecting Outliers: • Standardize data and then identify outliers in terms
of number of standard deviations.• Examine data using Box Plots, Stem & Leaf,
and Scatterplots.• Multivariate detection (Mahalanobis D2).Univariate methods
– examine all metric variables to identify unique or extreme observations. • For
small samples (80 or fewer observations), outliers typically are defined as cases
with standard scores of 2.5 or greater. • For larger sample sizes, increase the
threshold value of standard scores up to 4. • If standard scores are not used,
identify cases falling outside the ranges of 2.5 versus 4 standard deviations,
depending on the sample size.Bivariate methods – focus their use on specific
variable relationships, such as the independent versus dependent variables: • use
scatterplots with confidence intervals at a specified Alpha level.Multivariate
methods – best suited for examining a complete variate, such as the independent
variables in regression or the variables in factor analysis: • threshold levels for the
D2/df measure should be very conservative (.005 or .001), resulting in values of
2.5 (small samples) versus 3 or 4 in larger samples.Impact of Dimensionality
:Increased dimensionality (i.e., increased number of variables) dramatically
impacts outlier detection and designation in three ways:1. Distance measures
become less useful• higher levels of dimensionality create a “natural” dispersion
among observations that makes distance measures less useful for identifying
observations.2. Impact of irrelevant variables• as dimensionality increases, the
presence of irrelevant variables has higher likelihood, thus confounding the ability
to identify outliers.3. Compatibility of dimensions• as dimensionality increases
through the use of multiple sources of data, especially unstructured data, methods
for assessing comparability among observations becomes more difficult.Dealing
with Outliers:1Outlier Designation• Researcher judgment should guide
designation of outliers versus a strictly empirical designation.2Outlier
Description and Profiling• Outliers should be described on the variables used to
compare between observations.• Profiles on additional variables should be
generated when possible to provide more insight into the character of
outliers.3Retention versus Deletion• Should be retained unless demonstrable
proof indicates that they are truly aberrant and not representative of any
observations in the population.• If possible generate results with and without
outliers to assess impact.• Methods to minimize outlier influence (e.g., robust
methods) are available.Need For Testing of Assumptions:Foundation for
making statistical inferences and results.Need is increased in multivariate analysis
because the complexity of the analysis:1. Makes the potential distortions and
biases more potent when the assumptions are violated.2. May mask the indicators
of assumption violations apparent in the simpler univariate analyses.Important
Note: Must test for assumptions twice –• Individual variables – to understand
basic sources of problems.• Variate – to assess the combined effect across all
variables.Four Important Statistical Assumptions:1Normality• Comparison of
distribution to normal distribution.• Basis for statistical inference from sample to
population.2Homoscedasticity• Variance of the error terms appears constant over
a range of predictor variables.• Heteroscedasticity is when error terms have
increasing or modulating variance.• Analysis of residuals best illustrates this
point.3Linearity• Relationship represented by a straight line (i.e., constant unit
change (slope) of the dependent variable for a constant unit change of the
independent variable.4 Non-correlated Errors• Prediction errors are
uncorrelated with each other.Normality Assumptions : Univariate versus
Multivariate Normality.• 1Univariate normality – each individual variable.•
Multivariate normality – combinations of variables.2Impacts of Assumption
Violations• Shape of Distribution – skewness versus kurtosis.• Impact of sample
size – increased sample size reduces detrimental effects.3Testing for Normality
Assumptions• Visual check of histogram or normal probability plot.• Statistical
tests of skewness and kurtosis.4Remedies• Most often some form of data
transformation.HomoscedasticityAssumption.: Impact of Heteroscedasticity –
inflates/deflates standard errorsc. 1Sources :• Variable type – common in
percentages or proportions.• Skewed distribution – one or both variables.2Tests :•
Graphical test.• Statistical tests.A) Levene test (univariate)B) Box’s M
(multivariate).3Remedies for • Transformation of variable(s).• Use of
heteroscedasticity-consistent standard errors (HCSE).Nonlinear relationships:•
can be very well defined, but seriously understated unless:A) data is transformed
to a linear pattern, or B) explicit model components are used to represent the
nonlinear portion of the relationship.Correlated errors:• arise from a process that
must be treated much like missing data: A) Researcher must first identify and
define the “causes” among variables, either internal or external to the dataset (e.g.,
grouping or time series).B)If they are not found and remedied, serious biases can
occur in the results, many times Unknown to the researcher. • Remedies:A)
Inclusion of omitted causal factor underlying correlation of errors.B) Apply
specialized model forms (e.g., multi-level linear models).Data
Transformations:Provide a means of modifying variables for one of four
reasons:1. Enhancing statistical properties.• Primarily to achieve normality,
homoscedasticity or linearity.2. Ease of interpretation.• Standardization –
performed across cases to provide common metric for comparison• Centering –
performed within-case to allow for comparison across variables.3. Representing
specific relationship types.• Transformed variables represent unique
relationships – (e.g., elasticity).4. Simplification.• Binning – categorization of
values into a smaller number of categories (i.e., reduce cardinality).A)
Dichotomization – frequently employed to form two groups (e.g., mean-split).B)
Extreme groups – define three categories, eliminate middle group to accentuate
differences.• Smoothing – use of response surface methods or other techniques to
represent generalized patterns in the data.Guidelines for Transforming
Data:When explanation is important, beware of transformation• 1)To judge the
potential impact of a transformation, calculate the ratio of the variable’s mean to
its standard deviation: A)Noticeable effects should occur when the ratio is less
than 4. • B)When the transformation can be performed on either of two variables,
select the variable with the smallest ratio .• 2)Generally applied to the
independent variables except in the case of heteroscedasticity.•A)
Heteroscedasticity can be remedied only by the transformation of the dependent
variable in a dependence relationship. If a heteroscedastic relationship is also
nonlinear, the dependent variable, and perhaps the independent variables, must be
transformed.•3) Transformations may change the interpretation of the variables.
A)• For example, transforming variables by taking their logarithm translates the
relationship into a measure of proportional change (elasticity). Always be sure to
explore thoroughly the possible interpretations of the transformed variables.B)•
Use variables in their original (untransformed) format when profiling or
interpreting results. Dummy variable: a nonmetric independent variable that has
two distinct levels that are coded 0 and 1. These variables act as replacement
variables to enable multi-category (3 or more) nonmetric variables to be used as
metric variables.CH3 EFA:An interdependence technique whose primary purpose
is to define the underlying structure among the variables in the analysis.• EFA is a
summarization and data reduction technique that does not have independent and
dependent variables, but is an interdependence technique in which all variables
are considered simultaneously. Types of Factor Analysis:(EFA)• used to
discover the factor structure of a construct and examine its reliability. It is data
driven.Confirmatory Factor Analysis (CFA)• used to confirm the fit of the
hypothesized factor structure to the observed (sample) data. It is theory driven.
STAGE 1: OBJECTIVES OFEFA: Specifying Unit of Analysis:• Variables –to
summarize charactersitics, use R-factor analysis.• Cases – grouping cases, similar
to cluster analysis, termed Q-factor analysis.Data Summarization/Data
Reduction• Data summarization – • Data reduction – .Variable Selection :
Three Elements in variable selection1. Variable specification .2. Factors are
always produced – 3. Factors require multiple variables . Using Factor Analysis
with Other Multivariate Techniques. Stage2 Designing EFA:Variable
selection-Specialized methods exist for the use of dummy variables, but a small
number of “dummy variables” can be included in a set of metric variables that are
factor analyzed.Sample size:mimum sample size 50 observation.Correlatin
matrx: (“R analysis”) most often performed, although similarity/distance matrix
(“Q analysis”) also an option.ROT3-1::Factor Analysis Design: 1) Factor
analysis is performed most often only on metric variables, although
specialized methods exist for the use of dummy variables; a small number of
“dummy variables” can be included in a set of metric variables that are
factor analyzed 2) If a study is being designed to reveal factor structure,
strive to have at least five variables for each proposed factor 3) For sample
size: • The sample must have more observations than variables • The
minimum absolute sample size should be 50 observations • Strive to
maximize the number of observations per variable, with a desired ratio of 5
observations per variable. Stage 3: Assumptions in Factor Analysis
1)Conceptual: assume a homogeneity of sub sample factor solution 2)Stastical
issuse:- A statistically significant Bartlett’s test of sphericity (sig. < .05) indicates
that sufficient correlations exist among the variables to proceed with an
exploratory factor analysis- the KMO predicts if data are likely to factor well
based on correlation and partial correlation- KMO statistic exists for each
individual variable and an overall value that can vary from 0 to 1.0.. If KMO
value must exceed .50 for both the overall test and each individual variable. Stage
4:Deriving factors & assessing overall fit:1Partitioning the
Variance:Common variance: Variance of a variable that is shared with all other
variables in the analysis.Unique variance – composed of Specific(cannot be
explained by the correlations to the other variables but reflects the unique
characteristics )and Error variance(due to unreliability in the data-gathering
process). Optimal scaling is a process to derive interval measurement properties
for variables which were originally nominal or ordinal measures. Principal
component (data reduction is a primary concern, prior knowledge suggests that
specific and error variance represent a relatively small proportion of the total
variance) vs Common factor analysis (obj = to identify latent dimensions or
constructs represented in common variance of the original variables, use when
reasearcher has little knowledge about specific &error variance). Stoping Rule:
Criteria for number of factors to extract: priori criteria: researcher knows
how many factors to extract and specifies for factor extraction . Latent root
criteria/kaiser rule: • factors having latent roots or eigenvalues greater than 1
are considered significant.• most applicable to principal components analysis
where the diagonal value representing the amount of variance for each variable is
1.0.• less accurate with a small number of variables or lower
communalities.Percentage of Variance: Extract enough components to achieve a
specified cumulative percentage of total variance extracted .Scree test: identify
the optimum no. of factors that can be extracted bf the amount of unique variance
begins to dominate the common variance structure. Elbow: the point at which the
curve first begins to straighten out.Parallel Analysis: empirical measure based on
the specific characteristics. Rules of thumb:• the principal component analysis
model is most appropriate when data reduction is paramount .:•decision on the
number of retained factors should be based on several considerations: - Factors
With Eigenvalues greater than 1.0. - Enough factors for a specified percentage of
variance explained (e.g., usually 60%).- Factors which have eigenvalues greater
than factors from randomly-generated data.- Factors above the threshold
established by parallel analysis. - More factors when there is heterogeneity among
sample subgroupsStage 5: Interpreting the Factors: 3 process: 1.1)Estimate the
Factor Matrix (compute unrotated factor loadings – correlation of variable and
factor).1.2)Factor Rotation:Impact• The ultimate effect of rotating the factor
matrix is to redistribute the variancefrom earlier factors to later ones to achieve a
simpler, theoretically more meaningful factor pattern.1.3Alternative Methods•
Orthogonal rotation – n the research goal is data reduction to either a smaller
number of variables or a set of uncorrelated measures for subsequent use in other
multivariate techniques.–Quartimax (simplify rows) - Varimax (simplify
columns) - Equimax (combination• Oblique rotation – obtaining several
theoretically meaningful factors or constructs because, realistically, very few
constructs in the “real world” are uncorrelated: .4) Significance of Facto•
Equivalent to zero: less than +/- .10• Minimal Level: in the range of +/- .30 to
+/- .40• Practically Significant: values of +/- .50 or greater• Well-defined
structure: exceeding +/- .70r 5)•. A smaller loading is needed given either a larger
sample size or a larger number of variables.• A larger loading is needed given a
factor solution . Intrepreting factor matrix:Five step process1. Examine the
factor matrix of loadings 2) PROCESS TO IDENTIFY:A). Identify potential
cross-loadings. B)Compute the ratio of the squared loadings.C)Designate the pair
of loadings as follows based on the ratio:• Between 1.0 and 1.5 — problematic
cross-loading • Between 1.5 and 2.0 — potential cross-loading• Greater than 2.0
— ignorable cross-loading.3). Assess the communalities of the variable: should be
greater than 0.5 .4). Respecify the factor model if needed.• Ignore the
problematic variable.• Deletion of variables from the analysis.• Use a different
rotational approach• Extract a different number of factors.• Change the method of
extraction. 5). Label the factors:Variables with higher loadings are considered
more important and have greater influence on the name or label selected to
represent a factor.STAGE 6: VALIDATION OFEXPLORATORY FACTOR
ANALYSIS:Use of Replication or a Confirmatory Perspective. Assessing Factor
Structure Stability.Detecting Influential Observations. STAGE 7:
ADDITIONAL USES OF EFA RESULTS-A)Selecting Surrogate Variables
for Subsequent Analysis:Adva: • simple to administer and interpret.Disadv:•
does not represent all “facets” of a factor. • prone to measurement error.Creating
Summated Scales: Advan:• compromise between the surrogate variable and
factor score options.• reduces measurement error.• represents multiple facets of a
concept.• easily replicated across studies.Disadvs:• includes only the variables
that load highly on the factor.• not necessarily orthogonal.• requires extensive
analysis of reliability and validity issues.Assessing Summated
Scales:a)Unidimensionality• Never create a summated scale without first
assessing its unidimensionality with exploratory or confirmatory factor
analysis.B)Reliability• should exceed a threshold of .70, although a .60 level can
be used in exploratory research. • the threshold should be raised as the number of
items increases, especially as the number of items approaches 10 or
more.C)Validity:• convergent validity = scale correlates with other similar
scales.• discriminant validity = scale is sufficiently different from other related
scales.• nomological validity = scale “predicts” as theoretically suggested.
Computing Factor Scores: Advant:• represents all variables loading on the
factor.• best method for complete data reduction. • by default, factors (and factor
scores) are orthogonal and can avoid complications caused by
multicollinearity.Disadvas:• interpretation more difficult since all variables
contribute through loadings. • difficult to replicate across studies.ch 5 : Multiple
Regression:Statistical technique that can be used to analyze the relationship
between a single dependent (criterion) variable and several independent
(predictor) variables.Variable • Dependent – metric• Independent – metric or
transformed non-metric (through dummy variable coding)Multiple Regression
Decision Process: Stage 1: Objectives of Multiple Regression researcher:
must consider three primary issues: 1) the appropriateness of the research
problem (Predictive purpose- maximize predictive accuracy by ensuring the
validity of the set of independent variables , model comparison by comparing two
or more independent variables to ascertain the predictive power of each variate vs
Explanation purpose – relative imp of independent variables by assessing
magnitude and direction of each independent variables, nature of relationship w/
dependent variables, nature of relationships among independent variables
(multicollinearity) ) 2) specification of a statistical relationship (Multiple
regression is appropriate when research is interested in statistical relation , not
functional relationship(expect no error in prediction, only calculate the exact
value). Some random component is always present (error in predicting variable)
in the relationship being examined; calculate the average value) 3) selection of the
dependent and independent variables (a. support theory either conceptual or
theoretical b. measurement error c. specification error). Measurement error:
problematic can be addressed through either of two approaches: Summated
scales- mitigate measurement error, or Structural equation modeling procedures-
can directly accommodate measurement error. Specification error: the exclusion
of relevant and inclusion of irrelevant independent variables. **when in doubt,
include potentially irrelevant variables (they can only confuse interpretation)
rather than omitting a relevant variable (which can bias all regression
estimates)Stage 2: Research Design of a Multiple Regression Analysis Issues
to consider 1)Sample Size 2) Creating additional variables : Nonmetric variable:
scan only be included in a regression analysis by creating dummy variables..
Dummy variables can only be interpreted in relation to their reference
category.Moderator Effect: When the moderator variable, a second independent
variable, changes the form of the relationship between another independent
variable and the dependent variable. The moderator term is a compound variable
formed by multiplying X1 by the moderator X2, >> (X1X2). The coefficient (b3)
of the interaction/moderator term indicates the unit change in the effect of X1 as
X2 changes. The coefficients (b1, b2) of the two independent variables now
represent the effects when the other independent variable is zero. Mediator
effect: when the effect of an independent variable may “work through” an
intervening variable (the mediating variable) to predict the dependent
variable.Stage 3: Assumptions in Multiple Regression Analysis:four primary
assumption- 1) Linearity of the phenomenon measured 2) Homoscedasticity:
Constant variance of the error terms 3) Normality of the error term distribution 4)
Independence of the error terms.STAGE 4: ESTIMATING THE
REGRESSION MODEL ANDASSESSING OVERALL MODEL
FIT:Variable Specification-Use variables in their original formAllows for use of
direct measures of the variables of interest. As number of variables increases,
interpretability may become problematic-Can be either software controlled or
user controlled.• Software controlled – software independently forms
dimensional reduction and then proceeds with analysis (e.g., principal
components regression).• User controlled – research performs some form of
dimensional reduction process (e.g., exploratory factor analysis) and forms
composites which are then substituted for original variables in the
analysis.Components of Model Fit:• Total Sum of Squares (SST)-total amount of
variation that exists to be explained by the independent variables. TSS = the sum
of SSE and SSR.• Sum of Squared Errors (SSE)- the variance in the dependent
variable not accounted for by the regression model = residual. The objective is to
obtain the smallest possible sum of squared errors as a measure of prediction
accuracy.• Sum of Squares Regression (SSR)- the amount of improvement in
explanation of the dependent variable attributable to the independent variables. F
statistic – statistical significance of overall model• Significance means that it is
unlikely your sample will produce a large R2 when the population R2 is actually
zero. • A rule of thumb is there must be <.05 probability for statistical
significance.R2 (Coefficient of Determination):R2 ranges from 0 to 1.0, with
large R2 indicating the linear relationship works well.Adjusted R2• based on the
number of independent variables relative to the sample size.Influential
observations include all observations that• lie outside the general patterns of the
data set.• disproportionate effect on the regression results.Three basic types
based upon the nature of their impact on the results:• Outliers are observations
that have large residual values and can be identified only with respect to a specific
regression model.• Leverage points are observations that are distinct from the
remaining observations based on their independent variable values. • Influential
observations are the broadest category, including all observations that have a
disproportionate effect on the regression results. Influential observations
potentially include outliers and leverage points but may include other
observations as well.Impacts: reinforcing, conflicting, shifting..Identifying
Influential Observations: Step 1: Examining Residuals(defined by a) cases
used to calculate and b) type of standardization) and Partial Regression
Plots(depict relationship of variable controlling for other variables)• .Step 2:
Identifying Leverage Points:Leverage points – substantially different on one or
more independent variable- Diagnostic measures – Hat matrix and Mahalanobis
distance (D2).Step 3: Single-case Diagnostics.Step 4: Selecting Influential
Observati:A. No issues – fit well and no extreme valuesB. High leverage, but not
outlier – very different on IVs, but still predicted well by modelC. Outliers, But
Acceptable Leverage – high residual, but no extreme values on IVD. Outliers and
High Leverage –poor prediction and quite different on IV.STAGE 5:
INTERPRETING THE REGRESSION VARIATE:key function:1prediction-
2. Explanantion: Interpretation with Regression Coefficients – primary measure of
the relative impact and importance of the independent variables in their
relationship with the dependent variable,Standardizing the Regression
Coefficients: Beta Coefficients – converts variables to a common scale and
variability, the most common being a mean of zero (0.0) and standard deviation of
one (1.0).Multicollinearity: relationship between two (collinearity) or more
(multicollinearity) independent variables. Multicollinearity occurs when any
single independent variable is highly correlated with a set of other independent
variables. Steps in Assessing and Addressing Multicollinearity1. Understand
new measures of correlation which incorporate multicollinearity.2. Assess the
degree of multicollinearity.3. Determine its impact on the results.4. Apply the
necessary remedies if needed. Three measure of correlation:Bivariate or zero-
order correlation,Semi-partial or part correlation,Partial correlation.identifying
Multicollinearity: Variance Inflation Factor (VIF) – measures how much the
variance of the regression coefficients is inflated by multicollinearity problems.
The square root of the VIF is the expected increase in the standard error of the
coefficients.Tolerance – the amount of variance of an independent variable that is
not explained by the other independent variables (i.e., an independent variable is
considered a dependent variable, predicted by all the other independent
variables).VIF and Tolerance are inversely related: VIF = 1 /
Tolerance.Effects of Multicollinearity: Impacts on Estimation- Decrease in
explained variance,Singularity,Reversal of signs of Coefficients, Increases in
standard error.Impacts on Explanation: Since coefficients only represent unique
explanation, multicollinearity can obscure the total effect of a variables, which
requires newer measures of relative importance. Bivariate correlations:Values of
.70 or higher may result in problems and lower values may be problematic if they
are higher than the correlations with the dependent variable.Tolerance or VIF•
Tolerance values up to .20, corresponding to a VIF of 5, almost always indicate
problems with multicollinearity.• VIF values of even 3 to 5 may result in
interpretation or estimation problems, particularly when the relationships with the
dependent variable are weaker.Remedies for Multicollinearity• Delete collinear
variable(s).• Apply dimensional reduction, such as composites from exploratory
factor analysis.• Specific estimation techniques – Bayesian or principal
components regression.• Do nothing – particularly if used solely for prediction,
but stillrisky.STAGE 6: VALIDATION OF THE RESULTS: • Additional or
Split Samples • Calculating the PRESS (Prediction Sum of Squares) statistic•
Comparing Regression Model• Forecasting.ILLUSTRATION OF A
REGRESSION ANALYSIS: Stage 1: ObjectivesStage 2: Research Design•
Thirteen independent variables (X6 to X18).• Meets minimum ratio of
observations per variable – 7:1 with adequate power.Stage 3: Assumptions•
Linearity – graphical analysis did not reveal nonlinear relationships.•
Homoscedasticity – only two variables (X6 and X17) had minimal violations.•
Normality – six variables indicated violations, thus requiring further analysis.ch 8
Logistic Regression: A specialized form of regression that is designed to predict
and explain a binary (two-group) categorical variablerather than a metric
dependent measure.CharacteristicsIts variate is similar to regular regression and
made up of metric independent variables. • It is less affected than discriminant
analysis when the basic assumptions, particularly normality of the independent
variables are not met.When Logistic Regression is Preferred: • Discriminant
analysis assumes multivariate normality and equal variancecovariance matrices
across groups, and these assumptions are often not met. Logistic regression does
not face these strict assumptions and is much more robust when these assumptions
are not met, making its application appropriate in many situations. • Even if the
assumptions are met, some researchers prefer logistic regression because it is
similar to multiple regression. It has straightforward statistical tests, similar
approaches to incorporating metric and nonmetric variables and nonlinear effects,
and a wide range of diagnostics.DECISION PROCESS FOR LOGISTIC
REGRESSION: Stage 1: Objectives of Logistic Regression: explanantion and
classification Stage 2: Research Design for Logistic Regression:Use of the
Logistic Curve:• Since the dependent variable can only take on two values (1 or
0), need a relationship form that is non-linear so that the predicted values will
never be below 0 or above 1.• Not possible with multiple regression, thus use
logistic curve.• Also violatesassumptions of multiple regression, requiring
different model form as well.sample size: should be 400 Stage 3: Assumptions of
Logistic Regression. Advantage:• The primary assumption is the independence of
observations, which if violated requires some form of hierarchical/nested model
approach.• An inherent assumption that should be addressed with the Box–
Tidwell test is the linearity of the independent variables, especially continuous
variables, with the outcome. Stage 4: Estimation of the Logistic Regression
Model and Assessing Overall Fit-:Uses the specific form of the logistic curve,
which is S-shaped, to stay within the range of 0 to 1.Estimating the Logistic
Regression Model: Two basic steps:• Transforming a probability into odds and
logit values.• Model estimation using a maximum likelihood approach, not least
squares as in regular multiple regression.The logistic transformation has two
basic steps:• Restating a probability as odds, then• Calculating the logit
values.Maximum Likelihood Estimation:• Maximizes the likelihood that an
event will occur – the event being a respondent is assigned to one group versus
another.• The basic measure of how well the maximum likelihood estimation
procedure fits is the likelihood value.Comparisons of the likelihood values
follow three steps:• Estimate a Null Model – which acts as the “baseline” for
making comparisons of improvement in model fit.• Estimate Proposed Model –
the model containing the independent variables to be included in the logistic
regression.Measures of Model Fit: Global Null Hypothesis Test:• Test for the
significance of any of the estimated coefficients.Pseudo R2 Measures:•
Interpreted in a manner similar to the coefficient of determination in multiple
regression.• Different pseudo R2 measures vary widely in terms of magnitude and
no one version has been deemed most preferred. • For all of the pseudo R2
measures, however, the values tend to be much lower than for multiple regression
models.Issues in Model Estimation: 1 Small sample size 2. complete separation
3. Quasi complete separation(zero cell effect).Predictive accuracy – ability to
classify observations into correct outcome group.Generates Four Outcomes:•
True Negatives• True Positives• False Negatives• False Positives.Predictive
Accuracy of Actual Outcomes• Sensitivity: true positive rate – percentage of
positive outcomes correctly predicted.• Specificity: true negative rate – percentage
of negative outcomes correctly predicted.Predictive Accuracy of Predicted
Outcomes• PPV (positive predictive value): percentage of positive predictions
that are correct.• NPV (negative predictive value): percentage of negative
predictions that are correct. Two Types of- Casewise Diagnostics Similar to
Multiple Regression• Residuals- Both residuals (Pearson and deviance) reflect
standardized differences between predicted probabilities and outcome value (0
and 1). Values above ± 2 merit further attention.• Influential Observations-
Influence measures reflect impact on model fit and estimated coefficients if an
observation is deleted from the analysis.- Comparable to those measures found in
multiple regression.Stage 5: Interpretation of the Results l:ogistic regression a
zero coefficient corresponds to odds of 1.0 –no change.Wald Statistic• Measure
of statistical significance for each estimated coefficient so that hypothesis testing
can occur just as it does in multiple regression.Interpreting the Coefficients: •
Original logistic coefficient -estimated parameter from the logistic model that
reflects the change in the logged odds value (logit) for a one unit change in the
independent variable.- It is similar to a regression weight or discriminant
coefficient. Exponentiated logistic coefficient• Antilog of the logistic
coefficient, which is used for interpretation purposes in logistic regression.• The
exponentiated coefficient minus 1.0 equals the percentage change in the
odds.Stage 6: Validation of the Results: Involves ensuring both the internal and
external validity of the results.Holdout versus Estimation Samples . • The most
common form of estimating external validity is creation of a holdout or validation
sample and calculating the hit ratio.Cross-validation .• Typically achieved with a
jackknife or “leave-one-out” process of calculating the hit ratio. Ch 4 Cluster
Analysis is a group of multivariate techniques whose primary purpose is to group
objects based on the characteristicsthey possess. Conceptual Development with
Cluster analysis• Data reduction – reduces population to smaller number of
homogeneous groups• Hypothesis Generation – means of developing or assessing
hypothesesNecessity of Conceptual Support• Strong conceptual support of
existence of clusters helps negate criticisms:• Cluster analysis is descriptive,
atheoretical, and non-inferential• Cluster analysis will always create clusters,
regardless of the actual existence of any structure• The cluster solution is not
generalizable because it is totally dependent upon cluster variate. Three Basic
Questions: How do we measure similarity? • We require a method of
simultaneously comparing observations on the clustering variables. Several
methods are possible, including the correlation between objects or perhaps a
measure of their proximity in two-dimensional space such that the distance
between observations indicates similarity.How do we form clusters? • No matter
how similarity is measured, the procedure must group those observations that are
most similar into a cluster, thereby determining the cluster group membership of
each observation for each set of clusters formed. How many groups do we
form? • The final task is to select one cluster solution (i.e., set of clusters) as the
final solution. In doing so, the researcher faces a trade-off: fewer clusters and less
homogeneity within clusters versus a larger number of clusters and more
withingroup homogeneity.two key elements:1. Subjectivity in selecting final
solution• While there are empirical diagnostic measures to assist the researcher in
selecting the final solution(s), there are no methods by which one solution is
deemed optimal.• Thus, it still falls to the researcher to make the final decision as
to the number of clusters• to accept as the final solution.2. Judgment required of
the researcher• Judgment of the researcher in selecting the characteristics to be
used, the methods of combining clusters, and even the interpretation of cluster
solutions makes any final solution unique to that researcher.DECISION
PROCESS:Stage 1: Objectives of Cluster Analysis:Primary Goal• to partition
a set of objects into two or more groups based on the similarity of the objects for a
set of specified characteristics (the cluster variate).Two key issues:• The research
questions being addressed, and• The variables used to characterize objects in the
clustering process.Research Question: How to form the taxonomy,How to
simplify the data,Which relationships can be identified.Selection of Clustering
Variables: Two Issues in Variable Selection .1. Conceptual considerations -
Variables characterize the objects being clustered- Relate specifically to the
objectives of the cluster analysis.2. Practical considerations.-Should always use
the “best” variables available (i.e., little measurement error, etc.).Cluster analysis
is used for:• Taxonomy description – identifying natural groups within the
data.• Data simplification – the ability to analyze groups of similar observations
instead of all individual observations.• Relationship identification – the
simplified structure from cluster analysis portrays relationships not revealed
otherwise.• Theoretical, conceptual and practical considerations must be observed
when selecting clustering variables for cluster analysis:• Only variables that relate
specifically to objectives of the cluster analysis are included, since “irrelevant”
variables can not be excluded from the analysis once it begins.• Variables are
selected which characterize the individuals (objects) being clustered.Stage 2:
Research Design in Cluster Analysis: Types of Variables Included: •Can
employ either metric or non-metric, but generally not in mixed fashion.• Multiple
measures of similarity for each type.Number of Clustering Variables• Can
suffer from “curse of dimensionality” when large number of variables analyzed.•
Can have impact with as few as 20 variables.Relevancy of Clustering Variables•
No internal method of ascertaining the relevancy of clustering variables.•
Researcher should always include only those variables with strongest conceptual
support. Detecting Outliers: They should be removed if the outlier represents:
Aberrant observations not representative of the population. Observations of small
or insignificant segments within the population. They should be retained if the
outlier represents: an under-sampling/poor representation of relevant groups in
the population. In this case, the sample should be augmented to ensure
representation of these groups.Outliers can be identified: • Finding observations
with large distances from all other observations – pairwise similarities or
summated measure of squared differences from mean of each clustering variable.•
Graphic profile diagrams or parallel coordinate graphs highlighting outlying
cases.• Their appearance in cluster solutions as single-member or very small
clusters.Interobject similarity• an empirical measure of correspondence, or
resemblance, between objects to be clustered. • calculated across the entire set of
clustering variables to allow for the grouping of observations and their
comparison to each other.Three methods most widely used in applications of
cluster analysis: • Distance measures – most often used.• Correlational measures
– less often used as they measure patterns, not distance.• Association measures –
applicable for non-metric clustering variables.Types of distance measure: Many
different distance measures, most common are:- Euclidean (straight line) distance
is the most common measure of distance.- Squared Euclidean distance is the sum
of squared distances and is the recommended measure for the centroid and Ward’s
methods of clustering.-Mahalanobis distance (D2) accounts for variable
intercorrelations and weight each variable equally. When variables are highly
intercorrelated, Mahalanobisdistance is most appropriate.Data Standardization:
should be standardized whenever possible to avoid problems resulting from the
use of different scale values among clustering variables.Two approaches to
standardization• Relative to other cases:-most common standardization is Z
scores.• Relative to other responses within an object-If groups are to be identified
according to an individual’s response style, then within-case or row-centering
standardization is appropriate.Stage 3: Assumptions in Cluster Analysis :
Structure Exists• Since cluster analysis will always generate a solution,
researcher must assume that a “natural” structure of objects exists which is to be
identified by the technique.Representativeness of the Sample• Must be
confident that the obtained sample is truly representative of the
population.Impact of multicollinearity• Multicollinearity among subsets of
variables is an implicit “weighting” of the clustering variables• Potential remedies
for multicollinear subsets of variables- Reduce the variables to equal numbers in
each set of correlated measures.- Use a distance measure that compensates for the
correlation, like Mahalanobis Distance.- Take a proactive approach and include
only cluster variables that are not highly correlated.Stage 4: Deriving Clusters
and Assessing Overall Fit: Hierarchical• Most common approach is where all
objects start as separate clusters and then are joined sequentially such that each
step forms a new cluster joining by two clusters at a time until only a single
cluster remains. Two types:Agglomerative Methods • Buildup: all observations
start as individual clusters, join together sequentially.A multi-step process• Start
with all observations as their own cluster.• Using the selected similarity measure
and agglomerative algorithm, combine the two most similar observations into a
new cluster, now containing two observations.• Repeat the clustering procedure
using the similarity measure/agglomerative algorithm to combine the two most
similar observations or clusters (i.e., combinations of observations) into another
new cluster.• Continue the process until all observations are in a single
cluster.Most widely used algorithms• Single Linkage (nearest neighbor) – shortest
distance from any object in one cluster to any object in the other.• Complete
Linkage (farthest neighbor) – based on maximum distance between observations
in each cluster.• Average Linkage – based on the average similarity of all
individuals in a cluster. • Centroid Method – measures distance between cluster
centroids.• Ward’s Method – based on the total sum of squares within
clusters..Divisive Methods • Breakdown: initially all observations in a single
cluster, then divided into smaller clusters. Non-hierarchical• the number of
clusters is specified by the analyst and then the set of objects are formed into that
set of groupings.1. Determine number of clusters to be extracted2. Specify
cluster seeds.3. Assign each observation to one of the seeds based on
similarity.• Sequential Threshold = selects one seed point, develops cluster; then
selects next seed point and develops cluster, and so on. Observation cannot be
reassigned to another cluster following its original assignment.• Parallel
Threshold = sets all seed points simultaneously, then develops clusters.•
Optimization = allow for re-assignment of observations based on the sequential
proximity of observations to clusters formed during the clustering process.Pros
and Cons of Nonhierarchical Methods: Pros• Results are less susceptible to:•
outliers in the data,• the distance measure used, and• the inclusion of irrelevant or
inappropriate variables.• Can easily analyze very large data sets.Cons• Best
results require knowledge of seed points.• Difficult to guarantee optimal solution.•
Generates typically only spherical and more equally sized clusters.• Less efficient
in examining wide number of cluster solutions.Pros and Cons of Hierarchical
Methods: Pros• Simplicity – generates tree-like structure which is simplistic
portrayal of process.• Measures of similarity – multiple measures to address many
situations.• Speed – generate entire set of cluster solutions in single
analysis.Cons• Permanent combinations – once joined, clusters are never
separated.• Impact of outliers – outliers may appear as single object or very small
clusters.• Large samples – not amenable to very large samples, may require
samples of large populations. Hierarchical clustering solutions are preferred
when:• A wide range, even all, alternative clustering solutions is to be examined.•
The sample size is moderate (under 300-400, not exceeding 1,000) or a sample of
the larger dataset is acceptable.Nonhierarchical clustering methods are
preferred when:• The number of clusters is known and initial seed points can be
specified according to some practical, objective or theoretical basis.• There is
concern about outliers since nonhierarchical methods generally are less
susceptible to outliers.Combination approach – using a hierarchical approach
followed by a nonhierarchical approach is often advisable.1. A nonhierarchical
approach is used to select the number of clusters and profile cluster centers that
serve as initial cluster seeds in the nonhierarchical procedure.2. A nonhierarchical
method then clusters all observations using the seed points to provide more
accurate cluster memberships.Should The Cluster Analysis Be
Respecified:Primary focus• Identification of single object or very small clusters
that represent disparate observations that do not match the research objectives.•
Similar considerations to outlier identification and many times operate on the
same conditions in the sample..If respecification occurs• Should reanalyze
remaining data, especially when using hierarchical procedures.Determining the
Number of Clusters: Stopping rules• Criteria used with hierarchical techniques
to identify potential cluster solutions.• Foundational principle – a natural increase
in heterogeneity comes from the reduction in number of clusters.• Common to all
stopping rules:-• evaluating the trend in heterogeneity across cluster solutions to
identify marked increases. • substantive increases in this trend indicate relatively
distinct clusters were joined and that the cluster structure before joining is a
potential candidate for the final solution.• Issues in applying stopping rules- The
ad hoc procedures must be computed by the researcher and often involve fairly
complex approaches.• Many times measures are software-specific.Two Classes of
Stopping Rules:Class 1: Measures of Heterogeneity Change• measures
heterogeneity change between cluster solutions at each successive decrease in the
number of clusters. A cluster solution is a candidate for the final cluster solution
when the heterogeneity change measure makes a sudden jump.• Measures of
heterogeneity change -• Percentage Changes in Heterogeneity – simple
percentage change in heterogeneity.• Measures of variance change – use of root
mean square standard deviation (RMSSTD) to compare solutions.• Statistical
measures of heterogeneity change – pseudo T2 statistic compares goodness-of-fit
between k and k-1 clusters. Thus, large pseudo T2 value at six clusters indicates a
seven cluster solution is the possible final solution.Class 2: Direct Measures of
Heterogeneity• directly measure heterogeneity of each cluster solution and then
allow analyst to evaluate each cluster solution against a criterion measure.•
Measures of heterogeneity:- Comparative cluster heterogeneity – the cubic
clustering criterion (CCC) is a SAS measure of the deviation of the clusters from
an expected multivariate normal distribution. Choose cluster solution(s) with high
values of CCC.- Statistical significance of cluster variation – pseudo F statistic
measures the separation among all the clusters by the ratio of between-cluster
variance (separation of clusters) to withincluster variance (homogeneity of
clusters). Higher values indicate a possible cluster solution.- Internal validation
index – characterize a cluster solution on two dimensions: separation and
compactness. Common measure is the Dunn index ratio, the ratio between the
minimal within-cluster distance to maximal between-cluster distance. Higher
values indicate better solutions.Rules of Thumb – Deriving The Final Cluster
Solution: • Measures of heterogeneity change: • These measures, whether they
be percentage changes in heterogeneity, measures of variance change (RMSSTD)
or statistical measure of change (pseudo T2), all evaluate the change in
heterogeneity when moving from k to k - 1 clusters.• Candidates for a final cluster
solution are those cluster solutions which preceded a large increase in
heterogeneity by joining two clusters (i.e., a large change in heterogeneity going
from k to k –1 clusters would indicate that the k cluster solution is better).• Direct
measures of heterogeneity:• These measures directly reflect the compactness
and separation of a specific cluster solution. These measures are compared across
a range of cluster solutions, with the cluster solution(s) exhibiting more
compactness and separation being preferred.• Among the most prevalent
measures are the CCC (cubic clustering criterion), a statistical measure of cluster
variation (pseudo F statistic) or the internal validation index (Dunn's
index).Additional Approaches to Clustering: Density-based approach•
Fundamental principle – clusters can be identified by “dense” clusters of objects
within the sample, separated by regions of lower object density.• Researcher
must decide• ε, the radius around a point that defines a point’s neighborhood,
and• the minimum number of objects (minObj) necessary within a neighborhood
to define it a cluster• Has advantages of• Ability to identify clusters of any
arbitrary shape• Ability to process very large samples,• Requires specification of
only two parameters,• No prior knowledge of number of clusters,• Explicit
designation of outliers as separate from objects assigned to clusters,• Applicable
to a “mixed” set of clustering variables (i.e., both metric and nonmetric). Model-
Based Approach• varies from other approaches in that it is a statistical model
versus algorithmic.• uses differing probability distributions of objects as the basis
for forming groups rather than groupings of similarity in distance or high density.•
basic model – mixture model where objects are assumed to be represented by a
mixture of probability distributions (known as components), each representing a
different cluster.• Advantages• Can be applied to any combination of clustering
variables (metric and/or nonmetric).• Statistical tests are available to compare
different models and determine best model fit to define best cluster solution.•
Missing data can be directly handled.• No scaling issues or transformations of
variables needed.• Once the cluster solution is finalized, can include
antecedent/predictor and outcome/validation variables.Stage 5: Interpretation of
the Clusters:intrepretation:• Involves examining each cluster in terms of the
cluster variate to name or assign a label accurately describing the nature of the
clusters. • The cluster centroid, a mean profile of the cluster on each clustering
variable, is particularly useful in the interpretation stage.- Interpretation involves
examining the distinguishing characteristics of each cluster’s profile and
identifying substantial differences between clusters-Cluster solutions failing to
show substantial variation indicate other cluster solutions should be examined.•
The cluster centroid should also be assessed for correspondence with the
researcher’s prior expectations based on theory or practical experience.Stage 6:
Validation and Profiling of the Clusters: Validation is essential in cluster
analysis since the clusters are descriptive of structure and require additional
support for their relevance.Two approaches• Cross-validation – empirically
validates a cluster solution by creating two subsamples (randomly splitting the
sample) and then comparing the two cluster solutions for consistency with respect
to number of clusters and the cluster profiles.• Criterion validity – achieved by
examining differences on variables not included in the cluster analysis but for
which there is a theoretical and relevant reason to expect variation across the
clusters.Profiling A Cluster Solution: Describing the characteristics of each
cluster on a set of additional variables (not the clustering variables) to further
understand the differences between clusters• Examples include descriptive
variables (e.g., demographics) as well as other outcome-related measures.•
Provides insight to researchers as to nature and character of the clusters.Clusters
should differ on these relevant dimensions. This typically involves the use of
discriminant analysis or ANOVA.Rules of Thumb – Deriving the Final Cluster
SolutionThere is no single objective procedure to determine the ‘correct’ number
of clusters. Rather the researcher must evaluate alternative cluster solutions on the
following considerations to select the “best” solution:• Single-member or
extremely small clusters are generally not acceptable and should generally be
eliminated.• For hierarchical methods, several ad hoc stopping rules are available
to indicate the number of clusters based on the rate of change in a total similarity
measure as the number of clusters increases or decreases or measures of
heterogeneity.• All clusters should be significantly different across the set of
clustering variables.• Cluster solutions ultimately must have theoretical validity
assessed through external validation.Implications for Big Data Analytics:
Primary advantage• Simplification by reducing the large number of observations
into a much smaller number of groupings from which the general nature and
character of the entire dataset can be observed.Challenges• Increasing sample
sizes pose difficulties for clustering methods, particularly hierarchical methods.•
Clustering high-dimensional data creates difficulties in:- establishing object
similarity.- ensuring variable relevancy.Stage 1: objectives of cluster analysis:
Primary Objective• develop a taxonomy that segments objects (HBAT customers)
into groups with similar perceptions. • Once identified, strategies with different
appeals can be formulated for the separate groups—the requisite basis for market
segmentation.Stages 2( detecting outliers, defining similarity, sample size,
standardization) and 3: Research Design and Assumtpions in Cluster
Analysis(sample representation, multicollinerarity). Stages 4- Hierarchical
and Nonhierarchical Methods. Stage 5:Profiling the Hierarchical Cluster
Results/ nonhierrarchial method. Stage 6: Validation and Profiling the
Clusters. Ch6:ReEmergence of Experimentation:Experimentation long been
the foundational principle of the scientificmethod of research.Yet while social
sciences have been less frequent users of experimentationin the past, that
trend is being reversed.• Traditional randomized laboratory experiments
becoming increasingly common inconsumer behavior and behavioral economics.•
Field experiments have become increasingly common both in academic
settingand the business sector.What Distinguishes The Experimental
Approach?Primary objective: Focus on a single or small number of “causes” to
establisha very specific “effect.”Basic elements of an experiment• Treatment à
Outcome.• Treatment/factor – a conceptual “cause” of a specific outcome (i.e.,
cause à effect).• Experimental design – research approach based on developing a
narrowly focusedresearch program which can be tested in a hypothesis or set of
hypotheses.• Distinguished from the techniques/methods typically used in
analyzing this type ofdata (e.g., ANOVA and MANOVA) by its rigid research
controls to isolate theproposed “cause” as the only factor which has a relationship
with the outcome.MANOVA Defined:Multivariate extension of ANOVA for
assessing the differences betweengroup means on more than one dependent
variable at the same time.Basic elements• A variate is tested for equality.-
Actually two variates are compared – one for the dependent variables and another
for theindependent variables.- The dependent variable variate is of more interest
because the metric-dependent measurescan be combined in a linear combination,
as we have already seen in multiple regression anddiscriminant analysis.- The
unique aspect of MANOVA is that the variate optimally combines the multiple
dependentmeasures into a single value that maximizes the differences across
groups.• Cells/groups- Formed by combinations of independent variables
(factorial designs) (e.g., 2 x 2 = four cells).The relationship between the
univariate and multivariate procedures is shown below:
Differences Between MANOVA and Discriminant Analysis: To some extent, they
are “mirror images” of each other:• The dependent variables in MANOVA (a set
of metric variables) are the independent variables in discriminant analysis.• The
single nonmetric dependent variable of discriminant analysis becomes the
independent variable in MANOVA.• Both use the same methods in forming the
variates and assessing the statistical significance between groups.Differences are
in the Research Objectives• In discriminant analysis, the groups formed by the
nonmetric dependent measure are assumed as given and interest is in the set of
metric independent variables for their ability to discriminate among the groups.
• In MANOVA, the set of metric variables used as dependent measures are
assumed given and interest is in finding the nonmetric variable(s) that form
groups with the greatest differences on the set of dependent
measures.DECISION PROCESS FOR MANOVA:Stage 1: Objectives of MANOVA-
Objectives• Analyze a dependence relationship represented as the differences in
a set of dependent measure across groups formed by one or more nonmetric
independent measures.• To provide insights into the nature and predictive
power of the independent measures as well as the interrelationships and
differences in the multiple dependent measures.Benefits• Controlling the
experiment-wide error rate when there is some degree of intercorrelation
among dependent variables.• More statistical power than ANOVA when 5 or less
dependent variables.• May detect combined effects among dimensions (i.e.,
variates) of the dependent measures.Types of Multivariate Questions Suitable
for MANOVA: Multiple Univariate Questions• Control for overall experiment-
wide error rate with overall test followed up by univariate tests .Structured
Multivariate Questions• Specific relationships between dependent measures
requires joint testing.Intrinsically Multivariate Questions• Principal concern is
how the whole set of independent variables differ across groups, with combined
effect more of interest than separate effects.Selecting the Dependent
Measures: Researchers should include only dependent variables that have
strong theoretical support• Inappropriate or irrelevant variables can not be
empirically removed (e.g., no sequential inclusion method) and thus can impact
the overall tests.Levels of correlation among dependent measures• If inter-
correlations are very high (> .7 or .8), they have a tendency to create
redundancies and reduce the statistical efficiency. This may become especially
impactful if multiple composite variates (i.e., discriminant functions) are formed
when analyzing two or more independent variables. • If correlations are very low
(< .3), the analyst should consider running separate ANOVAs and adjusting for
the experiment-wide error (e.g., Bonferroni adjustment) Stage 2: Issues in the
Research Design of MANOVA: MANOVA is Applicable to Multiple Research
ApproachesExperimentation: Randomization of Groups• Controlled experiment
– respondents are randomly assigned to treatment or control group to ensure
that causal inferences can be made.• Field experiment – implementation of a
controlled experiment in a “natural” setting (i.e., outside a controlled laboratory
environment) in an attempt to increase external validity, but also raising threats
to internal validity.Experimentation: Non-Randomization • Quasi-experiment –
similar to a controlled experiment except it lacks assignment to the two groups
through randomization.• Natural experiment – non-random form of
experimental research where the treatment occurs naturally (e.g., natural
disaster, etc.).Non-experimental• Observational study (cross-sectional study) –
non-random research design limited since respondent selection is not controlled
by the researcher and confounds are very difficult to identify and then account
for in the analysis.Types of Variables in Experimental Research: Basic
Relationship: primary interest of the research• Treatment/factor – independent
variable which is hypothesized as the reason/”cause” of the outcome variable.•
Outcome – dependent variable which represents the values arising from the
different levels of the treatments/factors.• Main effect – individual effect of a
treatment variable on the dependent variable(s).Stage 3: Assumptions of
ANOVA and MANOVA. Stage 4: Estimation of the MANOVA Model and
Assessing Overall Fit Stage 5: Interpretation of the MANOVA Results Stage 6:
ValidationoftheResult

Q1 :When should I delete the "star"
outliers and when should I delete the "circle" outliers? You should always
delete the outliers represented as "stars". However, for the "circles", sometimes
deleting all of them will result in the lost of too many observations. You can
decide by yourself whether to delete or not. A good option is to only delete the
observations that repeatedly marked as circle in many items and keep the ones
that are marked as circle for only 1 item. Q2Why use multivariate analysis?
multivariate analysis is a tool to find patterns and relationships between several
variables simultaneously. It lets us predict the effect a change in one variable will
have on other variables. ... This gives multivariate analysis a decisive advantage
over other forms of analysis. Q3.Why is knowledge of measurement scales
important in using multivariate analysis? Since the type of measurement should
be defined by the researcher (even it will be nonmetric/qualitative or
metric/quantitative) for each variable, in MV analysis defining data has great
effect what the data present. For computers, the values are just numbers; in
nominal scales, 1 can be assigned to male, and 0 to female. But the data present
gender and researcher determine how it can be analyzed. Since nonmetric data
(ordinal and nominal scales) used as independent variable in most MV
techniques. Knowledge of measurement scales important in order not to use
wrong scale for the variables. As exemplified above, 0 and 1 are just nonmetric
data to present gender; not having any numerical meaning like female are less
than male, or we cannot calculate any mean value Q4.What is the difference
between component analysis and common factor analysis? In factor analysis,
the original variables are defined as linear combinations of the factors the goal in
factor analysis is to explain the covariance’s or correlations between the
variables. Use principal components analysis to reduce the data into a smaller
number of components. Q5.When should multiple regression be used? It is
used when we want to predict the value of a variable based on the value of two
or more other variables. The variable we want to predict is called the dependent
variable (or sometimes, the outcome, target or criterion variable). Q6.When
should a linear regression be used? Simple linear regression is appropriate when
the following conditions are satisfied. The dependent variable Y has a linear
relationship to the independent variable X. To check this, make sure that the XY
scatterplot is linear and that the residual plot shows a random pattern. 8.How do
you use regression coefficients? when the regression line is linear (y = ax + b) the
regression coefficient is the constant (a) that represents the rate of change of
one variable (y) as a function of changes in the other (x); it is the slope of the
regression line. Q7.Should we use factors scores or summated ratings in follow
up analysis? It’s hard to answer without context. Usually summated rating is use
when the items are designed as summated scale but it is very rare. Usually for
latent constructs, in my field, they use factor score. However, in some fields, like
economics, sometimes I see summated rating. I don’t really know why they use
it. Maybe it is a way to put weight on the items that they think are important.
Summated rating is when the score of all items add up into a score of the
construct right? So if the items are measure on different scales. For example
anitem is 1 to 5, another is 1 to 9. When they add up, the item with 1 to 9 scale
will havehigherweight
Logistics regression9 class notes)- Log is used to choose the exponential value -
x^0=1. Sample size-Likelihood is not same as the probability.More than 400 is
better to achieve good results.Generates four outcomes True positives and
true negatives - are true results whether you are true or false.False positives
and False negative - not sure results whether it’s true or false.This kind of
problems can be reduced by increasing more sample size.Predictive accuracy
of actual outcomes -If one only sensitivity is good specificity will be low Use of
aggregated data-If we have multiple dependent variables, each variable is two
levels(2X2X2) (2X2X2) because one binary is (2X2X2). Assumption of logistics
regression-When we do multiple regression, we don’t our sample to be
contaminated. We need our data to be independent. We can run hierarchical
test in order to make it independent. How to keep data uncontaminated?-
More research .Fitting the logistics curve to sample data -The curve going more
to the 1 is success and more to the 0 is failure-odd and log odds is not same as
probability.-Those data we want divided by overall is probability. Maximum
likelihood model-Better the model we estimate, better the maximum
likelihood model.-We don’t want too many data to fall in between of the curve
data which will lead to poorly fitted relationship curve.If we don’t know what
hypothesis to test go through two steps - Null hypothesis should be rejected-
which will lead to support in our alternative hypothesis. -2LL shows if model is
good or not. Measure of Model Fit-we use pseudo R2 in logistics regression,
not the normal R2.Estimate the issue- we have three problematic issues. -
cannot decide when sample size is low-0 and 1 is not defined in log odd. - third
problem is most frequently encountered (QUASI-COMPLETE SEPARATION).N=1
which means no control group. Influential obersativation -Delete the influential
observation. Two form of coefficient-It’s okay to use original logistics
coefficients if we don’t know how to use exponential logistics coefficient.
 EFA- interdependency
 Multiple regression dependent variable
 Used when there’s a theory
 Overlapping is good but not much (multi COLLINEARITY which is not
good that cause all result could be reverse, signs change x and y from
positive to negative). Theory tells whether there is overlapping or not.
That’s why we should test if there multi collinearity issues.
 Univariate- one variable (in multi regression its dependent)
 Simple regression is independent.
 Text mining-gaining knowledge from existing data. (internet etc)
 Nature of relationships (usually we do the linear)
 How to select Dependent and Independent Variables
 Focus on all the dependent on how it shows relation with the
independent variables
 we care the most for measurement error in dependent variables
 It is rare to see measurement error in the multiple regression
 Specification theory is easy to error as long as you have good theory
(right perspective to choose the theory).
 You can add the potentially irrelevant variables intentionally in order
to know if it will cause any changes.
 When you use same measurement in method is Common method
biases.
  Generalize the research to the real world
 Detail interesting factor can be found using MANOVA and ANOVA
rather than multiple regression.
 Homoscedasticity- put relevant data together.
 Test the assumption before you run the data.
 Use standardized for the data that study Across
  Log can be used if there’s too much Homoscedasticity.
 Linear regression is preferable.
 Forward, backward, stepwise are the variables we will be using the
most
 Deal with multicollinearity problem first before using original form
 In multiple regression we use Confirmatory (Simultaneous)
 Constrained situation is rare to use because it is only use when we
know what to expect.
 TSS = SSE + SSR.
 SSE TSS


 SSR
 When you run multiple regression there will be two results for
multiple regression and MANOVA
  When you run MANOVA there will be also two results for multiple
regression and MANOVA
 Just ignore the additional one
 For the Experiment , use the MANOVA
  For the Secondary data , use the multiple regression
 Test Anova result before jumping into multiple regression.
 Significant should be greater than 0.5 even though MANOVA result is
okay.
 R2 is bigger the better
 Adjusted R2 is lower than R2
 IF T and P value is significant but the value is too small means Practical
significance is not good.
 C, f and d can be problematic
 Explanation is done theory based but if there no theory at least
prediction.
 Multicollinearity occurs when Adopt new measurement.
 Nowadays VIF is up to 0.30
Relative importance of independent variables
 X1Family income,x2profession,x3bank,x4 social class- these kind of
variables are comparable.
 In finance, we use secondary data.
 First check we check multicollinerary , relative importance of
comparable variables.
 Sum-mated constraints (several variables combine to represent one
constraints) are better than single constraints.
 In case wanna Add Additional measures but we need to collect data
and do research.
 In multiverse no need to distinguish shared and unique variance.
 How can we know when the value come up with the valid result
 - additional or split samples :If we have multiple data sets we can test it
and results will be similar.
 -when we calculate hypothesis best word is to use confirm or support
not prove.
 - comparing regression models: if the difference id huge data is not
good. - No significant result.
 -when we choose variable which updated when time goes by. So that
people can use your data in near future.
 Higher the R2 more valid
 If VIf is above 0.5 there is multicollinaery but R2 is okay (DONT
TRUST THE R2)