Chapter1: Overview of Multivariate Method data (Hot/Internal or cold deck/external imputation: substitute a value from another of factors that
on: substitute a value from another of factors that exist within a set of variable and belong to which construct. -CFA is
Multivariate Analysis: all statistical methods simultaneously analyze multiple
measurements on each individual or object under investigation; any simultaneous
analysis of > 2 variables Variate: a linear combination of variables with empirically
determined weights of a sef of variables specified by researchers. Weights are
determined by multivariate technique to best achieve specific objective. Multivariate
technique: extension of univariate analysis (analysis of single distribution) &
bivariate analysis (cross-classification, correlation, analysis of variance, and simple
regression used to analyze two variables) Variate value (Y’) = w1X1 + w2X2 + w3X3
+ . . . + wnXn (where Xn is the observed variable and wn is the weight determined by
the multivariate technique) Types of data: 1) Nonmetric data (Qualitative)
describes differences in type by indicating the presence or absence of a characteristic
or property; only represent categories or classes 1.1) Nominal (categorical) scale
only label represent categories or classes and don’t imply amounts of an attribute or
characteristic e.g. sex, religions 1.2 ) Ordinal scale can be compared, only the order
of the values ; only the order will be known but not the amount of difference btw
values e.g. ranking, yes-no-greater 2) Metric data (Quantitative) used when subjects
differ in amount or degree on a particular attribute 2.1) Interval Scales Interval scales
and ratio scales provide the highest level of measurement precision, permitting
nearly any mathematical operation to be performed. These two scales have constant
units of measurement, so differences between any two adjacent points on any part
of the scale are equal. This uses an arbitrary zero point e.g. time, temperature, no
true zero, not always relatable 2.2) Ratio scales represent the highest form of
measurement precision because they possess the advantages of all lower scales plus
an absolute zero point. All mathematical operations are permissible with ratio-scale
measurements, relatable e.g. height, weight (0.0) Measurement error: degree to
which the observed values are not representative of the true values; caused by data
entry errors, imprecision of the measurement, inability of respondents to accurately
provide info. Validity: the degree to which a measure accurately represents what it is
supposed to. E.g. if we want to measure discretionary income, we should not ask
about total household income Reliability: the degree to which the observed variable
measures the true value and is error free thus, it is the opposite of measurement
error. If the same measure is asked repeatedly, e.g., more reliable measures will
show greater consistency.
Summated scale: method of combining several variables that measure the same
concept into a single variable in an attempt to increase the reliability of the
measurement through multivariate measurement. The separate variables are
summed and then their total or average score is used in the analysis for which
several variables are joined in a composite measure to represent a concept.
Objective is to avoid the use of only a single variable to represent a concept and
instead use several variables representing differing facets of the concept to obtain a
more well-rounded perspective. thus enables the researcher to more precisely
specify the desired responses >> place total reliance on the “average to a set of
related responses.
Variable Specification: specifying the variable whether 1) use original/indi variables
- retain most detailed attribute, but may suffer from multicollinearity or 2) perform
dimensional reduction - finding combinations of the indi variables tht captures
multicollinearity among set of variables and allow for a single composite value
representing the set of variables .
Variable Selection: identifying the variables to be included in the analysis .
Multicollinearity: the degree of correlation among the variables in the variate may
result in a confounding effect (a situation in which a relationship btw exposure and
outcome is distorted by the presence of another variable) in the interpretation of the
indi variables of the variate; the measure of the shared variance with other variables
in the variate
Types of Statistical Error: 1)Type I error/ Alpha (α): probability of rejecting the null
hypo when it is actually true; false positive. By specifying an alpha level,researcher
sets acceptable limits for error and indicates probability of concluding that
significance exists when it really doesn’t 2) Type II error/Beta (β): the probability of
not rejecting the null hypo when it is actually false; accept false Power or 1 – β:
probability of correctly rejecting the null hypo when it should be rejected Type I and
Type II errors are inversely related Type I error becomes more restrictive (moves
closer to zero) as probability of a Type II error increases. Reducing Type I errors
reduces the power of the statistical test.
Statistical Power (1 – β): is determined by 1) Effect size: actual magnitude of effect
of interest; helps determine whether the observed relationship (difference or
correlation) is meaningful. e.g. If a weight loss firm claims its program leads to an
average weight loss of 25 pounds, the 25 pounds = effect size 2) Alpha (α): as alpha
becomes more restrictive (smaller), power decreases . Typical
α= moderate
** 0.05. 3) Sample
effect size:
size power
at any reaches
given alpha
acceptable
level, increased
levels at sample sizes ,ofpower
100 or
increases.
more for alpha levels of both .05 and .01
** when small effect size, statistical tests have little power
E.g. if the effect size is small a sample of 200 with an alpha of .05 still has only a 50
percent chance of significant differences being found. ** if the researcher expects that
the effect sizes will be small the study must have much larger sample sizes and/or
less restrictive alpha levels (e.g., .10).
ROT::STATISTIC power analysis: 1) Researcher should design study to achieve a power
level of 0.80 at the desired significance level. 2) More stringent significance levels (eg
0.01 instead of 0.05) require a larger samples to achieve the desire power level 3)
power can be increased by choosing a less stringent α (alpha) level ( eg 0.10 instead of
0.05) 4) Smaller effect sizes require large sample sizes to achieve the desired power 5)
An increase in power is most likely achieved by increasing the sample size
Multivariate Technique: 1) Dependence technique: a variable or set of variables is
identified as the dependent variable to be predicted/explained by other independent
variables. Can be categorized by two characteristics: 1.1) number of dependent
variables (single, several dependent var) 1.2) type of measurement scale employed
by the variables (metric, nonmetric) /level of attribute & respondent Conjoint
/metric (DV & IVs)  Multiple regression // nonmetric DV& metric IVs  multiple
discriminant analysis & linear probability models // metric DVs & nonmetric IVs
MANOVA // non metric DV&metric IVs  Logistic regression // nonmetric
(DVs&IVs)  dummy variable  canonical analysis both// if a set of dependent or
independent variable relationships is postulated  structural equation modeling 2)
Interdependence technique: analysis of all variables in the set w/out distinction btw
dependent and independent variables. analyzes the structure of the
interrelationships among a large number of variables to determine a set of common
underlying dimensions (factors).  EFA&CFA // each object is similar to the other
objects in the cluster  cluster analysis // identifies “unrecognized” dimensions 
perceptual mapping(Multidimensional scaling) // correspondence analysis
Guidelines for Multivariate Analysis: 1) Practical Significance: means of assessing
multivariate analysis results based on their substantive findings whether the result is
useful. 2) Statistical significance: determines whether the result is attributable to
chance in in achieving the research objectives 3) Sample Size – large or small affect
result 3) Know Your Data – outlier, assumption violation, missing data can create
substantial effects 4) Strive for Model Parsimony- simple model w/ greater
explanatory predictive power 5)Look at error 6)simplify model 7)Validate results
Chapter2: Examining Your Data
Graphical Examination: 1) Univariate profiling: examine the shape of
distribution(histogram) - the frequencies are plotted to examine the shape of the
distribution of values 2) Bivariate profiling: examining relationship between
variables, use scatterplot; examining group differences, use box-whisker plot 3)
Multivariate profiles: use multivariate graphical display - Shape: Histogram, Bar
Chart, Box & Whisker plot, Stem and Leaf plot - Relationships: Scatterplot, Outliers
Missing Data: Info not available for subject/case about whom other info is available.
Occurs when respondents fail to one or more answer in a survey.Impact: 1) reduce
sample size 2) distorts results. 4 Step to Identify Missing Data and Applying
Remedies Step 1: Determine type of missing data 1) ignorable missing data - take
data from sample population rather than whole population, data collection
instrument(skip pattern), censored data(data not yet observed) 2) or not ignorable
missing data: known process (procedural factor-data entry/mgt), unknown process
(the respondent refuse to respond a certain question) Step 2 : Determine the extent
of missing data determine whether the extent/amount of missing data is low enough
to not affect the result, even if it operates in a nonrandom manner by tabulating 1) %
of variables with missing data 2) no. of cases w/ missing data. if the extent missing
data is high → step 3, if it’s low → step 4
ROT2-1::How Much Missing Data Is Too Much?: 1) Missing data under 10% for an
individual case or observation can generally be ignored (acceptable), except when the
missing data occurs in a specific nonrandom fashion (e.g. concentration in a specific
set of questions, attrition at the end of the questionnaire) 2) no.of cases with no
missing data must be sufficient for the selected analysis technique if replacement
values will not be substituted (imputed) for the missing data.
Step 3: Diagnose the randomness of the missing data processes 1) Missing at
random (MAR): Classification of missing data applicable when missing values of Y
depend on X, but not on Y. When missing data are MAR, observed data for Y are a
truly random sample for the X values in the sample, but not a random sample of all Y
values due to missing values of X. 2) Missing completely at random (MCAR): when
missing values of Y are not dependent on X. When missing data are MCAR, observed
values of Y are a truly random sample of all Y values, with no underlying process that
lends bias to the observed data.
ROT2-2::Deletions Based on Missing Data: 1) Variables with as little as 15 percent
missing data are candidates for deletion , but higher levels of missing data (20% to
30%) can often be remedied 2) Be sure the overall decrease in missing data is large
enough to justify deleting an individual variable or case 3) Cases with missing data for
dependent variable(s) typically are deleted to avoid any artificial increase in
relationships with independent variables
Step 4: Select the imputation method Imputation is the process of estimating the
missing value based on valid values of other variables and/or cases in the sample.
Nonmetric variables are not amenable to imputation bc even though estimates of
the missing data can be made w/ such a mean of all value as mean of all valid values,
no comparable measures are available for nonmetric variables Imputation of a MAR
Missing Data Process: use model-based approach (Maximum likelihood and EM:
direct estimation of means and covariance matrix vs Multiple imputation: generating
multiple datasets w/ imputed data differing in each dataset). Imputation of a MCAR
Missing Data Process: 1) Use only valid data (Complete case approach: include only
observations with complete data; use only case w/ no missing data vs Using all
available data: imputes the distribution characteristics/relationships from every valid
value; the imputation process occur by using the obtained correlation on just the
case w/ valid data as representative for entire sample) 2) Using known replacement
Dummy variable Binary metric variable used to represent a single category of a
source for the missing values vs Case substitution: entire observations w/ missing
data are replaced by choosing another non-sampled observation)
3) Calculating replacement value (Mean substitution: replace missing values for a
variable w/ the mean value of tht variable calculated from all valid response vs
Regression imputation: predict the missing values of a variable based on its
relationship to other variables in the dataset)
ROT2-3::Imputation of Missing Data: 1) Under 10% – Any of the imputation methods
can be applied when missing data is this low, although the complete case method has
been shown to be the least preferred. 2) 10 to 20% – The increased presence of
missing data makes the all available, hot deck case substitution and regression
methods most preferred for MCAR data, while model-based methods are necessary
with MAR missing data processes 3) Over 20% – If it is necessary to impute missing
data when the level is over 20%, the preferred methods are: the regression method
for MCAR situations // model-based methods when MAR missing data occurs.
Outliers: an observation/response with a unique combination of characteristics
identifiable as distinctly different from the other observations/responses. Types: 1)
Error 2) Interesting 3 )Influential. Occurs bc 1) Procedural Error (data entry error/ a
mistake in coding) is better if it’s detected and cleaned on data cleaning stage 2)
Extraordinary Event (hurricane occurs when we try to track average daily rainfall) –
should decide whether event fit the research objective or not. If not, should be
deleted. 3) Extraordinary Observations (unique profile that unexpectedly occurs) –
could be represent an emerging element, must use judgement in retention/deletion
selection 4) Observations unique in their combination of values (the observation falls
within the ordinary range of values on each variables) – should retain unless there is
specific evidence that not stated this outlier as valid member of population**Impact
of outliers: 1distance measure become less useful 2) distort results >>irrelevant
varieable. Dealing with Outliers: 1) Identify outliers(designation): after finding
outliers in detecting outliers steps, select only observations that show real
uniqueness compared to the rest of the Q:When should I delete the "star" outliers
and when should I delete the "circle" outliers? You should always delete the outliers
represented as "stars". However, for the "circles", sometimes deleting all of them will
result in the lost of too many observations. You can decide by yourself whether to
delete or not. A good option is to only delete the observations that repeatedly
marked as circle in many items and keep the ones that are marked as circle for only 1
item.
ROT2-4::Outlier Detection: 1) Univariate methods: Examine all metric variables to
identify unique or extreme observations • For small samples (80 or fewer
observations), outliers typically are defined as cases with standard scores of 2.5 or
greater • For larger sample sizes, increase the threshold value of standard scores up to
4 • If standard scores are not used, identify cases falling outside the ranges of 2.5
versus 4 standard deviations, depending on the sample size 2) Bivariate methods:
Focus on specific variable relationships, such as the independent versus dependent
variables • Use scatterplots with confidence intervals at a specified alpha level 3)
Multivariate methods: Best suited for examining a complete variate, such as the
independent variables in regression or the variables in factor analysis • Threshold
levels for the D2/df measure should be conservative (.005 or .001), resulting in values
of 2.5 (small samples) versus 3 or 4 in larger samples
population 2) Describe outliers: generate profile of each outlier, identify the
variable(s) that makes it an outlier. Assign them to the outlier classes to help on
retain/delete decision. 3) Delete/Retain: Outliers should be retained except there
are proof that show they doesn’t represent any observation in the population. If
they do portray a representative element or segment of the population, they should
be retained to ensure generalizability to the entire population. As outliers are
deleted, the researcher runs the risk of improving the multivariate analysis but
limiting its generalizability. Some techniques are less affected by violating certain
assumptions, which is termed robustness.
Testing multivariate analysis Assumptions: foundation for making statistical
interferences & results. Four important assumptions 1. Normality: referring to the
shape of the data distribution for an individual metric variable and its
correspondence to the normal distribution. Univariate normality vs multivariate
normality. Nonnormality: kurtosis- the peakedness or flatness of the distribution
compared w/ normal distribution. (+)value indicates a relatively peaked distribution,
(-) value indicates a relatively flat distribution. Skewnes- distribution is unbalanced &
shift to one side. Impact: serious effects in small samples (fewer than 50 cases), but
the impact effectively diminishes when sample sizes reach 200 cases or more. How
to test normality: 1) Graphical analyses: check of histogram (compare observed data
w/ a distribution) or normal probability plot/straight diagonal line/z-value
(comparing the cumulative distribution of actual data values w/ cumulative
distribution of a normal distribution). If below or above 0, it means not normal.if the
actual data closely follow diagonal, it’s normal 2) Check Kurtosis: peakedness
/flatness of distribution compared to normal distribution.Remedies: data
transformation 2. Homoscedasticity Variance of the error terms appears constant
over a range of predictor variables. It is desirable bc the variance of the dependent
variable being explained in the dependence relationship should not be concentrated
in only a limited range of the independent values. Opposite is termed
Heteroscedasticity: The result of heteroscedasticity is to cause the predictions to be
better at some levels of the independent variable than at others. heteroscedasticity
are result of non-normality in one or more variables. How to test Homoscedasticity
1. Graphical: boxplots; 2. Statistical test: Levene test (univariate), Box’s M
(multivariate). 3. Linearity:the concept that the model possesses the properties of
additivity and homogeneity. Linear models predict values that fall in a straight line by
having a constant unit change (slope) of the dependent variable for a constant unit
change of the independent variable. Y = b0 + b1X1 + e, the effect of a change of 1 in X1
is to add b1 (a constant) units to Y. How to test: 1)Examine scatterplots of the
variables to identify nonlinear patterns in the data. 2) Run a simple regression
analysis to examine the residuals (=Portion of a dependent variable not explained by
a multivariate technique) which reflect the unexplained portion of the dependent
variable. Remedies for Nonlinearity:transform one or both variables to achieve
linearity or explicit model components used to represent the nonlinear portion of the
relationship. 3) Use curve fitting that reflect non-linear elements 4. Non-correlated
Errors Correlated errors arise from a process that must be treated much like missing
data. Correlated error occurs bc 1) data collection process 2) time series data (data
for any time period is highly related to the data time periods both bf and afterward.
How to identify: 1) finding differences in the prediction errors in the two groups 2)
for time series data, see any patterns when we order/sort the data. Remedies:
including the omitted causal factor into the multivariate analysis. 5. Autocorrelation
can’t YX check by seeing D
Data Transformations: A variable may have an undesirable characteristic that
detracts from its use in a multivariate technique. A transformation, such as taking the
logarithm or square root of the variable, creates a transformed variable that is more
suited to portraying the relationship. Transformations may be applied to either the
dependent or independent variables, or both. The need and specific type of
transformation may be based on theoretical reasons (e.g., transforming a known
nonlinear relationship) or empirical reasons (e.g., problems identified through
graphical or statistical means). Help achieve 4 outcomes 1) enhancing statistical
properties – achieve normality, homoscedascity, linearity 2) ease of interpretation-
standardization(provide common metric for comparison) vs centering (allow
comparison across variables) 3) representing specific relationship types 4)
simplification ( binding: categorization of values into smaller no. of categories vs
smoothing: use of response surface method to represent generalized patterns in
data
ROT: Transforming Data
Guideline/Rules of Thumb 2–6 (Transforming Data)
-To judge the potential impact of a transformation, calculate the ratio of the variable’s
mean to its standard deviation:
**Noticeable effects should occur when the ratio is less than 4.
**When the transformation can be performed on either of two variables, select the
variable with the smallest ratio.
-Transformations should be applied to the independent variables except in the case of
heteroscedasticity.
- Heteroscedasticity can be remedied only by the transformation of the dependent
variable in a dependence relationship. If a heteroscedastic relationship is also
nonlinear, the dependent variable, and perhaps the independent variables, must be
transformed.
-Transformations may change the interpretation of the variables. For example,
transforming variables by taking their logarithm translates the relationship into a
measure of proportional change (elasticity). Always be sure to explore thoroughly the
possible interpretations of the transformed variables.
Use variables in their original (untransformed) format when profiling or interpreting
results.
Dummy Variables: a nonmetric independent variable that has two (or more) distinct
levels that are coded 0 and 1(dichotomous). These variables act as replacement
variables to enable non-metric variables to be used as metric variables. To account
for L levels of a nonmetric variable,L- 1 dummy variables are needed. For example,
gender is measured as male or female and could be represented by two dummy
variables (X1 and X2). When the respondent is male, X1 = 1 and X2 = 0 Likewise,
when the respondent is female, X1 = 0 and X2 = 1. However, when X1 = 1, we know
that X2 must equal 0. Thus, we need only one variable, either X1 or X2, to represent
the variable gender. If a nonmetric variable has three levels, only two dummy
variables are needed.We always have one dummy variable less than the number of
levels
Simple approach to understand data: - Tabulation = a listing of how respondents
answered all possible answers to each question. This typically is shown in a
frequency table. - Cross Tabulation = a listing of how respondents answered two or
more questions. This typically is shown in a two-way frequency table to enable
comparisons between groups. - ANOVA= a statistic that tests for significant
differences between two means.
Chapter 2: Tutorial Class
How to Check Missing Data: Click on Analyze > Descriptive Statistics >
Frequencies Click on Statistic box > Check on Minimum, maximum, and arrange in
dispersion part > continue>ok. How to check outliers via Descriptive Statistics
Click on Analyze > Descriptive Statistics > Explore>> insert items into dependent
list>> In statistics choose outliers>>continue>>OK Dealing w/ missing values:
Click on Transform>Replace missing value> insert items w/ missing values into
box (new variable) >In Method choose Series mean>OK
Factor analysis
Exploratory factor analysis (EFA): EFA can be conducted without knowing how many
factors really exist or which variables belong to which constructs. -Explore how many
factors are needed to best present the data. –EFA= the factors were derived from
statistical results, not from theory. It is data driven. Factors can only be named after
the factor analysis is performed. How to get EFA in SPSS: Choose Factor
AnalysisExtractionBased on Eigenvalue > 1
-Confirmatory factor analysis (CFA):CFA is similar to EFA in some respects, but
philosophically it is quite different. -Researcher must know specify both the number
used to provide a confirmatory test our measurement theory. It is theory driven. -
How to get CFA in SPSS -Enter exact number of factor in fixed number of factors. -
How to get CFA in SPSS: Choose Factor Analysis > Extraction > Fixed number of
factors > factor to extract 1
Chapter 3: EFA Factor Analysis
EFA: an interdependence technique whose primary purpose is to define the
underlying structure among the variables in the analysis; examines interrelationships
among a large no. of variables and attempts to explain in terms of common
underlying dimensions ( factors or components); A summarization and data
reduction technique that does not have independent and dependent variables, but is
an interdependence technique in which all variables are considered simultaneously.
Factor Analysis Decision Process: Stage 1: Objectives1. EFA used to discover the
factor structure of a construct and examine its reliability. It is data driven. CFA used
to confirm the fit of the hypothesized factor structure to the observed (sample) data.
It is theory driven 2. Specify the unit of analysis( Variables: R-factor analysis to
analyze a set of variables to identify the dimensions for the variable that are
latent/not easily observed vs Cases: Q factor analysis to combine or condense large
no. of cases into distinctly different groups w/thin a larger population; use cluster
analysis to group indi cases) 3. Data summarization (derive underlying dimensions
that describe the data in a much smaller of concepts than the original indi
variables)and/or reduction (deriving empirical value (factor or summated scale score)
for each dimension (factor) and then substituting this value for the original values in
subsequent analysis)4. Using factor analysis with other techniques Stage 2: Designing
a Factor Analysis Three Basic Decisions: 1. Calculation of input data – R vs. Q analysis.
2. Design of study in terms of number of variables, measurement properties of
variables, and the type of variables. 3.Sample size necessary.
ROT3-1::Factor Analysis Design: 1) Factor analysis is performed most often only on
metric variables, although specialized methods exist for the use of dummy variables; a
small number of “dummy variables” can be included in a set of metric variables that are
factor analyzed 2) If a study is being designed to reveal factor structure, strive to have
at least five variables for each proposed factor 3) For sample size: • The sample must
have more observations than variables • The minimum absolute sample size should be
50 observations • Strive to maximize the number of observations per variable, with a
desired ratio of 5 observations per variable
Stage 3: Assumptions in Factor Analysis 1)Conceptual: assume a homogeneity of sub
sample factor solution 2)Addressing Multicollinearity :Multicollinearity is assessed
using MSA (measure of sampling adequacy). MSA is measured by the Kaiser-Meyer-
Olkin (KMO) statistic. As a measure of sampling adequacy, the KMO predicts if data
are likely to factor well based on correlation and partial correlation. KMO can be
used to identify which variables to drop from the factor analysis because they lack
multicollinearity. There is a KMO statistic for each individual variable, and their sum
is the KMO overall statistic. KMO varies from 0 to 1.0 Overall KMO should be .50
to proceed with factor analysis. If it is not, remove the variable with the lowest
individual KMO statistic value one at a time until KMO overall rises above .50, and
each individual variable KMO is above .50. • Homogeneity of sample factor solutions
(MSA Measure calculated both for the entire correlation matrix and each individual
variable evaluating the appropriateness of applying factor analysis. Values above .50
for either the entire matrix or an individual variable indicate appropriateness.)
ROT3-2::Testing Assumptions of Factor Analysis: 1) A strong conceptual foundation
needs to support the assumption that a structure does exist before the factor analysis
is performed 2) A statistically significant Bartlett’s test of sphericity (sig. < .05) indicates
that sufficient correlations exist among the variables to proceed 3) Measure of
sampling adequacy (MSA) values must exceed .50 for both the overall test and each
individual variable; variables with values less than .50 should be omitted from the
factor analysis one at a time, with the smallest one being omitted each time
Stage 4: Deriving Factors and Assessing Overall Fit 1)Selecting the factor extraction
method: common variance (variance if a variable that is shared w/ all other variables
in the analysis) vs. unique variance (Specific variance: can’t be explained by
correlations to other variables vs Error variance: due to unreliability in data gathering
process, measurement error)2)Determining number of factors to represent the data
3) Principal component (data reduction is a primary concern, prior knowledge
suggests that specific and error variance represent a relatively small proportion of
the total variance) vs Common factor analysis (obj = to identify latent dimensions or
constructs represented in common variance of the original variables )
Stoping Rule: Scree test: identify the optimum no. of factors that can be extracted bf
the amount of unique variance begins to dominate the common variance structure.
Elbow: the point at which the curve first begins to straighten out
Parallel Analysis: empirical measure based on the specific characteristics
Stage 5: Interpreting the Factors 1) Factor Interpretation: 1.1)Estimate the Factor
Matrix (compute unrotated factor loadings – correlation of variable and factor)
1.2)Factor Rotation (employ rational method to achieve simpler and theoretically
more meaningful factor solutions) 1.3) Factor Interpretation and Respecification of
factor model, if needed may involve Deletion of variables from analysis, Desire to use
a different rotational approach, Need
ROT3-3::Choosing Factor Models and Number of Factors: 1) Although both
component and common factor analysis models yield similar results in common
research settings (30 or more variables or communalities of .60 for most variables): •
The component analysis model is most appropriate when data reduction is paramount
• The common factor model is best in well-specified theoretical applications 2) Any
decision on the number of factors to be retained should be based on several
considerations: • Use of several stopping criteria to determine the initial number of
factors to retain: - Factors with eigenvalues greater than 1.0 - A predetermined
number of factors based on research objectives and/or prior research - Enough factors
to meet a specified percentage of variance explained, usually 60% or higher - Factors
shown by the scree test to have substantial amounts of common variance (i.e., factors
before inflection point) - More factors when heterogeneity is present among sample
subgroups • Consideration of several alternative solutions (one more and one less
factor than the initial solution) to ensure the best structure is identified
to extract a different number of factors. -Desire to change method of extraction)
2)Factor Extraction 3) Rotation of Factors :The ultimate effect of rotating the factor
matrix is to redistribute the variance from earlier factors to later ones to achieve a
simpler, theoretically more meaningful factor pattern. Orthogonal: axes are
maintained at 90 degrees. vs Oblique: axes are not maintained at 90 degrees).
Orthogonal Rotation Methods- Quartimax (simplify rows) - Varimax (simplify
columns) - Equimax (combination 4) Significance of Factor Loadings (Practical
Significance vs Statistical Significance: sample size) 5) Interpreting Factor Matrix
rotation (factor loading matrix, factor loading for each variable, communalities of
variables, respecify model if needed, label factor)
ROT3-4::Choosing Factor Rotation Methods: 1) Orthogonal rotation methods: • Are
the most widely used rotational methods • Are the preferred method when the
research goal is data reduction to either a smaller number of variables or a set of
uncorrelated measures for subsequent use in other multivariate techniques 2) Oblique
rotation methods: Are best suited to the goal of obtaining several theoretically
meaningful factors or constructs, because, realistically, few constructs in the real
world are uncorrelated
ROT3-5::Assessing Factor Loadings: 1) Although factor loadings of ±.30 to ±.40 are
minimally acceptable, values greater than ±.50 are generally considered necessary for
practical significance 2) To be considered significant: • A smaller loading is needed
given either a larger sample size or a larger number of variables being analyzed • A
larger loading is needed given a factor solution with a larger number of factors,
especially in evaluating the loadings on later factors 3) Statistical tests of significance
for factor loadings are generally conservative and should be considered only as starting
points needed for including a variable for further consideration
ROT3-6::Interpreting the Factors: 1) An optimal structure exists when all variables
have high loadings only on a single factor 2) Variables that cross-load (load highly on
two or more factors) are usually deleted unless theoretically justified or the objective
is strictly data reduction 3) Variables should generally have communalities of greater
than .50 to be retained in the analysis 4) Respecification of a factor analysis can
include such options as the following: • Deleting a variable(s) • Changing rotation
methods • Increasing or decreasing the number of factors
Stage 6: Validation of Factor Analysis 1)Replication vs Confirmatory Perspective
2)Assessing Factor Structure Stability (large sample provide more confidence as to
generalizability and stability) 3) Detecting Influential Observations(estimate model
w/ and w/out observations identified as outliers to assess impacts on the results)
Stage 7: Additional uses of Factor Analysis Results 1)Selecting Surrogate Variables
2)Creating Summated Scales 3) Computing Factor Scores
ROT3-7::Summated Scales: 1) A summated scale is only as good as the items used
to represent the construct; even though it may pass all empirical tests, it is useless
without theoretical justification 2) Never create a summated scale without first
assessing its unidimensionality with exploratory or confirmatory factor analysis 3)
Once a scale is deemed unidimensional, its reliability score, as measured by
Cronbach’s alpha: • Should exceed a threshold of .70, although a .60 level can be
used in exploratory research • The threshold should be raised as the number of
items increases, especially as the number of items approaches 10 or more 4) With
reliability established, validity should be assessed in terms of: • Convergent validity
scale correlates with other like scales • Discriminant validity scale is sufficiently
different from other related scales • Nomological validity scale “predicts” as
theoretically suggested
ROT3-8::Representing Factor Analysis in Other Analyses: 1) The Single Surrogate
Variable Advantages: • Simple to administer and interpret Disadvantages: • Does not
represent all “facets” of a factor • Prone to measurement error 2) Factor Scores
Advantages • Represent all variables loading on the factor • Best method for complete
data reduction • Are by default orthogonal and can avoid complications caused by
multicollinearity Disadvantages • Interpretation more difficult because all variables
contribute through loadings • Difficult to replicate across studies 3) Summated Scales
Advantages • Compromise between the surrogate variable and factor score options •
Reduce measurement error • Represent multiple facets of a concept • Easily
replicated across studies Disadvantages • Include only the variables that load highly on
the factor and excludes those having little or marginal impact • Not necessarily
orthogonal • Require extensive analysis of reliability and validity issues
VOCABS Anti-image correlation matrix Matrix of the partial correlations among
variables after factor analysis, representing the degree to which the factors explain
each other in the results. The diagonal contains the measures of sampling adequacy
for each variable, and the off-diagonal values are partial correlations among variables
Bartlett test of sphericity Statistical test for the overall significance of all correlations
within a correlation matrix. Cluster analysis Multivariate technique with the
objective of grouping respondents or cases with similar profiles on a defined set of
characteristics. Communality Total amount of variance an original variable share
with all other variables included in the analysis. Component analysis Factor model in
which the factors are based on the total variance. Correlation matrix Table showing
the intercorrelations among all variables. Cronbach’s alpha Measure of reliability
that ranges from 0 to 1, with values of .60 to .70 deemed the lower limit of
acceptability. Cross-loading A variable has two more factor loadings exceeding the
threshold value deemed necessary for inclusion in the factor interpretation process.
nonmetric variable. Eigenvalue Column sum of squared loadings for a factor; also
referred to as the latent root. It represents the amount of variance accounted for by
a factor.EQUIMAX One of the orthogonal factor rotation methods that is a
“compromise” between the VARIMAX and QUARTIMAX approaches, but is not widely
used. Factor Linear combination (variate) of the original variables. Factors also
represent the underlying dimensions (constructs) that summarize or account for the
original set of observed variables. Factor loadings Correlation between the original
variables and the factors, and the key to understanding the nature of a particular
factor. Squared factor loadings indicate what percentage of the variance in an
original variable is explained by a factor. Factor rotation Process of manipulation or
adjusting the factor axes to achieve a simpler and pragmatically more meaningful
factor solution. QUARTIMAX A type of orthogonal factor rotation method focusing
on simplifying the rows of a factor matrix. Generally considered less effective than
the VARIMAX rotation. Q factor analysis Forms groups of respondents or cases based
on their similarity on a set of characteristics. R factor analysis Analyzes relationships
among variables to identify groups of variables forming latent dimensions (factors).
Surrogate variable Selection of a single variable with the highest factor loading to
represent a factor in the data reduction stage instead of using a summated scale or
factor score VARIMAX The most popular orthogonal factor rotation methods
focusing on simplifying the columns in a factor matrix. Generally considered superior
to other orthogonal factor rotation methods in achieving a simplified factor
structure.
Chapter 3: EFA Tutorial
Factor Analysis Steps: 1. Click on Analyze > Dimension Reduction > Factor Select> put
it in “variable box” 2. Click on Descriptive and Check: Initial Solution, KMO &
Bartletts, Anti-image, and Continue 3. Click on Extraction and then [Methods: Select
“Principle components”] -> Check: Correlation Matrix, Unrotated factor solution,
Based on Eigenvalue “type 1 in the box”, Scree plot (optional) 4. Click on Rotation
and Check: a. VARIMAX, b. Rotated Solution, c. Max Iteration 25 -> Continue 6. Click
on Options and Check “Sorted by size” Reliability Analysis Steps All items remained
from the results of factor analysis will be adopted to proceed the reliability test. 1.
Click on Analyze -> Scale >Reliability Analysis -> Select the (items) variables from the
left panel (but you do need to make sure that the items are the formal results of
factor analysis steps) and move it to the right panel “Items box” 2. Click on Statistics -
> Check: Scale if item deleted, Correlations -> Continue -> OK
EFA RT
KMO > 0.5 predicts if data are likely to Item-to-total correlation > 0.50
factor well based on correlation and correlation between an individual item
partial correlation + can be used to and the total score without that item. to
identify which variables to drop from the check if any item in the set of tests is
factor analysis because they lack inconsistent with the averaged behavior of
multicollinearity the others, and thus can be discarded.
Component Metrix “Factor Loading” > Cronbach’s alpha (α) > 0.60 exploratory &
0.6 > 0.70 confirmatory Measure of reliability
see correlation between the original that ranges from 0 to 1, with values of .60
variables and the factors, and the key to to .70 deemed the lower limit of
understanding the nature of a factor acceptability
Tot. Var. Explained – “Eigen value” Total Factor loadings: plays a large role in that
> 1 to determine the number of factors to factor. The relationship of each var to the
extract underlying factor (eg. XX has high factor
Tot. Var. Explained - “Cumulative %” > loading means it is the strongest
60% way of expressing frequency association to the underlying
distribution. It calculates the percentage var)Communality: amount of var in each
of the cumulative frequency within each var accounted for by all components//
interval, much as relative frequency strength of factor in explaining each var.
distribution calculates the percentage of Low communality means the var has little
frequency. common with other variable. Cumulative
(Explained variance): the %of var
accounted for by the n components. Item-
Communality > 0.5 Total amount of to-correlation: the α if item deleted, we
variance an original variable share with all can see that when remove any item and α
other variables included in the analysis is lower, so reconsider not to remove.
Cronbach’s Alpha (α), the items are
reliable
Some Issues w/ EFA: KMO does not appear in the results Problem: If the correlation
matrix is nonpositive definite (some of the eigenvalues of the correlation matrix are
not positive numbers) and the KMO will not be displayed. Why: This may happen if
variables depend on other variables, such as one variable being a product or sum of
other variables. Solution: Go back and check and try to delete the troublesome
items. Always delete 1 item each time and observe the changes. Factor Rotation:
There are two ways of rotations Orthogonal rotations (varimax, quatimax, equamax)
constrain the factors to be uncorrelated vs Oblique rotations (direct oblimin, promax)
permit the factors to be correlated with one another. Solutions: Try oblique rotation
first, if no correlation is detected  try orthogonal rotation **Run Oblique rotation>
Check Component Correlation Matrix>If all correlation coefficients are greater than
0.32  factors are correlated  should use oblique rotation. Factor Analysis
Option: Apart from “sort by size and exclude cases listwise” > You can choose
Suppress Small Coefficients to have a better look of the matrix but should not be
higher than 0.3. Cross-loading: When 1 item is loaded on more than 1 factor, or in
other words, the difference between the loadings across factors is greater than 0.3
(strict) or 0.2 (more flexible).If the difference is too small, you can delete the item.If
the difference is not too small, and one loading meets requirement in one factor,
you can consider to categorize it into the factor with higher loadings. Negative
Loading: When consider the value of a loading or coefficient, we need to consider
two components: The magnitude (absolute value)- When we say a coefficient must
be greater than 0.6, it means the absolute value should be greater than 0.6. The sign
(+ or -) showing the direction the item is related to the factor. In case negative
loadings appear, you should reconsider your item statement and your data, most
cases, it is because of your item is a reversed one. Cronbach Alpha: based on
standardized item. Result: If the resultsare closed to criteria, you might consider to
keep the items instead of deleting. Factor: One factor should have at least 2 items
(Either delete the item, which means deleting the factor, or Try to force the item into
the most correlated factor). Eigen Value: Consider to delete Eigen value in the
summary table (optional)
Chapter 5: Multiple Regression
Simple Regression: when obj of regression involves a single independent variable.
Multiple Regression: a statistical technique that can be used to analyze the
relationship between a single dependent (criterion) variable and several independent
(predictor) variables. Formula: Y’ = b0 + b1X1 + b2X2 + . . . + bnXn + e Y=Dependent
Variable, b0=intercept (constant), b1= (regression coefficient or parameter estimate
how much of which X is depended on Y), X1 = Independent Variable 1, e = prediction
error (residual). (A variate value (Y’) is calculated for each respondent. The Y’ value is
a linear combination of the entire set of variables that best achieves the statistical
objective. Impact of Multicollinearity: Collinearity: the association measured as the
correlation btw two independent variables. Multicollinearity: the correlation among
three or more independent variables and evidenced when one is regressed against
others. Impact measures of predictive power: the impact of multicollinearity is to
reduce any single independent variable’s unique predictive power by the extent to
which it’s associated w/ other independent variables. If collinearity increases, the
unique variance explained by each independent variable decrease >> shared
prediction percentage increases. Multicollinearity has the impact of reducing the
impacted variable’s regression coefficients. Favors variables w/ low multicollinearity:
bc it will maximize the prediction from the given no. of independent variables
Multiple Regression Decision Process: Stage 1: Objectives of Multiple Regression
researcher must consider three primary issues: 1) the appropriateness of the
research problem (Predictive purpose- maximize predictive accuracy by ensuring the
validity of the set of independent variables , model comparison by comparing two or
more independent variables to ascertain the predictive power of each variate vs
Explanation purpose – relative imp of independent variables by assessing magnitude
and direction of each independent variables, nature of relationship w/ dependent
variables, nature of relationships among independent variables (multicollinearity) ) 2)
specification of a statistical relationship (Multiple regression is appropriate when
research is interested in statistical relation , not functional relationship(expect no
error in prediction, only calculate the exact value). Some random component is
always present (error in predicting variable) in the relationship being examined;
calculate the average value) 3) selection of the dependent and independent
variables (a. support theory either conceptual or theoretical b. measurement error c.
specification error). Measurement error: problematic can be addressed through
either of two approaches: Summated scales- mitigate measurement error, or
Structural equation modeling procedures-can directly accommodate measurement
error. Specification error: the exclusion of relevant and inclusion of irrelevant
independent variables. **when in doubt, include potentially irrelevant variables
(they can only confuse interpretation) rather than omitting a relevant variable (which
can bias all regression estimates)
Stage 2: Research Design of a Multiple Regression Analysis Issues to consider
1)Sample Size 2) Creating additional variables
ROT4-2::Sample Size Considerations: Statistical power1) Simple regression can be
effective with a sample size of 20, but maintaining power at .80 in multiple
regression requires a minimum sample of 50 and preferably 100 observations for
most research situations. Generalizability 2) The minimum ratio of observations to
variables is 5 to 1, but the preferred ratio is 15 or 20 to 1, and this should increase
when stepwise estimation is used. 3) Maximizing the degrees of freedom improves
generalizability and addresses both model parsimony and sample size concerns.
ROT4-3::Variable Transformations: 1) Nonmetric variables can only be included in a
regression analysis by creating dummy variables. 2) Dummy variables can only be
interpreted in relation to their reference category. 3) Adding an additional
polynomial term represents another inflection point in the curvilinear relationship.
4) Quadratic กำลังสอง and cubic กำลังสาม polynomials are generally sufficient
to represent most curvilinear relationships. 5) Assessing the significance of a
polynomial or interaction term is accomplished by evaluating incremental R2, not
the significance of individual coefficients, due to high multicollinearity.
Moderator Effect: When the moderator variable, a second independent variable,
changes the form of the relationship between another independent variable and the
dependent variable. The moderator term is a compound variable formed by
multiplying X1 by the moderator X2, >> (X1X2). The coefficient (b3) of the
interaction/moderator term indicates the unit change in the effect of X 1 as X2
changes. The coefficients (b1, b2) of the two independent variables now represent the
effects when the other independent variable is zero. Mediator effect: when the
effect of an independent variable may “work through” an intervening variable (the
mediating variable) to predict the dependent variable
Stage 3: Assumptions in Multiple Regression Analysis 1) Linearity of the
phenomenon measured 2) Homoscedasticity: Constant variance of the error terms
3) Normality of the error term distribution 4) Independence of the error terms.
ROT4-4:: Assessing Statistical Assumptions: 1) Testing assumptions must be done not only
for each dependent and independent variable, but for the variate as well. 2) Graphical
analyses (i.e., partial regression plots, residual plots, and normal probability plots) are the
most widely used methods of assessing assumptions for the variate. 3) Remedies for
problems found in the variate must be accomplished by modifying one or more independent
variables
Lin
earity: relationship btw dependent and independent variables represents the degree
to which the change in the dependent variable is associated w/ the independent
variable. Examined through residual plots and comparison to null plot Q: how to
determine which independent variable to select for corrective action - use partial
regression plot: shows the relationship of a single independent variable to the
dependent variable, controlling for the effects of all other independent
variables .Corrective action 1) data value transformation 2)include polynomial in
regression model 3) non-linear regression method . Homoscedasticity การกระจายของค่า
คลาดเคลื่อน (residuals) : Diagnosis with graphical plots or simple statistical tests.
Remedies include: 1) variable transformation 2) weighted least squares - weight
each observation based on its variance and mitigates the variation in variance of
residual seen in heteroscedasticity 3) heteroscedasticity-consistent standard errors
(HCSE) - estimates of standard errors are corrected for any heteroscedasticity that
may be present. Normality: normal probability plot- the standardized residuals are
compared w/ normal distribution. Normal distribution makes straight diagonal line
and plotted residuals are compared w/ diagonal. If distribution is normal, the residual
line closely follows the diagonal. Regression considered robust to violations of
normality when sample size exceeds 200. Independence of error term: Predicted
values are not correlated to any variables in the analysis. Residuals plotted versus
offending variables. Offending variables 1) time series data - represent the
observations on the same unit over multiple occasions 2)cluster
data/grouping/sequencing variable - when data is hierarchically distributed e.g.
student and classroom Stage 4: Estimating the Regression Model and Assessing
Overall Model Fit Variable specification 1)Use variables in their original form: Allows
for use of direct measures of the variables of interest. As the number of variables
increases, interpretability may become problematic.2) Dimensional reduction: either
software controlled (principal component regression or user controlled (EFA).
Variable selection 1) user-controlled 1.1) confirmatory: allow direct testing of a pre-
specified model 1.2) Combinatorial (All-Possible-Subsets): allowing the researcher to
review the entire set of roughly equivalent models in terms of predictive accuracy.
2)Software controlled 2.1) Sequential search method 2.1.1) Forward inclusion and
backward elimination: trial and error process for finding the best regression
estimates. Forward inclusion: builds regression equation starting w/ a single
independent variable 2.1.2)Backward elimination: starts w/ regression equation
including all independent variables and then deletes independent variables that do
not contribute significantly 2.1.3) Stepwise: variables possible removed once
included in regression equation Q: Dif btw forward inclusion & backward
elimination and stepwise: stepwise has the ability to add or delete variables at each
stage. Once variable is deleted in forward or backward, action can’t be reserved at a
later stage. 2.2) Constrained: A variant of sequential methods whereby variables
regression weights are constrained to maximize parsimony in the final model results.
2.2.1) Ridge regression: method employing shrinkage; the regression estimates are
shrunk based on the tuning parameter or ridge estimator. Obj: to reduce the
regression estimates that are indicated by the effects of multicollinearity loser tw
their true value. 2.2.2) Lasso: adding a variable selection component, which comes
from the addition of additional constraint on the estimated coefficient. Result= small
coefficient can be reduced to 0 >> eliminating them from the variate.
ROT4-5::Estimation Techniques: 1) No matter which estimation technique is chosen,
theory must be a guiding factor in evaluating the final regression model because:
• Confirmatory Specification, the only method to allow direct testing of a pre-
specified model, is also the most complex from the perspectives of specification error,
model parsimony and achieving maximum predictive accuracy. • Sequential search
(e.g., stepwise), while maximizing predictive accuracy, represents a completely
“automated” approach to model estimation, leaving the researcher almost no control
over the final model specification. • Combinatorial estimation, while considering all
possible models, still removes control from the researcher in terms of final model
specification even though the researcher can view the set of roughly equivalent
models in terms of predictive accuracy. 2) No single method is “Best” and the prudent
strategy is to use a combination of approaches to capitalize on the strengths of each
to reflect the theoretical basis of the research question.
Components of Model Fit: 1) Total Sum of Squares (SST): total amount of variation
that exists to be explained by the independent variables. TSS = the sum of SSE and
SSR. 2)Sum of Squared Errors (SSE): the variance in the dependent variable not
accounted for by the regression model = residual. Obj is to obtain the smallest
possible sum of squared errors as a measure of prediction accuracy. 3) Sum of
Squares Regression (SSR): the amount of improvement in explanation of the
dependent variable attributable to the independent variables. We want SSR to be
good.
Measures of model good: 1) F statistic is used to determine if the overall regression
model is statistically significant. If the F statistic is significant, it means it is unlikely
your sample will produce a large R2 when the population R2 is zero. To be considered
statistically significant, a rule of thumb is there must be <.05 probability for
statistical significance 2) R2 (Coefficient of Determination) – strength of overall
variate relationship. If the R2 is statistically significant, we then evaluate the strength
of the linear association between the dependent variable and the several
independent variables. R2, also called the coefficient of determination, is used to
measure the strength of the overall relationship. R2 ranges from 0 to 1.0. The sign (+
or -) indicates the direction of the relationship. The value can range from +1 to -1,
with +1 indicating a perfect positive relationship, 0 indicating no relationship, and -1
indicating a perfect negative or reverse relationship. It represents the amount of the
dependent variable “explained” by the independent variables combined. A large R2
indicates the straight line works well while a small R 2 indicates it does not work well.
Even though an R2 is statistically significant, it does not mean it is practically
significant. We also must ask whether the results are meaningful. For example, is the
value of knowing you have explained 4 % of the variation worth the cost of collecting
and analyzing the data? 3) Adjusted R2 is based on no. of independent variables
relative to sample size >> makes allowance for degree of freedom of each model.
Fewer observations will reduce adjusted R2 Significant Test for Regression
Coefficient: a statistically based probability estimates of whether the estimated
coefficient across large no. of samples of a certain size will indeed be different from
zero 1) Establishing confidence interval: desired alpha=0.05, standard error:
expected sampling error of coefficient, confidence interval: no. of standard error
based on alpha level times the value of the standard error 2) Applying confidence
interval: statistical significance established if confidence interval does not include
zero , sample size has direct effect on standard error (increase sample size will
decrease standard error), must ensure practical significance when using large sample
size. Influential Observations: all observation that lie outside general pattern of data
set and has disproportionate effect on the regression result. Types: 1) Outliers:
observations that have large residual values and can be identified only with respect
to a specific regression model. 2) Leverage points: observations that are distinct
from the remaining observations based on their independent variable values.
3)Influential observations are the broadest category, including all observations that
have a disproportionate effect on the regression results. Influential observations
potentially include outliers and leverage points but may include other observations
as well. Impact: 1) Reinforcing: reinforcing general pattern and lowering standard
error of the prediction and coefficient. It‘s leverage point but has small/zero residual
value bc it’s predicted well by regression model 2) Conflicting: has effect that is
contrary to the general pattern of remaining data but still have small residuals 3)
Shifting: affect all results in similar manner 4 groups of influential observations: 1)
no issues: fit well and no extreme values 2)high leverage but no outlier: very
different but still predicted well by the model 3) outlier but acceptable leverage:
high residual, no extreme value 4)outlier and high leverage: poor prediction and
quite different << we want to delete this type. Corrective Actions: 1)An error in
observations or data entry : remedy by correcting the data or deleting the case 2)A
valid but exceptional observation that is explainable by an extraordinary situation:
remedy by deletion of the case unless variables reflecting the extraordinary situation
are included in the regression equation 3)An exceptional observation with no likely
explanation: presents a special problem bc there is no reason for deleting the case,
but its inclusion cannot be justified either, suggesting analyses w/ and w/ouut the
observations to make a complete assessment, and 4) An ordinary observation in its
individual characteristics but exceptional in its combination of characteristics:
indicates modifications to the conceptual basis of the regression model and should
be retained.Stage5: Interpreting the Regression Variate
Multicollinearity: relationship btw two (collinearity) and more (multicollinearity)
independent variables. Occurs when any single variable independent variable is
highly corelated w/ a set of other independent variables. Measure of correlation
incorporating multicollinearity: 1) Bivariate/zero-order correlation: association btw
two variables, not accounting for the variation shared w/ any other variables 2) semi-
partial/ part correlation: unique predictive effect 3) partial correlation: incremental
predictive effect. Multicollinearity Identification: 1) Variance Inflation Factor (VIF):
measures how much the variance of the regression coefficients is inflated by
multicollinearity problems. If VIF equals 0, there is no correlation between the
independent measures. A VIF measure of 1 is an indication of some association
between predictor variables, but generally not enough to cause problems. A
maximum acceptable VIF value would be 5; anything higher would indicate a
problem with multicollinearity. 2)Tolerance the amount of variance in an
independent variable that is not explained by the other independent variables. If the
other variables explain a lot of the variance of a particular independent variable we
have a problem with multicollinearity. Thus, small values for tolerance indicate
problems of multicollinearity. The minimum cutoff value for tolerance is typically
0.20. That is, the tolerance value must be smaller than 0.20 to indicate a problem of
multicollinearity.
Residual Plots 1) Histogram of standardized residuals: enables you to determine if
the errors are normally distributed. 2) Normal probability plot enables you to
determine if the errors are normally distributed. It compares the observed (sample)
standardized residuals against the expected standardized residuals from a normal
distribution. 3) Scatter Plot of residuals: can be used to test regression assumptions.
It compares the standardized predicted values of the dependent variable against the
standardized residuals from the regression equation. If the plot exhibits a random
pattern, then this indicates no identifiable violations of the assumptions underlying
regression analysis.
Stage6: Validation of the result: ensure that model represent general population
(generalizability) and appropriate for the situation in which in will be used
(transferability) 1)Additional or split sample: comparing results to ensure
comparability of results across different samples 2)calculating PRESS statistics:
employ jackknife procedure to calculate measure of predictive fit and is used to
calculate P2 (coefficient of prediction) 3)comparing regression model: R2 will increase
as variables added 4) forecasting: ensure comparability of new data to data set used
in estimation
VOCAB: Beta coefficient Standardized regression coefficient that allows for a direct
comparison between coefficients as to their relative explanatory power of the
dependent variable. Degrees of freedom (df) Value calculated from the total number
of observations minus the number of estimated parameters Partial F (or t) values
The partial F-test is simply a statistical test for the additional contribution to
prediction accuracy of a variable above that of the variables already in the equation
Chapter 05: Multiple Regression Tutorial /Multiple Regression Step by Step:
1. Compute mean scores: compute the mean score of each factor in independent
construct and the factor of the dependent construct. Should pick up the final formal
results of factor analysis stage (how many items remain after analysis). Step1: Click
transform > Compute Variable Step 2: In Target variable, type factor name Step3: In
numeric expression, enter target variable>OK 2. How to product result Step1: Click
Analyze >> Regression >> Linear >> Select a factor of Dependent construct from the
left panel to “Dependent” >> Select all the factors of independent constructs from
the left panel to “Independent” Step 2: Click on “Statistics” >> Select “R squared
change”, “Collinearity Diagnostics”, “Durbin-Watson”, and “Covariance Matrix” >>
Click on “Continue.” Step3: Click on Method” >>Stepwise>> OK For next round, you
may select “Enter” 3. How to check and select result Model Summary: R-square,
Adjusted R-square, Durbin-Walson (D-W) and other related values. ANOVA: F-value
and significant level (sig.) Coefficients: Standardized Coefficient (Beta—β), t-value, p-
value, and VIF (VIF range) Excluded Variables (Deletion/Reduction) need to compare
w/ t-value (>1.96) and p-value criteria (p <0.05 or <0.001 or 0.01)
(Coefficient)Beta (β) > 0.015 represents
how many units in the dependent variable R2 Change In stepwise, each time of
will increase if the independent variable adding more independent variables into a
increases by 1 unit. It shows the model, the program compares the new
magnitude of the relationship. When model with the previous one. R squared
interpreting Beta, you need to pay change represents the improvement of R
attention to the sign of Beta to check squared gained by adding those new
consistent with the direction of the independent variables. The R-square
hypothesized relationship. Also, Beta change is tested with an F-test, which is
coefficient of different independent referred to as the F-change. A significant
variables can be compared with each F-change means that the variables added
other. By doing that, we know which in that step significantly improved the
independent variables have stronger prediction.
effect on the dependent variable.
Adj.R2 > 0.1 modified version of R-
squared that has been adjusted for the
number of predictors in the model. The R2 > 0.1 represent the proportion of
adjusted R-squared increases only if the variance in the dependent variable
new independent variable improves the explained by the independent variables.
model more than would be expected by The higher the R squared, the better the
chance. It decreases when a predictor model can predict the dependent
improves the model by less than expected variable. (how much X can explain Y)
by chance. Show % the means explain the
variance
VIF range < 10 (ideally < 3) shows us the
DW (1.5 - 2.5) shows us the severity of
severity of multicollinearity.
autocorrelation. If autocorrelation is
Multicollinearity happens when
severe, that means the assumption that
independent variables are highly
each observation is independent is
correlated with each other. If greater than
violated (Cannot YX). If w/thin this
1 (predictor is moderately correlated, VIF
range, there’s no first order linear auto
must <3. If the VIP>10 the regression is
correlation in data
poorly estimated due multicollinearity.
P-Value < 0.05 (ANOVA, Sig ***p<0.001, F-Value≥ 4 (ANOVA) means that the
** p<0.01, *p<0.05) indicates how the regression model explain a significant
hypothesis significances. Represents the amount of the variance (variance = data
probability of type I error (when we think spread, differentiate among variables; if
that there is a relationship even though sig. at P.. level spread enough)
there is not). For example, a p-value of T-Value > 1.96 means coefficient level.
0.015 means there is 1.5% chance that the check sign negative or positive
relationship we found does not exist relationship of Y and X
How to interpret the result? :1) Check the Correlation between all variables Click on
Analyze > Correlate > Bivarate, then enter all variable 2) Report the multiple
regression table
Method: Stepwise (this method will choose the best IND variables into equation and
exclude the bad IND variables, then it generates models with different).
SAMPLE Multiple Rereession INTERPRETATIO:In the 1st and 2nd model, use the
stepwise method. This method will make the variables in the model could explain a
significant amount of additional variance. In the model 1, we found that the R² of
our model is 0.514 with the adjusted R² = .511. This means that the linear regression
explains 51.4% of the variance.
(There is no cut-off value of R². In business field, R² bigger than 0.1 indicate the
acceptable explanation of the variance, but many cases they accept the lower R²). In
Anova table, The F-test is highly significant (F=209.036, p<0.001), which means the
regression model explains a significant amount of the variance in Dependent
variable. (Criteria for F test: F>4, p<0.05). The “Behavior Innovation” significantly
predicted the “Technology Innovation” with the coefficient β = 0.717*** (p<0.001)
(criteria for beta: t>1.96, p<0.05) .Durbin-Watson test for auto-correlation test: The
Durbin-Watson d = 1.612, which is between the two critical values of 1.5 < d < 2.5.
Therefore, we can assume that there is no first order linear auto-correlation in our
multiple regression data. For the multicollinearity test in our multiple regression
model, the VIF = 1, there is no multicollinearity among factors. (If the VIF is greater
than 1, the predictors may be moderately correlated, and VIF should <3. If the
VIF>10, you can assume that the regression coefficients are poorly estimated due to
multicollinearity). In any model, if there is at least 1 beta significant and other criteria
are met, you could conclude that hypothesis is supported.
3) Check for normality of residuals with a normal P-P plot: Step1: Click on Analyze
>Regression >Linear >Plots Step2: Choose Scatter 1: Y-> *ZRESID X as *ZPRED In
Standardized Residual Plots, choose Histogram and Normal Probability Plot. Result:
The plot shows that the points generally follow the normal (diagonal) line with no
strong deviations. This indicates that the residuals are normally distributed.
Different between stepwise and enter method? Stepwise method allows the
program to test many competing models. It starts with a model with only 1
independent variable, then add more variables in the subsequent models to see if
there is any improvement. If the program finds that adding 1 more certain variable
will not improve much the predictive power of the model, then that variable will be
dropped and not showed in the result. Enter method gives full control to the
researcher. We can decide exactly which variables are included in the model. With
this method, we can test the model that has all independent variables included.
Which criteria are used to conclude whether a hypothesis is supported or not? The
most important indices are Beta, t-value and p-value, R-squared and adjusted R
squared. A hypothesis is supported only when all of those indices satisfy the criteria.
DW and VIF range are just for reference. It is better if they satisfy the criteria but if
they don't, that does not necessary mean the hypothesis is not supported.
Chapter 08: Logistics Regression: a specialized form of regression that is formulated
to predict and explain a binary (two-group) categorical variable rather than a metric
dependent measure. Q: Why Logistic Regression is preferred? ANS: 1) Logistic
regression does not face these strict assumptions unlink discriminant analysis 2)
similar to multiple regression (incorporating metric and nonmetric variables and
nonlinear effects and a wide range of diagnostics)
Logistic Regression Decision Process: Stage 1: Objectives of Logistic Regression 1)
Explanation: Identifying the independent variables that impact group membership in
the dependent variable; Estimating the importance of each independent variable in
explaining group membership 2) Classification: Establishing a classification system
based on the logistic model for determining group membership Stage 2: Research
Design for Logistic Regression
1)Representation of the Binary Dependent Variable (Represents discrete events or
objects in only two classes -dependent variable can only take on two values (1 or 0),
need a relationship form that is non-linear so that the predicted values will never be
below 0 or above 1) 2) Sample Size (400 to achieve best results with maximum
likelihood estimation, smaller samples sizes less efficient in model estimation. MLE
requires larger samples such that, all things being equal, logistic regression will
require a larger sample size than multiple regression. Overall optimal: 400 minimal:
at least 10 per group) 3) Use of Aggregated Data: (logistic regression can analyze
aggregated data if all the independent variables are nonmetric- e.g. binary variables
would result in 8 combinations (2 x 2 x 2) Stage 3: Assumptions of Logistic
Regression Advantage of logistic regression: the result of the general lack of
assumptions, not require linear relationships between IND var and dep var, not
requires Heteroscedasticity of the IND variables as does multiple regression Primary
assumption is independence of observation. Assumption should be addressed with
Box-tidwell - Stage 4: Estimation of the Logistic Regression Model and Assessing
Overall Fit
**logistic curve (S-shaped) within range of 0 and 1
1)Transforming a probability into odds and logit values- logit values are at 1 and 0. 2)
Model estimation using a maximum likelihood approach (Maximizes the likelihood
that an event will occur) .Comparison of likelihood values: Estimate a Null Model,
Estimate Proposed Model, Assess 2LL Difference(=-2loglikelihood; the minimum
value for -2LL is 0 which corresponds to a perfect fit where likelihood=1 thus the
lower -2LL value, the better the fit of the model ) . Measures of Model Fit: Global Null
Hypothesis Test(reflects that at least one of the estimated coefficient is
significant ,which is similar to F test in multiple regression), Pseudo R2 Measures
(Interpreted in a manner similar to the coefficient of determination in multiple
regression; Different pseudo R2 measures vary widely in terms of magnitude and no
one version has been deemed most preferred; ranges from 0.0 to 1.0). Issues in
Model Estimation: Small sample sizes- Difficult to accurately estimate coefficients
and standard errors; complete separation = Dependent variable is perfectly
predicted by an independent variable ; quasi-complete separation (zero cells effect) =
One or more of the groups defined by the nonmetric independent variable have
counts of zero ; when there is no controlled group
ROT7-1::Logistic Regression: 2) Sample size are primarily focused on the size of each
group, which should have 10 times the number of estimated model coefficients 4)
Model significance tests are made with a chi-square test on the differences in the log
likelihood values (-2LL) between two models 5) Coefficients are expressed in two
forms: original and exponentiated to assist in interpretation 6) Interpretation of the
coefficients for direction and magnitude is as follows: • Direction can be directly
assessed in the original coefficients (positive or negative signs) or indirectly in the
exponentiated coefficients (less than 1 are negative, greater than 1 are positive •
Magnitude is best assessed by the exponentiated coefficient, with the percentage
change in the dependent variable shown by: Percentage change in odds =
(Exponentiated coefficient - 1.0) × 100
ROT7-2::Model estimation and Model Fit: 1) Although stepwise estimation may
seem “optimal” by selecting the most parsimonious set of maximally discriminating
variables, beware of the impact of multicollinearity on the assessment of each
variables discriminatory power 2) overall model fit assesses the statistical
significance between groups on the discriminant Z scores, but doesn’t assess
predictive accuracy 3) with more than two groups, do not confine your analysis to
only the statistically significant discriminant functions, but consider if nonsignificant
functions (with sig. levels up to 0.3) add explanatory power.
ROT7-3::Assessing Predictive Accuracy: There are multiple criteria for comparison to
the hit ratio: 1) The maximum chance criterion for evaluating the hit ratio is the most
conservative, giving the highest baseline value to exceed. 2) Be cautious in using the
maximum chance criterion in situations with overall samples less than 10 and/or
group sizes under 20. 3) The proportional chance criterion considers all groups in
establishing the comparison standard and is the most popular 4) The actual
predictive accuracy (hit ratio) should exceed the any criterion value by at least 25%
The chi-square test is used to evaluate the reduction in the log likelihood value.
However, these statistical tests are particularly sensitive to sample size. Nagelkerke
R2 proposed a modification that had the range of 0 to 1. Both of these additional
measures are interpreted as reflecting the amount of variation accounted for by the
logistic model, with 1.0 indicating perfect model fit. *The Wald statistic is used to
assess significance in a manner similar to the t test used in multiple regression.
PREDICTIVE ACCURACY: based on cut-off value is 0.50 is applicable when the group
sizes of the dependent variable are equal; generate 4 outcomes (TN, TP,FN,FP).
Accuracy: no. of true positive and true negative divided by the total samples. In
assessing discriminant analysis, that is, measuring how well group membership is
predicted and developing a hit ratio(accuracy of +and - combined), which is the
percentage correctly classified. Chi-Square-Based Measure: This test provides a
comprehensive measure of predictive accuracy that is based not on the likelihood
value, but rather on the actual prediction of the dependent variable. The appropriate
use of this test requires an appropriate sample size. Hosmer and Lemeshow test is a
classification test of the statistical significance of the actual versus predicted
outcomes
Stage 5: Interpretation of the Results logistic regression model results in coefficients
for the independent variables much like regression coefficients and quite different
from the loadings of discriminant analysis. The Wald statistic provides the statistical
significance for each estimated coefficient so that hypothesis testing can occur just as
it does in multiple regression. Directionality of the Relationship A positive
relationship means an increase in the independent variable is associated with an
increase in the predicted probability, and vice versa. But the direction of the
relationship is reflected differently for the original and exponentiated logistic
coefficients. Original coefficient signs indicate the direction of the relationship.
Exponentiated coefficients are interpreted differently since they are the logarithms
of the original coefficients and do not have negative values. Thus, exponentiated
coefficients above 1.0 represent a positive relationship (=a relationship w/ no and
values less than 1.0 represent negative relationships. Magnitude of the Relationship
(to determine how much the odds will change given a one-unit-change in the
independent variable, the numeric value of the coefficient must be evaluated) The
magnitude of metric independent variables is interpreted differently for original and
exponentiated logistic coefficients:
Original logistic coefficients – are less useful since the reflect the change in the logit
(logged odds) value. Exponentiated coefficients : directly reflect the magnitude of
the change in the odds value. But their impact is multiplicative and a coefficient of
1.0 denotes no change (1.0 times the independent variable = no change).
Stage 6: Validation of the Results ensuring the external as well as internal validity of
the results. The most common approach for establishing external validity is the
assessment of hit ratios through either a separate sample (holdout sample) or
utilizing a procedure that repeatedly processes the estimation sample. External
validity is supported when the hit ratio of the selected approach exceeds the
comparison standards that represent the predictive accuracy expected by chance.
The goodness-of-fit for a logistic regression model can be assessed in two ways: (1)
using pseudo R2 values, similar to that found in multiple regressions, and (2)
examining predictive accuracy (i.e., the classification matrix in discriminant analysis)
Chapter 08: Step by step of Logistic Regression
1.Compute mean score(same like multiple regression)
2. Label or Divide Group Mean (i.e., 1 and 2) convert the dependent variable into
dichotomous variable by using K-Means Cluster method (KMC) (Analysis >> Classify
>> K-Means Cluster then After clicking on K-Means Cluster >> select and enter “var”
to “Variable” box >> click on “Save” >> click on “Cluster membership” box >> Click on
“Continue” Then, click on “Options” >> Click on “Initial cluster centers”, “ANOVA
tables”, and “Cluster information for each case” box >> click on “Continue” >> “OK.”
Then, go back to data view in SPSS, able to see K-Means >> change name of cluster
tht we made
Interpretation: Based on the result of K-Mean Cluster, the KMC will be separated
into 2 clusters (we consider 2 groups). The 1st group has 62 cases and the mean of
KMC of this group is 3.57. The 2nd group has 138 cases, the mean of KMC of this
group is 5.39. The 2 clusters are separated well. 3. How to produce logistic
regression results(Analyze >> Regression >> Binary Logistic Then, select Cluster
number case “dependent var” and enter it to “Dependent” panel and select
“independent var” mean score to “Covariates” panel 4. check and select the results:
As far as you got Chi Square is significant and at least a Wald test significant, so
you can conclude that the hypothesis is supported. #OddRatio in %=((Exp(B)) -
1.0)*100
Chi Squared test sig., with p<0.05 tests for significant differences between the
frequency distributions for two (or more) categorical variables (non-metric) in a
cross-tabulation table. Used to evaluate the reduction in the log likelihood value.
(>6.63)
Cox and Snell R2 (range from 0-0.75) OR Negelkerke R2 (range from 0-1) OR
“Pseudo” R square A value of overall model fit that can be calculated for logistic
regression; comparable to the R2 measure used in multiple regression. higher values
indicating greater model fit. No cut-off number of “pseudo” R-square
-2Log Likelihood approach 0 is more appropriate (100% ->likelihood=1,-2LL=0
Reflected in pseudo R square. No special criteria
The beta β: relationship between the independent variables and the dependent
variable, where the dependent variable is on the logit scale. No special criteria for
beta. Its value is depended on Wald tests.
The Wald number is used to determine statistical significance for each of the
independent variables (same role as t-value in linear regression). It should >1.96 at
p<0.05. HOWEVER, Wald method has been the subject of extensive criticism by
statisticians for exaggerating results. So, for getting a safe result, you should use
p<0.025 as suggested from some articles.
Hit-Ratio>62.5% (@ percentage correct). It is calculated as the number of objects in
the diagonal of the classification matrix divided by the total number of objects. Also
known as the percentage correctly classified.
SAMPLE INTERPRETATION
Chi-square and its significance level (sig.) identify the significance of overall model. In
this case, χ2(df=4) = 94.245, p < .001. The model explained a significant amount of
the variance in dependent variables (52.1% of Negelkerke R2) and correctly classify
78.5% of all cases. Competence Upgrading has significant impact on Technology
Innovation (beta=0.199, Wald=9.497 with p<0.001) Behavior Innovation also has
significant impact on Technology Innovation (beta=0.197, Wald=29.036 with p<0.001
CHAPTER 4 – Cluster Analysis Group objects based on characteristic they possess:
obj in the same cluster are similar w/in a cluster but different form obj in others.
Descriptive, a theoretical, non-inferential. Cluster will always create cluster
regardless any structure. The solution not generalizable (depends on cluster variate).
Cluster Variate: set of clustering var used to measure similarity. Concept: reduce
population to smaller no of homogeneous group. •It has been referred to Q analysis,
typology construct, numerical taxonomy. ROT :Cluster analysis is used for
•Taxonomy des: identify natural group w/n data• Data simplification: analyze group
of similar observation instead of all individual. •Relationship identification: simplified
structure from cluster analysis portrays relationships not revealed otherwise.
Conceptual and practical consideration: •only var that relate specifically to objective
are included irrelevant var cannot be excluded from analysis once it begins. •Var are
selected which characterize the individual being clustered •The essence of
approaches is the classification of data as suggested by natural grouping . 3 Basic
Question: 1. How do we measure similarity? Method of comparing observation on
the clustering var. Var w/n same cluster = stronger relation than between cluster.
Correlation between objects, measure of their proximity in two-dimensional space
such that the distance between observation indicates similarity. 2.How do we form
cluster? Group observation that are most similar into cluster.3. How many groups do
we form? Depends but few cluster> less homogeneity w/n cluster, larger
cluster>more homogeneity. Critisim: 1.subjectivity in selecting final decision: no
optimal solution final decision by researcher. 2.Judgment required of the researcher
in selecting characteristic to be used, the method of combining cluster,
interpretation of cluster solution make final solution unique to that researcher .
Stage1: Objective: •partition set of obj into 2 or< based on similarity. Key issue:
research question being address & var used to characterized obj in cluster process.
Selecting cluster Var: cluster var represent the sole means of measuring similarity
among obj. 2 issues in Var selection 1. Conceptual consideration: Var characterize the
objects being clustered. •relate specifically to the obj. 2. Practical considerations:
always use the BEST var.
Stage2:Research Design: Types of Var included: metric & non-metric, but not mixed.
Number of clustering var: can have impact as few as 20 var. “Curse of
dimensionality” when large number of Var analyzed. Relevancy of Clustering Var:
only include var with strongest conceptual support. Sample size: enough sample size
is needed to ensure representativeness of the population. Of particular interest is
the ability to detect small groups w/n the population. Min group sizes are based on
the relevance of each group.*Increase sample size may pose probs for hierarchy
clustering methods & required “hybrid” approach.* Detecting outlier: should be
removed if outlier present aberrant observation not population, insignificant
segment w/n population. Retain if represent under-sampling, poor representation of
relevant group in population. Defining and measuring Interobject similarity:
measure correspondence between objects to be clustered. Calculate across the
entire set of clustering var to allow grouping.3 methods •1. Association: non-metric
2. Correlation: less used, measure patterns not distance.3. Distance measure: most
often used with higher representing greater dissimilarity (distance between case).3.1
Euclidean(straight line) most common used 3.2 squared Euclidean: sum of square
distance.3.3 Mahalanobis(D2) var intercorrelations and weights each var equally.
Data Standardization: Rescaling data test to have common scale 2 Approach to
standardize: 1. Relative to other cases: most common is Z score 2. Relative to other
responses w/n an object. If groups are to be identified according to an individual’s
response style, then within-case or row-centering standardization is appropriate.
Stage 3:Assumption -1. Structure exists: assume that ”natural” structure of object
exists. 2. Representativeness of sample: obtained sample is truly representative of
population 3. Impact of multicollinearity: multicoll among subset of var is an implicit
weighting of cluster var. • potential remedies •reduce the var to equal number in
each set of correlated measure. •use distance measure that compensate for the
correlation •take proactive approach and include only cluster var that are not highly
correlated. Stage 4: Deriving cluster& assess overall fit-1.Hierarchical: all obj start at
separate then join tgt, each step form new cluster joining by 2 at a time and only
single cluster remain. 2.Non-hierarchical Approach: 2 types of Hierarchical
approach 1.Agglomerative:combine factor which has>1 observation, all observation
starts as individual and join tgt sequentially (•Single linkage: nearest in one cluster,
•complete linkage:farthest: max distance between observation in each cluster
•average: avg similarity of all individual in cluster, less affect by outlier •centroid
:distance btw cluster, less affected by outlier •Ward’s : total sum of square w/n
cluster easily distort by outlier) 2.Divisive: Breakdown from single cluster then
divided into smaller.(Pro: simplicity(tree like structure),speed, measure similarityto
address in many situations, Con: permanent combi: once join never separate Outlier
may appear, require Large sample) Non-hierarchical: 1.Determine number of cluster
to be extracted 2. Specify cluster seed: researcher specify 3.Assign each observation
to one of the seed based on similarity.(Pro: less sensitive to outlier, distance
measure, inclusion of irrelevant or inappropriate var & can easily analyze very large
data, Con: require knowledge of seed point, diff guarantee optimal solution, less
efficient, spherical and equally sized cluster ) Selecting btw hierar and non-hierar:
Hierarchical: •sample size is moderate 300-400 not exceed 1000, wide range of
alternative clustering solution. Non-Hierarchical: know the number of cluster and
initial seed points can be specified, concern about outlier since this method less
susceptible to outlier. Determine Number of cluster: Stopping rule: criteria used w/
hierarchical technique to identify potential cluster solution, Foundation principle: a
natural increase in heterogeneity comes from the reduction in number of cluster.
Deriving The Final Cluster Solution: No single objective procedure to determine the
correct number of clusters; rather the researcher can evaluate alternative cluster
solutions on two general types of stopping rules:1.Measures of heterogeneity change
:percentage changes in heterogeneity, when moving from k to k - 1
clusters .Candidates for a final cluster solution are a large increase in heterogeneity
by joining two clusters 2.Direct measures of heterogeneity :reflect the compactness
and separation of a specific cluster solution. These measures are compared across a
range of cluster solutions, with the cluster solution(s) exhibiting more compactness
and separation being preferred. Among the most prevalent measures are the CCC
(cubic clustering criterion), a statistical measure of cluster variation (pseudo F
statistic) or the internal validation index (Dunn's
EFA
When all criteria fit>> reliability test
IF run FA and see criteria doesn’t fit delete
that item out 1) communality – delete the
item w/ unmet criteria 2)component
matrix or factor loading 3)meaning related
Interpretation: item deleted bc of having
communalities of xxx, which is lower than
criteria, Cronbach’s alpha meet criteria =
item is reliable
How to name factor: 1) read questionnaire
item or 2) see which item gives the highest
factor loading>>name based on what is
that item abt.
Discriminant Analysis single, non-metric (categorical) dependent variable is
predicted by several metric independent variables. Examples of Dependent
Variables: Gender (Male vs. Female); Culture (USA vs. Outside USA) (Dependence
techniques) Logistic Regression A single nonmetric dependent variable is predicted
by several metric independent variables. This technique is similar to discriminant
analysis but relies on calculations more like regression (Dependence techniques)
Manova Several metric dependent variables are predicted by a set of nonmetric
(categorical) independent variables. Canonical Analysis Several metric dependent
variables are predicted by several metric independent variables. (Dependence
techniques) Conjoint Analysis is used to understand respondents’ preferences for
products and services. In doing this, it determines the importance of both: attributes
and levels of attributes based on a smaller subset of combinations of attributes and
levels. • Typical Applications: Soft Drinks, Candy Bars, Cereals, Beer, Apartment
Buildings/Condos, Solvents/Cleaning Fluids. (Dependence techniques) Structural
Equations Modeling (SEM) Estimates multiple, interrelated dependence
relationships based on two components: 1. Measurement Model 2. Structural Model
(Dependence techniques)
Overview: The “what” and “why” of factor analysis
Factor analysis is a method of data reduction. by seeking underlying unobservable
(latent) variables that are reflected in the observed variables (manifest variables).
Factor analysis is a technique that requires a large sample size (300,500,1000). Factor
analysis is based on the correlation matrix of the variables involved, and correlations
usually need a large sample size before they stabilize. minimum of 10 observations
per variable is necessary to avoid computational difficulties.
Kaiser-Meyer-Olkin Measure of Sampling Adequacy – This measure varies between 0
and 1, and values closer to 1 are better.Bartlett’s Test of Sphericity – This tests the
null hypothesis that the correlation matrix is an identity matrix. An identity matrix is
matrix in which all of the diagonal elements are 1 and all off diagonal elements are 0.
You want to reject this null hypothesis. **should pass KMO & batlett bf doing FA
Communalities – This is the proportion of each variable’s variance that can be
explained by the factors t is also noted as h2 and can be defined as the sum of
squared factor loadings for the variables.Initial Eigenvalues – Eigenvalues are the
variances of the factors. Total – This column contains the eigenvalues. The first
factor will always account for the most variance. % of Variance – This column
contains the percent of total variance accounted for by each factor.Extraction Sums
of Squared Loadings – The number of rows in this panel of the table correspond to
the number of factors retained.Rotation Sums of Squared Loadings – The values in
this panel of the table represent the distribution of the variance after the varimax
rotation. Varimax rotation tries to maximize the variance of each of the factors.
Factor Matrix – This table contains the unrotated factor loadings, which are the
correlations between the variable and the factor. Because these are correlations,
possible values range from -1 to +1. Logistic Regression Overall Percentage – This
gives the percent of cases for which the dependent variables was correctly predicted
given the model. B – This is the coefficient for the constant (also called the
“intercept”) in the null model.
S.E. – This is the standard error around the coefficient for the constant.
Wald and Sig. – This is the Wald chi-square test that tests the null hypothesis that
the constant equals 0. hypothesis is rejected because the p-value (listed in the
column called “Sig.”) is smaller than the critical p-value of .05 (or .01). Hence, we
conclude that the constant is not 0. Usually, this finding is not of interest to
researchers.
df – This is the degrees of freedom for the Wald chi-square test.
Exp(B) – This is the exponentiation of the B coefficient, which is an odds ratio. This
value is given by default because odds ratios can be easier to interpret than the
coefficient, which is in log-odds units. Score and Sig. – This is a Score test that is used
to predict whether or not an independent variable would be significant in the model.
Why use multivariate analysis? multivariate analysis is a tool to find patterns and
relationships between several variables simultaneously. It lets us predict the effect a
change in one variable will have on other variables. ... This gives multivariate analysis
a decisive advantage over other forms of analysis. 2.Why is knowledge of
measurement scales important in using multivariate analysis? Since the type of
measurement should be defined by the researcher (even it will be
nonmetric/qualitative or metric/quantitative) for each variable, in MV analysis
defining data has great effect what the data present. For computers, the values are
just numbers; in nominal scales, 1 can be assigned to male, and 0 to female. But the
data present gender and researcher determine how it can be analyzed. Since
nonmetric data (ordinal and nominal scales) used as independent variable in most
MV techniques. Knowledge of measurement scales important in order not to use
wrong scale for the variables. As exemplified above, 0 and 1 are just nonmetric data
to present gender; not having any numerical meaning like female are less than male,
or we cannot calculate any mean value 3.What is the difference between
component analysis and common factor analysis? In factor analysis, the original
variables are defined as linear combinations of the factors the goal in factor analysis
is to explain the covariance’s or correlations between the variables. Use principal
components analysis to reduce the data into a smaller number of components.
6.When should multiple regression be used? It is used when we want to predict the
value of a variable based on the value of two or more other variables. The variable
we want to predict is called the dependent variable (or sometimes, the outcome,
target or criterion variable). 7.When should a linear regression be used? Simple
linear regression is appropriate when the following conditions are satisfied. The
dependent variable Y has a linear relationship to the independent variable X. To
check this, make sure that the XY scatterplot is linear and that the residual plot
shows a random pattern. 8.How do you use regression coefficients? when the
regression line is linear (y = ax + b) the regression coefficient is the constant (a) that
represents the rate of change of one variable (y) as a function of changes in the other
(x); it is the slope of the regression line. 9.Should we use factors scores or
summated ratings in follow up analysis? It’s hard to answer without context.
Usually summated rating is use when the items are designed as summated scale but
it is very rare. Usually for latent constructs, in my field, they use factor score.
However, in some fields, like economics, sometimes I see summated rating. I don’t
really know why they use it. Maybe it is a way to put weight on the items that they
think are important. Summated rating is when the score of all items add up into a
score of the construct right? So if the items are measure on different scales. For
example an item is 1 to 5, another is 1 to 9. When they add up, the item with 1 to 9
scale will have higher weight
Three converging trend:
1)Riseof big data:
volume,variety,velocity(faster),veracity(accuracy),variability :impact on organization
decision and analyst. 2)Statistical: structure (small to large dataset, theory based):
Explanation what happen now, specific model. Inference through statistic test vs
data mining (unstructured, very large dataset):’Predictive accuracy’ based on analysis
past, algorithms (neutral network, decision tree, support vector machine) 3)Casual
Difference: strong statement of cause-effect in nonexperimental situation/drawing
conclusion about casual connection based on condition of occurrence.