School of Business

First Edition.
Published by the Center for Teaching and Learning, Northcentral University, 2020

Contributors:
John Bennett, Mary Dereshiwsky, Robert Dodd, David Fogarty, John Frame, Raymie
Grundhoefer, Larry Hughes, Sharon Kimmel, Vicki Lindsay, Edward Maggio, Gordon
McClung, NCU Library Team, Susan Petroshius, Lonnie K. Stevans, Gergana Velkova,
Steve Ziemba

In addition to the collaborative process that engendered this guide, it was also informed by
the quantitative methods and statistics courses in the School of Business.

For comments or suggestions for the next edition, please contact John Frame: jframe@ncu.edu
2 Back to Table of Contents
TABLE OF CONTENTS
Foreword
Introduction
Research Ethics and the IRB
Research Questions
Four Main Designs
Population and Sample
Sampling Method, Sample
Design, and Sample Size
Surveys and Questionnaire Design
Pilot Study
Datasets
Analyzing Secondary Data
Observational Research
Multivariate vs. Univariate Analysis
Measurement of Variables
Descriptive Statistics and Exploratory Data Analysis (EDA)
Inferential Statistics
Alpha Level (level of significance, or p-value)
Hypotheses
Hypothesis Diagrams
Hypothesis Testing
T-Test
Analysis of Variance (ANOVA)
ANOVA Examples
Correlation
Regression Analysis
Factor Analysis
Power (Statistical Power)
Power Analysis
Measuring Validity and Reliability
Internal/External Validity
Selection of Parametric vs. Nonparametric Techniques
Presentation of Statistical Results and Explaining Quantitative Findings in a Narrative Report
Back to Table of Contents 3
Foreword
Dear School of Business Community,

Welcome to the Best Practice Guide for Quantitative Research Design and Methods in Dissertations!

With well over 700 doctoral students in the School of Business working on their dissertation this year, this
guide serves as an important resource in helping us shape and implement quality doctoral-level research.
Its primary purpose is to offer direction on quantitative research in School of Business dissertations, serving
students as they craft and implement their research plans, and serving faculty as they mentor students and
evaluate research design and methods in dissertations.

We encourage you to explore this guide. It is filled with details on important topics that will help ensure
quality and consistency in quantitative research in the School of Business.

Thank you to the faculty and staff of the School of Business and wider NCU community that worked to cre-
ate this guide. It is a great contribution to our School, and each of these individuals played an important
role in its development.

- School of Business Leadership Team

1 Back to Table of Contents

Introduction
As an accredited university, NCU aims to have
robust expectations and standards for dissertations
produced by its students. This guide, developed
collaboratively by NCU School of Business (SB)
faculty, aims to provide guidance on best practice
in quantitative research design and methods for SB
dissertations.
While this guide can serve as a refresher to those
less familiar with quantitative methods, it will also
help ensure good practice and rigor across com-
mittees and students. To that end, this document is
a guide to help students, as well as faculty, when
judging the merits of student dissertation prospec-
tuses, proposals, and manuscripts. Students should
be familiar with the best practices in this guide and
apply them to their dissertation.
Additional supports related to quantitative research
design and methods are available in the NCU
Dissertation Center (including several webinars),
Back to Table of Contents
and statistics experts are available for 1-1 coaching
through the NCU Academic Success Center.
Importantly, before students plan to embark on
a quantitative research design, they need to be
comfortable with quantitative analysis, including
data analysis computer software, such as SPSS. If
students are not comfortable with their level of skill
in quantitative analysis, it is recommended that they
consider how qualitative methods could be used to
explore their research interests. Students interested
in qualitative methods should consult the SB’s Best
Practice Guide for Qualitative Research Design and
Methods in Dissertations, published in 2019, and
available in the Dissertation Center.
Research Ethics and the IRB
Research involving human participants involves
certain ethical responsibilities on the part of the
student and dissertation Chair. These responsibili-
ties are an important part of the overall educational
experience for the student, in that they learn that
2
obtaining data and other information from partici-
pants needs to be done in a manner that respects
the rights of the participant and the wishes of other
organizations that might become involved in the
research. As part of the research ethics review
process, the Institutional Review Board (IRB) at
NCU serves as a resource to provide guidance to
students and faculty to ensure the ethical principles
of Respect for Persons, Beneficence, and Justice
are incorporated into the research design. The IRB
review process is as much a part of students’ doc-
toral education as any other part of the dissertation
process. The intention is not only to ensure studies
are conducted ethically, but also that students un-
derstand the importance of ethics in research and
how to design and conduct research that is consis-
tent with federal regulations.
It is important to keep in mind that recruitment
and data collection can only occur after receiving
NCU IRB approval. The IRB process starts with
IRB Manager, an online system that facilitates the
submission and management of studies for review.
Students should plan ahead and be sure to leave
time for the IRB review to take place, as it may take
up to 15 business days after submission of the IRB
application to receive notification of the IRB’s de-
termination. Also, it is possible that the application
will not be approved the first time through, due to
the need for additional information or clarification.
These factors need to be kept in mind when con-
structing the study timeline. Additional variables
that can impact the timeline include: securing site
permission, site IRB approval (if applicable), in-
ternational research, research involving sensitive
topics or vulnerable populations, research in one’s
place of employment, research involving the De-
partment of Defense or Veterans’ Affairs, and the
development of appropriate recruitment materi-
als and an informed consent form, or, for studies
involving minors, child assent and parental consent
3 Back to Table of Contents
forms. These and other items need to be submitted
as part of the IRB application and can significantly
delay the review process if not present. For exam-
ple, the inclusion of an informed consent form that
does not use (or where the researcher has altered)
the NCU informed consent form template will result
in the application being returned to the student. As
indicated in the Student-Chair Engagement section
of this guide, it is important for students to work
closely with their Chair in the lead up to the IRB
approval process.
A variety of resources are available for students
and faculty as they navigate the IRB process. Guid-
ance materials are available directly within IRB
Manager and are easily accessed from within the
application. Resources are also available via the
NCU Dissertation Center. Finally, when questions
come up, the IRB can be contacted at irb@ncu.edu.
When doing so, be sure to include the name of the
student in the subject line.
Please review the IRB website for further informa-
tion and resources: https://ncu.libguides.com/irb/
home
Research Questions
Research Questions outline the problem to be inves-
tigated in a study, stated in the form of a question.
Research questions that describe data are called
descriptive. Descriptive research questions typically
ask “How,” or “What.” (As explained elsewhere in
this guide, descriptive research design and meth-
ods that are solely descriptive are not sufficient for
a doctoral-level dissertation. Thus, rigorous research
questions that go beyond descriptive research need
to be included in a dissertation, as explored below.)
Research questions that compare one or more
groups are called comparative. Comparative re-
search questions typically ask, “What is/are the Four Main Designs
difference(s)” (for example, what are the differences
There are four main designs that can be used with
between X and Y?).
a quantitative methodology: experimental, quasi-ex-
perimental, correlational, and descriptive. Students
Research questions that examine relationships are
need to look at their research study to figure out
called correlational, or relationship, questions.
which design will be most appropriate to answer
More specifically, these questions typically ask,
their research questions (but, as indicated else-
“What is the strength and direction of a linear rela-
where in this guide, a descriptive design is insuffi-
tionship between the two variables in question.”
cient for a doctoral-level dissertation). The Methods
Map Online Tool (see link below) is a fun and inter-
Research questions that consider predictions are
esting interactive website that provides an overview
called predictive research questions. These types of
of a number of methodological procedures.
questions typically ask, “To what extent does X pre-
dict Y?” Predictive analysis may have one or many
Before researchers can begin to think about their
independent variable(s), which may be expressed
research design, it is essential for them to begin at
as predictive variables. The dependent variable
the foundation of the business research process:
may be expressed as the outcome variable.
defining the problem. It is extremely important to
define the problem carefully because this will deter-
Each research question, with the exception of de-
mine the purpose of the research and the research
scriptive research questions, contains a minimum of
design.
two hypotheses: the null hypothesis and the alterna-
tive hypothesis.
A brief introduction to the four main research de-
signs are as follows:
Students are encouraged to get 1-1 coaching at the
NCU Academic Success Center on their research
Experimental Research
questions and/or sign up for live group sessions of-
Experimentation is conducted in order to test a
fered weekly by the NCU Academic Success Center.
causal hypothesis (that is, if a researcher wants to
More in formation can be found at: https://vac.ncu.
determine if an independent variable (X) is the sole
edu/resources-for-statistics/
cause of any change in the dependent variable
(Y)). In an experiment, a researcher manipulates
the independent variable and measures its impact
References and/or Suggested Reading:
on the dependent variable while, at the same
Cramer, D., & Howitt, D. (2004). The SAGE dictio- time, controlling for all other variables that may
nary of statistics. London: SAGE Publications, Ltd. have influenced the dependent variable. These are
http://dissertation.laerd.com/how-to-structure-quanti- referred to as extraneous or potentially confounding
tative-research-questions-p2.php variables. An experiment is internally valid if it can
be shown that the independent variable is the sole
https://www.youtube.com/watch?v=Wd2a-OSPEL8
cause of any change in the dependent variable. In
order to do so, three pieces of evidence are need-
ed: (1) for X to be a cause of Y, X must precede Y

Back to Table of Contents 4

in time; (2) X and Y must vary together; (3) for X to
be a cause of Y, other possible causes of Y (alterna-
tive explanations) must be eliminated. In contrast to
internal validity, external validity refers to whether
the results of the experiment can be generalized to
other populations, settings, etc. For instance, with
respect to generalizing to the population, there
would be better external validity if the sample was
selected randomly from the population. This would
have no impact on internal validity, however. Note
that there is often a tradeoff between internal and
external validity and the experimental setting (a lab
vs. field experiment). A laboratory experiment is
an artificial setting that allows the researcher better
control over extraneous/potentially confounding
variables. However, the artificiality of a lab ex-
periment tends to lessen the external validity since
a researcher will want to be able to generalize
to a more realistic setting. Essentially, laboratory
vs. field experiments represent opposite ends of a
continuum having to do with the artificiality of the
setting.
Quasi-Experimental Research
Quasi-experimental designs are used when it is not
viable to randomly assign participants to treatment
groups. In many real-life social situations, groups
of interest may be naturally occurring or pre-ex-
isting. There may also be ethical reasons when
randomization to groups is not practical. Manipu-
lation of an independent variable (also referred to
as a treatment variable), comparison groups (also
referred to as experimental units), and outcomes
measures are present in quasi-experiment designs.
Unequivalent groups are also present because of
the inability to randomly assign participants to
comparison groups. Because of the inability or
decision to not use random assignment to groups,
it is difficult to compare and infer treatment-caused
changes. Quasi-experiment designs are used by
researchers in these situations. Common non-time
5 Back to Table of Contents
series quasi-experiment designs include Cohort
Designs, Counterbalanced Design, Non-equivalent
Control Group Design, Regression-discontinuity
Design, Separate-Sample Pretest-Posttest Designs,
and Separate-Sample Pretest-posttest Control Group
design.
Correlational Research
If the research questions focus on a relationship
between multiple variables, a correlational design
will likely be used. Research is correlational when
at least two, and often more, variables/conditions
are observed and measured and the extent of the
relationship is estimated based on tools such as the
Pearson Product Moment Correlation, the Spear-
man Rank Correlation Coefficient, or even Kendall’s
Rank Correlation Coefficient. In fact, correlational
research is often descriptive in that the associations
are reported to the reader, often in the same table,
as the means and standard deviations. In a pub-
lished research study, a reader can use correlations
and the other descriptive statistics to get a sense of
the data before reading about t-tests, ANOVA, or
multiple regression, whichever the author(s) used
in their analysis. This causal inference is distinct
from prediction or forecasting and a common error
made by students and novice researchers (Cook &
Stanley, 1979). A caution in correlational research
is that, as the famous phrase goes, “correlation
does not imply causality.” There are no “depen-
dent” or “independent” variables in correlational
research – we’re simply comparing the variables on
the basis of association and cannot assert that the
effect of one causes another.
Descriptive Research
Descriptive research (see the section, “Descriptive
Statistics and Exploratory Data Analysis (EDA)”
later in this guide) describes individuals in a study
that was typically conducted in one of three ways:
(a) observational – viewing and recording partici-
pants (See “Observational Research” in this guide); the results of statistical research. These are usually,
(b) case study – in-depth study of an individual but not always, people.
or group of individuals; and (c) survey – a brief
interview or discussion with an individual about Usually, it is not practical to do a census of an
a specific topic. Descriptive research designs are entire population in a single research study, due to
common in fields related to behavioral and social time and cost factors. For this reason, it is neces-
sciences to observe phenomenon such as: natural sary to select a sample from that population.
behavior, consumer habits, individual morality,
and ethical climate. The observations of the subject
should occur in an unchanged natural environment.
The weaknesses of descriptive research designs
are that observational studies are not repeatable
and not replicable. Descriptive research designs
are often designed in a manner which allows it to
be a precursor to quantitative research. Descriptive
research does not involve statistical testing, thus it
is considered to lack reliability, validity, and scien-
tific rigor. As discussed elsewher e in this guide, a
descriptive research design alone is insufficient for
a doctoral-level dissertation at NCU.

References and/or Suggested Reading:

Cook, T.D. & Stanley, D.T. (1979). Quasi-experimen-

tation: Design and analysis issues for field settings.
Boston, MA: Houghton Mifflin.

Research Methods Knowledge Base website: https://

socialresearchmethods.net/kb/quasiexp.php

The SAGE handbook of social research meth-
ods (2008). London, United Kingdom: SAGE
Publications, Ltd. https://doi-org.proxy1.ncu.
edu/10.4135/9781446212165
Sage Methods Map Online Tool: http://methods.
sagepub.com.proxy1.ncu.edu/methods-map
Population and Sample
The population represents the totality of units under
study, or to whom we wish to generalize or project
Back to Table of Contents
A sample in a dissertation needs to be a substantial
number (see “Power Analysis” in this guide), and
should be determined based on best practices in
quantitative research. Students should be aware
that quantitative research demands a suitable
amount of data, and that the response rate from
samples (such as the response rate for surveys),
will typically be very low. Thus, a large number of
persons will need to be surveyed in order to obtain
an adequate amount of data.
After sampling, it is possible to generalize the
sample results to the population from which it was
6
selected. Here are some commonly applied ways to fraction. The items in the list should be scrambled
select a sample in quantitative research: in random order before beginning to cycle through
that list.
Simple random sample:
Every element of the target population has an equal Sampling Method, Sample
chance of being selected for the sample. This is es-
Design, and Sample Size
pecially valuable when doing experimental studies.
In most cases, a researcher will not contact every-
Stratified sample: one in the population (a census), but rather take a
subset of the population, a sample. While the selec-
In this sampling method, it is recognized that there
tion of a sample can involve a non-random (non-
is not one overall homogeneous population, but,
probability) procedure, in most quantitative studies
instead, subpopulations where the subgroups differ
researchers strive to use a probability procedure
from one another. For example, a researcher may
in which every unit in the population has a known
want to see if there is a significant difference in an
chance of being included in the sample. There are
average number of units of Product X purchased by
a number of alternative ways to generate a ran-
men and women. The researcher would subdivide,
dom sample that may vary over time, with respect
or ‘stratify,’ the overall population into men vs.
to cost, and the amount of information needed to
women (strata or subgroups), and randomly select
draw the sample. The researcher often needs a list
a sample from each gender to ensure it is ade-
of the population in order to select the sampling
quately represented in the overall sample.
elements (a sampling frame) and has to determine
Cluster sample: the size of the sample as well.

In this sampling method, the researcher randomly

Students should be aware that, because response
draws intact groups, (‘clusters’) instead of individ-
rates to surveys are usually very low, a very large
uals, for the study. For instance, a cluster could be
number of surveys will need to be sent out in or-
an entire division or department of an organiza-
der to obtain the sample size expected (see “Power
tion. The researcher then includes all sampling units
Analysis” in this guide). Students should discuss the
(e.g., persons, employees) in that randomly drawn
number of surveys they will need to send out with
cluster in our study. The idea is to simulate the
their Chair, and should also obtain guidance from
randomness of a true random sample, but without
the statistics coaches at the NCU Academic Success
having to select individuals one by one.
Center.
Systematic sample:
In addition to sampling, a critical issue that is
In this sampling method, the researcher lists the
unique to quantitative studies is the measurement of
elements of the target population, makes a random
variables. A variable is what a researcher calls the
start in the list (‘sampling frame’), and then sys-
construct that is identified in the research question
tematically cycles through the list in a predictable
and hypotheses. Examples would include gender,
pattern (e.g., every third sampling unit; every fifth
job satisfaction, behavioral intention, attitudes, etc.
sampling unit) to select subjects. This pattern of
Often these concepts are abstract and not directly
cycling through the list is known as the sampling
related to physical reality, such as a person’s
7 Back to Table of Contents
intelligence. Before a concept can be measured it
needs to be defined, both conceptually and oper-
ationally. An operational definition specifies the
operations necessary to measure the construct. For
instance, intelligence may be conceptually defined
as the ability to think abstractly. It may be opera-
tionalized as the score on an IQ test. Measurement
is much more complex than emerging scholars
believe. This is in part due to the complexity of op-
erationalizing an abstract object that is not related
to physical reality. In addition, many constructs are
multi-dimensional. The good news is that scholars
build on one another’s work. And this includes
the measurement of particular constructs. While
researchers may not agree on how a construct is
to be operationalized, this information is typically
shared in the publication of the research. In fact,
there are entire books and websites that are devot-
ed to providing measurement scales. This is another
reason why it is so important to know the literature
in the discipline and the specific topic of interest.
Back to Table of Contents
Sample Design
The two major decisions in designing sampling
plans are the sampling method and the sample
size. Given the desire to generalize the results of a
quantitative study, researchers will use a probability
procedure, if at all possible. This includes a simple
random sample, systematic sample, stratified sam-
ple, cluster sampling (See “Population and Sample”
in this guide). A common form of cluster sampling
is area sampling, where the clusters are the geo-
graphical area. The decision of what method is
used is dependent on a number of factors including
the cost, information, and knowledge of the popu-
lation, accuracy and the time required. The factors
that have to be specified to determine the appro-
priate size is the variability in the population, the
degree of acceptable error, and the confidence in-
terval. Note that it is not the size of the population
that is important but the degree of heterogeneity
of the population. While a researcher can deter-
mine the necessary sample size statistically, this
8
may have to be modified due to other factors. For instance, if a survey was being conducted to obtain the
desired sample size given the level of precision and confidence desired, the initial sample may have to be
larger. This may be due to the completion rate (the number of selected respondents who actually complete
the interview or questionnaire, which, as stated above, is typically very low), as well as the incidence rate
(the percentage of people eligible for participation) in the population. If a study is not designed adequate-
ly, then it may be largely a waste of time. A researcher must have a large enough sample to assure that
all statistical assumptions are met. At the same time, cost, and the ability to collect the desired number of
sample elements, have to be considered.

Sample Size
There are four factors involved in calculating sample size:

Statistical test: the sample size is partly a function of the statistical test used. Some
tests (e.g., Chi-squared) require larger samples to detect a difference than others
(e.g., ANCOVA).

Expected/estimated effect size: the effect size is potency of the strength of the rela-
tionship being investigated. In the language of statistics, an effect size is a difference
between the mean scores of two groups divided by the pooled standard deviation.
This is called ‘Cohen’s d’. A researcher will calculate an effect size as part of the
analysis of the data in order to determine that something meaningful has been found
(not merely statistically significant). However, in advance of doing a study, a
researcher must estimate the effect size in the study.

Alpha: the alpha level is the probability of a Type I error—of rejecting the null
α hypothesis when it is true. By convention, this is set at . It is best to use the literature,
as well as good judgment, to justify an alpha level that makes sense for a study. This
justification will involve looking at the danger of a Type I error versus the cost in
resources of avoiding it. Given this, the most common used α levels are .01, .05,
and .10.

Beta: the beta level is the probability of a Type II error of accepting the null
β
hypothesis when it is false. In other words, of failing to detect a difference when
one exists. As with alpha, a researcher sets beta based upon a judgment. The
convention is .2, which yields a power of .8 (1– β) acceptable level (see “Power
Analysis” in this guide).

9 Back to Table of Contents

Surveys and Questionnaire Design
The term ‘questionnaire’ is often confused with
‘survey,’ but they are actually quite different. A
questionnaire is a measuring instrument used in
conjunction with the survey. Basically, it consists of
a list of questions used to gather information from
respondents.
A survey is a research method involving communi-
cation with respondents. In quantitative data, the
asking of questions in mass form, such as with the
use of questionnaires, by phone or interview, is
called a survey. A survey is the distribution of the
questions and the creation of data.
If there is a problem remembering the difference
between a questionnaire and a survey, think of a
survey as the process of disseminating, collecting,
and computer entry of the research items. Surveys
can be on paper, telephone, face-to-face, or web-
based.
A questionnaire is used with all surveys. In a tele-
phone or personal interview, the interviewer is
using the questionnaire to ask the questions and
record responses. In this way, all respondents are
asked the same questions.
The design of questionnaires is much more complex
than one would think. Many researchers believe it
to be more art than science. A researcher needs
to avoid leading or double-barreled questions, as
explored further below. In addition, all respondents
should be given the same questions in the same
order.
In terms of facilitating a survey, an internet survey,
while low cost and quick, lacks control of the sam-
ple, and tends to have low response rates. A re-
searcher needs to carefully consider the objectives
Back to Table of Contents
of the study, the questions that need to be asked,
and the target respondents, in addition to the pros
and cons of the alternatives. A combination of
methods is also possible. For instance, self-admin-
istered questionnaires could be hand-delivered to
encourage participation, but left with envelopes to
be returned via mail.
Once data is collected, it is too late to make chang-
es, so it is critical that sufficient time and effort is
placed in the development of a questionnaire.
This involves careful articulation of conceptual and
operational definitions, as well as the measurement
scales used in consideration of the intended anal-
ysis. In fact, a researcher should have a plan for
analysis before any data is collected.
Thorough communication between the student and
Chair is imperative when designing a question-
naire, as well as during data collection.
Types of Questions
What are good questions, and what are bad
questions? Developing an instrument is difficult, and
a researcher should consider if another research-
er’s questionnaire can be used. Questions must be
worded a certain way to make ensure consistency
between the sample and the target population, and
that the items truly measure what they are supposed
to measure. A big problem that could happen if
the questions in the questionnaire are not valid,
nor reliable, is the wrong questions being asked
and, subsequently, the responses not relating to the
research questions. If this happens, the question-
naire basically did not measure what it intended
to measure and, thus, the data collected will not
answer the research questions. Therefore, the items
on the questionnaire must ask exactly what is meant
10
to be asked. A student must communicate well with
his/her dissertation Chair about this issue.
There are many methodological textbooks that state
numerous ways to form a suitable questionnaire.
However, each questionnaire changes with each
topic. Therefore, it is best when one learns through
repetition. The work referenced below by Krosnick
& Presser (2009) is highly encouraged, as they
discuss a number of ideas for writing the perfect
questionnaire. Some of these are:
* The questions need to be at a basic educational
level. Questions must not use colloquialisms, jar-
gon, or slang that is not familiar to the participants.
The simplest wording is the best wording for a
question. Shorter questions are better.
* The wording of each question should be exclusive
and exhaustive. This means that if it is a closed-end-
ed question, all of the possible answers must
appear in the answers. In addition, the answers
should not overlap each other. For example, if there
is a question that asks someone’s age, and the
X
participants are from 18 through 90, the student
should not offer these as answers:
a. 18-29
b. 29-39
c. 39-49
d. 49-79
If questions such as this appear as the only an-
swers, they are not exclusive or exhaustive. If
someone was age 29, what answer would that per-
son pick? If the person was age 82, what answer
would that person pick?
* Do not ask leading or loaded questions. A lead-
ing question contains non-neutral wording. It will
suggest that something is good or bad. It could also
11 Back to Table of Contents
lead a person toward an answer. An example is:
Do you like your Honda because of the comfortable
seating?
* Give respondents a way out of answering a
question. If the respondent does not want to answer
a question, what should the respondent do? Unless
a question must have a “yes” or “no” response, a
researcher should allow a respondent a way out,
for example, by offering the option “I don’t know”
or “I don’t wish to answer.” If not, the respondent
may stop answering the questionnaire.
* Don’t ask “double-barreled” questions. These
questions ask two questions instead of one. An
example of this would be: When was the last time
that you updated your computer and your printer?
Finally, remember that actual behavior cannot be
measured via a survey; a survey only measures
reports of behavior.
References and/or Suggested Reading:
Bryman, A. (2006). Integrating quantitative and
qualitative research: How is it done? Qualitative
Research, 6(1), 97-113.
Heale, R. & Twycross, A., (2015). Validity and reli-
ability in quantitative studies. Evidence Based Nurs-
ing, 18(3), 66-67.
Krosnick, J.A., & Presser, S. (2009). Question and
questionnaire design. In J.D. Wright & P.V. Marsden
(Eds.) Handbook of Survey Research (2nd Ed.). San
Diego, CA: Elsevier. Retrieved from: https://web.
stanford.edu/dept/communication/faculty/krosnick/
docs/2009/2009_handbook_krosnick.pdf
SurveyMonkey (2019). 5 common survey question
mistakes that’ll ruin your data. Retrieved from Avoid
Bad Survey Questions, Loaded Questions, Leading
Questions, SurveyMonkey: https://www.surveymon-
key.com/mp/5-common-survey-mistakes-ruin-your- References and/or Suggested Reading:
data/ NCU IRB (2019). Pilot studies and field tests. Avail-
Avoiding poor survey questions. Available able at: https://commons.ncu.edu/sites/default/
files/entity/paragraph/2019/Pilot%20Studies%20
at: https://www.google.com/url?sa=t&rct=-
j&q=&esrc=s&source=web&cd=1&ved=2a- and%20Field%20Tests%2004092019.pdf
hUKEwjg8PiD44vkAhUC7aYKHcIYBWcQFjAAe-
gQIABAC&url=http%3A%2F%2Ffaculty.nps.
edu%2Fmjdixon%2Fresources%2FStats-Class%2F-
Bad-Questions-Lecture-Examples.doc&usg=AOv-
Datasets
Vaw3YPdVRsW1WuTxoRi1jA8pl A dataset (also spelled ‘data set’) is a collection
of raw statistics and information generated by a
research study. Datasets produced by government
Pilot Study agencies or nonprofit organizations can usually be
downloaded free of charge. However, some non-
A pilot study is a preliminary small-scale study. Its
profit organizations may charge a fee for access
purpose is to test certain aspects of what will be
to their datasets, or restrict access. Datasets devel-
the main research study. For example, a newly
oped by for-profit companies are often available for
developed survey may undergo testing through a
a fee.
pilot study for refinement purposes. This is just one
example, as a pilot study can be applied in many
Most datasets can be located by identifying the
different situations. Pilot studies are useful for de-
agency or organization that focuses on a specific
termining the best research methodology to use,
research area of interest. For example, if one is
troubleshooting of a research instrument, collecting
interested in learning about public opinion on so-
preliminary data for a grant application, or for
cial issues, Pew Research Center would be a good
determining if a research study is even feasible.
place to look. For data about population, the U.S.
government’s Population Estimates Program from
A pilot study is still research. If it is determined
American Factfinder would be a good source.
that a pilot study is needed, it will need to under-
go the same processes for approval as any other
An “open data” philosophy is becoming more
research study. This includes going through the IRB.
common among governments and organizations
As a result, the inclusion of a pilot study in one’s
around the world, with the belief that data should
research will add time and this added time can be
be freely accessible. Open data efforts have been
substantial. Generally speaking, a pilot study will
led by both the government and non-governmental
be discouraged for a doctoral dissertation because
organizations, such as the Open Knowledge Foun-
of the time involvement. If one is using a preexist-
dation. Learn more by exploring The Open Data
ing survey or other research instrument which has
Handbook.
already been vetted and accepted, a pilot study is
not necessary. However, a pilot study may be need-
One factor to consider when utilizing a dataset that
ed in cases where the instrument is being devel-
is not publicly available is the presence of confiden-
oped for the first time. In such cases, determination
tial information in the dataset. For example, patient
of validity and reliability may require a pilot study.
disease registries are an increasingly common way
to conduct medical research by constructing data-
Back to Table of Contents 12
sets of patients with a common diagnosis, to assess
disease progress and the best treatment over time.
Such datasets would often include protected health
information, requiring additional safeguards for
their use.
When submitting a study that involves the use of a
dataset to the IRB, be prepared to indicate whether
it is publicly available or if permission is needed to
access it. If permission is needed, documentation
of having received permission needs to be part
of the IRB application. If protected or confidential
information is present in the dataset, a description
of how that information will be safeguarded is also
required. See “Analyzing Secondary Data” in this
guide for further information about datasets.
Some links to business datasets include:
• Damodaran Online: Corporate Finance and Val-
uation - NYU, Stern School of Business, Dr. Aswath
Damodaran
• International Monetary Fund Data & Statistics -
The IMF publishes a range of time series data on
IMF lending, exchange rates and other economic
and financial indicators.
• IMF DataMapper
• IMF Fiscal Rules Dataset (1985-2013)
• Mergent Online – An NCU Library database pro-
viding detailed financial records for company re-
search, including up to 15 years of historical data.
• National Longitudinal Surveys - Bureau of Labor
Statistics
• Organization for Economic Co-Operation and
Development Data
• Quandl - “Time-series” numerical only data for
economics, finance, markets & energy; Features
13 Back to Table of Contents
step-by-step wizard for finding and compiling data.
• SAGE Edge Datasets – Click on Links to Business
Datasets to download a Word file containing links
to business datasets available online.
• Statistical Abstract of the United States (2012):
Banking, Finance, & Insurance
• Statistical Abstract of the United States (2012):
Business Enterprise
• Surveys of Consumers - Thomson Reuters & Uni-
versity of Michigan
• U. S. Bureau of Economic Data
For more information on datasets, please see the
NCU Library’s Datasets LibGuide.
Analyzing Secondary Data
Secondary data is a term that relates to data that
was collected by someone else. Thus, data in a
dataset is secondary data. It is important to en-
sure the accuracy of secondary data. Poor data
will result in a poor study. With secondary data,
the manner in which the data was collected, and
consequently its quality, are beyond one’s control.
Therefore, it is important to carefully review the
manner in which a dataset was constructed. Some
factors to consider are:
• Purpose for which the data was originally collected
• Specific methods used in the data collection
• Population the data was collected from and the
validity of the sample used
• Credibility of the individual or organization who
collected the data
• Limits of the dataset
Another factor to consider is how the data was
categorized or coded in the dataset. This may
influence how data analysis will take place. For
example, the data may have been modified in
some manner, or the full range of data needed for
a study may be spread across different categories
of the dataset. Measurement error in the dataset,
whether or not that bias was intentional (i.e., a
dataset put out by a political party) may also be
present, and needs to be considered (see “Data-
sets” in this guide for additional information about
datasets and secondary data).
Descriptive statistics (See “Descriptive Statistics and
Exploratory Data Analysis (EDA)” in this guide), as
the term suggests, will involve the summarization of
data. For example, this may include the range of
data, number of data points, mean, median, and
mode, standard deviation, and 95% confidence
interval. As explained in the “Descriptive Statistics
and Exploratory Data Analysis (EDA)” section of
this guide, this type of analysis, while important, is
not sufficient for a doctoral-level dissertation.
Inferential statistics involve subjecting the data to
statistical tests, such as for significance. A common
inferential test involves the detection of a statistical-
ly significant correlation between two sets of data
from a dataset. Other types of statistical techniques
that are often applied to datasets include analysis
of variance, regression analysis, logistic regression,
etc. The type of statistical test applied depends on
the research question and the nature of the study
(see “Inferential Statistics” in this guide).
References and/or Suggested Reading:
https://www.managementstudyguide.com/second-
ary_data.htm
Observational Research
Observational research is a form of non-experimen-
tal research in which a researcher observes ongo-
ing behavior in their chosen surroundings (Sauro,
Back to Table of Contents
2015). Three types of observational research are
discussed below.
Naturalistic Observation
This form of research occurs in the everyday setting
of participants (as they live life normally). Thus,
there is no intervention by the researcher to influ-
ence the environment (McLeod, 2015).
Participant Observation
In participant observation, the experimenter in-
volves him/herself in the environment of the partic-
ipant, for example, as a member of a group. The
purpose of this is to focus on observing participant
behaviors that may not otherwise be discoverable
by the researcher. Such participant observations
can either be covert or overt in terms of the knowl-
edge or awareness made to the other participants.
The advantage to this form of observational re-
search is that it results in a greater insight into the
participants (McLeod, 2015).
Controlled Observation
This form of observational research is often used by
universities or labs and is carried out under spe-
cifically designed conditions. Such conditions are
discussed by the researcher in detail, and designed
with an attention to detail. Participants experience
the same situation so that their reactions can be
14
monitored. A key advantage of this form of obser- The following figure depicts this:
vational method is that the study is reproducible
(McLeod, 2015). CEO POPULATION: CEO
Experience ALL FIRMS WITH Gender
NUMBER OF
References and/or Suggested Reading:
EMPLOYEES > 100
CIRT (Center for Innovation in Research and Teach-
ing) Grand Canyon University (n.d.). Observation-
al method. Retrieved from https://cirt.gcu.edu/
research/developmentresources/research_ready/
descriptive/observational

McLeod, Saul (2015). Observation methods. Simply- 100+

Psychology. Retrieved from https://www.simplypsy-
chology.org/observation.html

Corporate CEO
Sauro, Jeff (2015). 4 types of observational research. Assets $ Compensation $
MeasuringU. Retrieved from
https://measuringu.com/observation-role/

Thus, in Multivariate Analysis, a researcher wants

Multivariate vs. Univariate
Analysis
When a statistical analysis is performed on a single
variable in a research setting, it is known as Uni-
variate Analysis. Statistical methods, such as single
and two sample t-tests and ANOVAs, are examples
of Univariate Statistical Procedures.
Consider the population consisting of all U.S. firms
with the number of employees greater than 100.
A researcher may wish to analyze CEO compen-
sation of these firms in a particular year. She may
consider CEO Compensation as the criteria, or
dependent variable (generically known as Y), and
examine three other factors as predictors or ex-
plainers of CEO compensation (Experience, Gen-
der, Corporate Assets).
to study many characteristics of a population
(whereas in Univariate Analysis, a researcher is
interested in analyzing a single characteristic of
a population). The most fundamental difference
between Univariate and Multivariate is that multi-
variate statistical techniques take into account the
inter-relationships among variables, while univari-
ate statistical methods do not.
Some widely used multivariate statistical methods
include:
• Regression and Correlation Analysis
• Canonical Correlation Analysis
• Principal Components and Factor Analysis
• Linear Structural Relations Models (LISREL)
• Multivariate Analysis of Variance Models
(MANOVA)
• Cluster Analysis

15 Back to Table of Contents

For tutorials on SPSS and Multivariate Analysis,
please see: https://www.youtube.com/results?-
search_query=spss+multivariate+analysis
Measurement of Variables
A central concept in statistics is the level of mea-
surement of variables. It’s so important to every-
thing a researcher does with data that it is usually
taught within the first week in every intro statistics
course. The four levels of measurement are (in
hierarchical order): nominal, ordinal, interval, and
ratio.
Nominal: These are unordered categorical vari-
ables. As Grace-Martin (n.d.) states, “These can
be either binary (only two categories, like gender:
male or female) or multinomial (more than two cate-
gories, like marital status: married, divorced, never
married, widowed, separated).” There is no logical
order to the categories, and it makes no sense to
rank, add, or subtract data that is nominally mea-
sured.
• Relevant Statistical Methods:
Since arithmetic operations are not relevant
for nominal variables, the only descriptive
measure that can be used is the calculation
of frequencies. In order to use statistical infer-
ence methods on nominal data, it must first be
converted to the interval scale.
Ordinal: These are ordered categories (still cate-
gorical, but in an order, such as Likert items, with
responses, such as, “Never, Sometimes, Often,
Always”) (Grace-Martin, n.d.). There is a logical
order to ordinal data, but since the difference
Back to Table of Contents
between ordinal values are meaningless, it makes no
sense to perform addition or subtraction on ordinal
data.
• Relevant Statistical Methods:
Only the median can be meaningfully calculated
using ordinal data. Parametric statistical inference
procedures should not be used with ordinal data.
Interval: As Grace-Martin (n.d.) states, these are “nu-
merical values without a true zero point. The idea
here is the intervals between the values are equal and
meaningful, but the numbers themselves are arbitrary. 0
(zero) does not indicate a complete lack of the quantity
being measured. IQ and degrees Celsius or Fahrenheit
are both interval.” Measurements belonging to this cat-
egory can be counted, ranked, added, or subtracted.
• Relevant Statistical Methods:
All descriptive measures of central tendency and
dispersion can be calculated using interval data.
In addition, all of the most powerful parametric
statistical methods (e.g., t-tests, ANOVA, regres-
sion/correlation, factor analysis, etc.), can mean-
ingfully use data that is measured at the interval
scale.
Ratio: Ratio data are numerical values having the same
properties as those from the interval scale, but with
a true zero point. Most measurements in the physical
sciences, engineering, and economics are done on
ratio scales.
• Relevant Statistical Methods:
All descriptive measures of central tendency and
dispersion can be calculated using ratio data. In
addition, all of the most powerful parametric sta-
tistical methods (e.g., t-tests, ANOVA, regression/
correlation, factor analysis, etc.), can meaningful-
ly use data that is measured at the ratio scale.
16
It is important to remember that the most powerful
statistical techniques only yield meaningful results
when the variables used are either measured at the
interval or ratio scale.
As Grace-Martin (n.d.) states, “Interval and Ra-
tio variables can be further split into two types:
discrete and continuous. Discrete variables, like
counts, can only take on whole numbers: number
of children in a family, number of days missed from
work. Continuous variables can take on any num-
ber, even beyond the decimal point. Not always
obvious is that these levels of measurement are not
only about the variable itself. Also important are the
meaning of the variable within the research context
and how it was measured.”
Discrete interval or ratio variables can be analyzed
using statistical procedures such as Chi-Squared
tests, Poisson regression, and the Negative Binomi-
al regression.
References and/or Suggested Reading:
Grace-Martin, Karen (n.d.). When a Variable’s Level
of Measurement Isn’t Obvious. The Analysis Factor.
Retrieved from
https://www.theanalysisfactor.com/level-of-measure-
ment-not-obvious/
not rigorous enough to be used exclusively for data
analysis in doctoral-level dissertations. Neverthe-
less, exploratory data analysis (EDA) should always
be performed and presented at the beginning of
any analysis. More rigorous and diverse statistical
procedures include Inferential Statistics (See “Infer-
ential Statistics” in this guide).
EDA should include both numerical summaries and
visual displays (e.g., figures, tables, graphs, and
charts). Numerical summaries are used to help
understand:
• the central tendency of the data (mean, median,
mode);
• the dispersion of data (variance, standard devi-
tion, range); and
• the shape and type of frequency distribution of
data (Normal Distribution or other).
Please note that, above, “data” refers to a hypo-
thetical two-dimensional spreadsheet where the
rows represent the subjects (or time periods) and
the columns represent the variables. For example, a
researcher can collect responses to 25 rating items
on an employee satisfaction survey for 100 sub-
jects. The columns for each subject would contain
their individual ratings on each of the 25 survey
questions.

Descriptive Statistics and

Exploratory Data Analysis (EDA)
Various kinds of statistical methods may be utilized
in any study (including a dissertation), and some of
these techniques prove to be more challenging than
others because of the concepts and mathematics in-
volved. Descriptive Statistics are numerical summa-
ries utilized to describe/explain empirical informa-
tion or data and are on the less difficult side of the
spectrum. Descriptive Statistics, by themselves, are

17 Back to Table of Contents

Given a sample of data, (n), from a population represented by a single random variable X, the formulas
for the sample measures of central tendency are:

The variance is not easily interpretable, since its for-
mula consists of sums of squares. A more interpreta-
ble measure of dispersion is the standard deviation,
which is calculated by taking the squared root of
the variance. Simply put, the standard deviation
tells one how far, on average, each data value lies
from the average (mean). In Finance and Econom-
ics, standard deviation is used as a measure of
error or “risk.” A larger standard deviation is as-
sociated with more error or higher risk (e.g., stock
market volatility).
It is also important to know what the overall pop-
ulation (or sample) looks like (e.g., is it Normally
Distributed or something else?). The best way to see
this is to plot a histogram and examine its shape.
This link can be helpful for how to create a histo-
gram in SPSS: https://www.youtube.com/results?-
search_query=SPSS+25+descriptive+statistics+his-
togram.
If the histogram looks “bell-shaped,” then the data
probably came from a Normal Distribution. How-
ever, sometimes looks are deceiving, so it is more
precise to test whether or not the data comes from
(or looks like) a Normal Distribution. Different
tutorials consisting of how to test for the presence or
absence of a Normal Distribution are available at
this link: https://www.youtube.com/results?search_
query=SPSS+25+normal+distribution+testing.

Back to Table of Contents 18

The reason why a researcher would want to know
whether or not their population is Normally Dis-
tributed is important. When the data are Normally
Distributed, things are better defined, because the
sampling distribution of the statistics used for esti-
mation and testing is also Normal. However, it is
not critical if the data are found to deviate substan-
tially from a Normal Distribution. There are two
reasons why:
• one can always use certain mathematical trans-
formations of the data to try to induce a Normal
Distribution (e.g., natural log, squared root,
etc.); and
• if the sample is large enough (n >100), it can be
assumed that, while the population may not be
Normal, the sampling distribution of the statistic
used to test the hypothesis will be Normal—re-
gardless of the actual shape of the distribution.
(This is known as the Central Limit Theorem.)
• In other words, as long as the sample is large
(n >100), one does not have to worry about
whether or not the data follows a Normal Distri-
bution.
Inferential Statistics
Inferential statistics is about taking samples and
using the sample results to make inferences about
population parameters, such as the mean and the
standard deviation. In other words, sample data
is used to gain insight into a whole population’s
characteristics. This makes sense because it is rare
to be able to take a complete census of a popula-
tion. Because of expense, time, and a variety of
other factors, it makes much more sense to draw a
random sample from this population.
An example of using interferential statistics include
a researcher who is interested in studying the
personality traits of accountants vs. sales represen-
tatives. She cannot survey or interview all accoun-
19 Back to Table of Contents
tants and sales representatives that exist. Therefore,
the researcher can observe a smaller segment, or
sample, of people who work in their fields, and
ensure that this sample is representative of the pop-
ulation under study.
There are two (2) requirements necessary for estab-
lishing that a sample is representative of the larger
population:
• when the sample is chosen, every element in the
population must have an equal chance of being
selected; and
• each sample selection is independent of all oth-
er sample selections.
Ensuring independence in the sampling process is
necessary, and not that difficult, when one is able
to run one’s own experiment and collect data. How-
ever, if a researcher is using data that was collect-
ed by another entity, then he or she must make sure
that the samples used are representative before
making any conclusions. The “equally-likely” issue
is not a problem when the population size is large.
As stated above, the purpose of taking a sample
from a larger population is to make inferences
about that population by using the information in the
sample. The sample mean ( X ) is used to estimate
the population mean ( μ ), and the sample stan-
dard deviation ( S ) is an estimate of the population
standard deviation ( σ ) . The sample histogram (see
“Descriptive Statistics and Exploratory Data Analysis
(EDA)” in this guide) can be used to estimate the
population histogram (or distribution). Sample statis-
tics, X and S, may be used to test hypotheses about
the population parameters, μ and σ.
Alpha Level (level of significance,
or p-value)
Alpha Level (level of significance or p-value) is the
probability of rejecting the null hypothesis when the
correct decision should be to fail to reject the null
hypothesis. Alpha levels are determined by the re-
searcher before performing the statistical analysis.
Determination of the alpha level is influenced by
Type I and Type II errors and repeated testing.
An alpha level of .05, or lower, is considered an
acceptable level of significance for a statistical test.
The alpha level drives decision making to reject,
or fail to reject, the null hypothesis. The lower
the alpha level, the less chance of a Type I error,
although this also places tighter controls around de-
tecting a difference, or relationship. The alpha levels
that are most often used are .01, .05, and .10.
References and/or Suggested Reading:
Cramer, D., & Howitt, D. (2004). The SAGE dictio-
nary of statistics. London: SAGE Publications, Ltd.
Hypotheses
Hypotheses are analytical statements (using popula-
tion parameters) about the relationships outlined in
the research question. Hypotheses should be close-
ly related to, and well aligned with, the research
questions guiding the study.
Each research question, with the exception of de-
scriptive research questions, contains a minimum of
two hypotheses: the null hypothesis and the alterna-
tive hypothesis (sometimes referred to as a research
hypothesis).
The null hypothesis is the hypothesis that the re-
searcher would like to disprove. This is the hypoth-
eses to be rejected, or nullified. Null hypotheses
for comparative research questions typically state
that the population means of two or more groups
are the same. Null hypotheses for correlational
Back to Table of Contents
research questions typically state that there is zero
(or no) correlation between the variables or con-
structs of interest. The alternative hypothesis is the
logical opposite of the null hypothesis, and it is
most often the hypothesis that the researcher would
like to prove. It is important to note that the null and
alternative hypotheses are mathematical statements
about population parameters (e.g., population
means, standard deviations, correlations, etc.) and
never contain statistics.
References and/or Suggested Reading:
Cramer, D., & Howitt, D. (2004). The SAGE dictio-
nary of statistics. London: SAGE Publications, Ltd.
Hypothesis Diagrams
One of the overall goals of quantitative research is
to seek theories that focus on possible relationships
among variables. A diagram is an effective method
to demonstrate the hypothetical pathways (relation-
ships) involved in a research project. It can assist
both the author/researcher and reader to follow the
intended suppositions.
Variables can have numerous different types of
relationships. For example, variable relationships
can be casual, conditional, reciprocal, symmetrical,
spurious, or controlling. When presenting a visual
representation of these relationships, pathways are
typically diagrammed, such as moderating, mediat-
ing, and confounding variable connections.
Example of diagramming a mediating relationship:
MEDIATING
VARIABLE
INDEPENDENT DEPENDENT
VARIABLE VARIABLE
20
Example of diagramming a moderating relationship:
MODERATING
VARIABLE
INDEPENDENT DEPENDENT
VARIABLE VARIABLE
References and/or Suggested Reading:
Creswell, J. W. & Creswell, J. D. (2017). Research
design: Qualitative, quantitative, and mixed methods
approaches. London: Sage Publications, Ltd.
Greenland, S. & Pearl, J. (2006). Causal diagrams.
Encyclopedia of epidemiology. Retrieved from
https://ftp.cs.ucla.edu/pub/stat_ser/r332.pdf
Hypothesis Testing
Hypothesis testing is the primary means for making
decisions based on statistical testing. Hypotheses
are declarative statements that outline relationships
or comparisons to be tested in a research study.
The null hypothesis is the core idea in hypothesis
testing. The null hypothesis is the hypothesis to be
rejected, or nullified. The steps in hypothesis testing
are:
1. State the null and alternative hypotheses (also
referred to as a research hypothesis), using only
population parameters.
2. Determine if a one-tailed or two-tailed test should
be used. Note: if the alternative hypothesis
contains a greater than (>) or less than (<) sign,
then a one-tailed test should be used. Conversely,
21 Back to Table of Contents
if the alternative hypothesis has a “not equal to”
sign , then a two-tailed test should be used.
3. Determine the alpha level (see “Alpha Level” in
this guide) to use in decision making to reject,
or fail to reject, the null hypothesis.
4. Select the appropriate statistical test to use.
5. Perform the statistical analysis. If a parametric
test is selected, be sure to test for assumptions.
6. Compare the probability value (p-value) from the
statistical analysis with the pre-determined alpha
level. If the p-value is at, or less than, the select-
ed alpha level, then reject the null hypothesis in
favor of the alternative hypothesis. Conversely,
if the p-value is greater than the selected alpha
level, then fail to reject the null hypothesis.
References and/or Suggested Reading:
Cramer, D., & Howitt, D. (2004). The SAGE dictio-
nary of statistics. London: SAGE Publications, Ltd.
T-Test
The T-test is used to analyze whether or not there is
a significant difference between the means of two
groups. T-tests are hypothesis testing tools which
give the researcher the ability to test an assumption
about a population, and are used to determine if
there is a significant difference between the means
of two groups. The process of finding differences
between two sets of data are analyzed through the
use of the t-statistic, t-distribution, and degrees of
freedom.
The goal is to test the null hypothesis that there is
no statistically significant difference between the
means of two groups. If the null hypothesis is reject-
ed, then there is a statistically meaningful difference
between whatever is being compared.
One version of the t-test is used to compare inde-
pendent groups. In other words, each group con-
tains unique membership, and no two items may
belong to both groups.
Another version is known as the paired-samples
t-test. This version is used to compare a single
group at two different time periods (before and
after). For example, say a group’s job satisfaction
was surveyed, and then a new incentive schedule
is announced. This incentive is an intervention that
may impact one’s satisfaction with their work. The
group will be surveyed again after the incentive is
announced to see if the potential for a reward ties
the group members closer to their jobs.
Here is a link to several video tutorials on how to
do independent and paired-sample t-tests in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+tutorial+t+tests
Analysis of Variance (ANOVA)
Analysis of Variance (ANOVA) is a statistical tech-
nique which allows a researcher to investigate
whether or not there are differences between group
mean scores. ANOVA essentially finds these differ-
ences by “splitting” the variability found inside a
data set into two parts: systematic and random. The
systematic factors are due to the variation between
groups, while the random component is dependent
upon the variation among the experimental units or
data observations. ANOVA is an “omnibus” test.
This means that the test will indicate whether or not
there are statistically significant mean differences
across the comparison groups. But it will not, neces-
sarily, identify which groups are different.
If the null hypothesis of no mean differences is
rejected in an ANOVA (i.e., there is a statistically
significant difference between two or more groups),
Back to Table of Contents
a researcher needs to dig deeper to find out exactly
where the nature of those differences lie. To do that,
the researcher has two options. One of the options
is to conduct a Tukey HSD test; the other is to con-
duct Dunnett’s C test. Tukey’s HSD is used when
assumptions are not violated, and Dunnett’s C test
is used when assumptions are violated. If the null
hypothesis is accepted, then there is no need to do
a Post Hoc test. A Post Hoc test is conducted only
when a researcher has found a statistically signif-
icant difference between group means and wants
to discover which groups have different, and which
groups have equivalent, means.
ANOVA Examples
A One-Way ANOVA is used when there is a single
variable which is measured across two (2) or more
groups. For example, a researcher is interested in
technology use across age groups, and observed a
large group of people and how they use their devic-
es. The people were randomly assigned into groups
aged 19-25, 26-35, and 36-45. In this situation,
ANOVA may be used to see if the different age
groups use technology differently, by examining the
mean differences of technology use across these
age groups. For a more detailed example, consider
a healthcare manager who is interested in studying
employee engagement in a hospital system. As
client and patient satisfaction surveys play a role
in the level of insurance reimbursement a hospital
will receive, these are important data to investigate.
The administrator consults a researcher who ex-
plains that job satisfaction is a predictor of patient
satisfaction (in other words, the worker’s emotional
ties to the job will rub off, so to speak, on patients
they care for). The manager then administers a job
satisfaction survey to three different departments in
the hospital: emergency room, intensive care unit,
and out-patient services. A fourth group, administra-
tive support staff, who have little patient interaction,
22
is used as a control group. The manager could then use a One-Way ANOVA to determine whether or not
job satisfaction is different across the four departments (to see if one department is more satisfied than the
other, or if the three patient-facing units are more satisfied than the control group). The results will help the
manager to triage the emotional attachment of the workers.

A Two (or More)-Way ANOVA is a more complex design in which there are two or more independent
variables measured over two or more groups. Consider an operations manager of a financial services firm
who is interested in surveying how committed the firm’s workers are to their jobs. The manager would like
to test that men and women have different levels of commitment and also that people over 40 years of age
have a different level of commitment than those under 40. The manager may use a Two-Way ANOVA in
order to compare two independent variables: gender and over-under 40. The result of this analysis will
lend empirical support for or against the manager’s belief.

In the above example, since both gender and over-under 40 contain two (2) groups, there are four (4)
means (2 x 2) to consider and test for differences. However, this testing is confounded when the presence
of “interaction” is considered. To keep it simple, in the two-variable case (Two-Way ANOVA considered
above), interaction would exist if the effect that being male or female had on commitment depended upon
whether a worker was over 40 or not (or vice versa). A full factorial ANOVA includes both main effects
(gender and above-below 40) and interaction effects (the interaction between gender and above-below 40).
23 Back to Table of Contents
It is important to note that in the absence of inter-
action, there are only two mean differences to test
for the dependent variable commitment—one due
to gender and the other from being over-under 40.
However, when interaction is present, there are six
(6) such mean differences to consider (in our ex-
ample—see if you can work it out). So, interaction,
while being more realistic from a research perspec-
tive--is also more complex. That is why the first step
in a Two (or More)-Way ANOVA is to test the null
hypothesis of the absence of interaction, which, if
accepted, would make things a lot easier.
Here is a link to several video tutorials on how to
do One (or More)-Ways ANOVAs in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+anova
Another common instance of ANOVA, known as
Analysis of Covariance (ANCOVA), is to control
for, or “hold constant,” interval or ratio scale vari-
ables. For example, continuing with the sex/age
example, say the manager thinks that the level of
education might matter in this analysis. He/she
could use ANCOVA techniques to compare women
and men and both age groups, controlling for their
level of education. In this way, any statistical effects
of educational level are removed from the analysis;
and the manager gets a “purer,” (so to speak), indi-
cation of the effects of sex and age on commitment
to the employer.
Here is a link to several video tutorials on how to
do ANCOVA in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+ancova
Back to Table of Contents
References and/or Suggested Reading:
Kerlinger, F., and Lee, H.B. (2000). Foundations of
behavioral research (4th ed.). Orlando, Florida:
Hartcourt College Publishers.
Saunders, M., Lewis, P., Thornhill, A. (2015). Re-
search methods for business students (7th ed.) Essex,
England: Pearson Education Unlimited.
Correlation
Correlation is a measure of the degree of “linear”
relationship between (or amongst) variables. The
most common is known as Pearson Product Mo-
ment Correlation, noted in the literature as “r.”
Correlation ranges from -1 (perfectly inverse/neg-
ative correlation) to a +1 (perfectly direct/positive
correlation). A correlation that hovers around zero
indicates that variables are not linearly related.
Correlation is only a measure of the “linear” or
straight-line relationship between variables. Cor-
relation does not measure the degree of a non-lin-
ear relationship. Moreover, researchers must be
careful when drawing conclusions using correla-
tion, as it does not assume causation (correlation
does not imply causation).
There are three types of correlation:
1. simple correlation (between one “dependent”
variable, Y, and one “explanatory” variable, X);
2. multiple correlation (between one “dependent”
variable, Y, and many “explanatory” variables
X1, X2, X3,…);
3. canonical correlation (between many “depen-
dent” variables and many “explanatory” vari-
ables).
Simple correlation is calculated between two vari-
ables. Multiple correlation is computed between
24
one variable, on one hand, and two or more variables, on the other (e.g., think of the relationship between
the weight of an individual as a function of height and average daily calories). So, it involves using many
variables.

It is important to note that correlation tells us not only the strength of a linear relationship
(close to -1 or +1), but also the direction.

In other words, a positive simple correlation indicates that increases in the X variable is associated with
increases in the Y variable (and vice versa). A negative simple correlation tells us that increases in the X
variable is associated with decreases in the Y variable (and vice versa).

This “scatterplots” (below) show, graphically, the different strengths and directions of linear relationships that can
exist between two variables. The straight blue line going through the points depicts the “linear” relationship.

CORRELATION
(INDICATES THE RELATIONSHIP BETWEEN TWO SETS OF DATA)

STRONG POSITIVE CORRELATION WEAK POSITIVE CORRELATION STRONG NEGATIVE CORRELATION

WEAK NEGATIVE CORRELATION MODERATE NEGATIVE CORRELATION NO CORRELATION

The source for this graphic can be accessed by clicking here.

25 Back to Table of Contents

A correlation matrix is used when a researcher
wants to display many different simple correla-
tions for multiple variables. The correlations are
all assembled into a table with the number one (1)
always going down the main diagonal of the table.
For instance, with four variables, there would be
four different simple correlations computed; the ma-
trix (table) would consist of these four correlations
in the off-diagonal elements, and the number one
(1) in the main diagonal of the table.
Here is a link to several video tutorials on how to
run simple correlations in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+correlation
Spurious correlations can also occur when two
variables seem to be correlated (numerically) but
are actually not correlated. Often, their correlation
is really driven by a third, hidden variable.
Correlation and regression analysis go hand in
hand. While correlation measures the strength and
direction of relationship, it does not give the actual
linear relationship. Regression analysis will yield
the equation of the straight line (the blue line in the
plots above) going thru any scatter of points,
where b0 is the “Y” intercept and b1 is the slope.
The correlation coefficient and the slope will always
have the same mathematical sign.
References and/or Suggested Reading:
Rodgers, J.L. & Nicewander, A.W. (1988). Thirteen
ways to look at the correlation coefficient. The
American Statistician, 42(1), 59-66.
Back to Table of Contents
Regression Analysis
Regression analysis is used to make predictions
about how one variable may influence another. The
concept of regression analysis was first conceived
by Sir Francis Galton, a cousin of Charles Dar-
win. Galton was studying the theory of evolution
and observed the concept of “regression toward
the mean,” when studying sweet peas. This led to
predictable measurement outcomes and, eventually,
some of the early concepts of regression analysis.
Regression analysis can be used to determine the
strength and direction of a relationship between
variables (similar to correlation analysis). However,
regression analysis differs from correlation analysis
in being able to predict the levels of the dependent
variable by knowing the values of the independent
variable.
Regression is used for many applications in industry
from sales forecasting to credit scoring. It is also
used extensively in government applications for esti-
mating budgets, economic forecasting, and improv-
ing the provision of public services to citizens.
Simple linear regression (also known as a bivariate
regression) is the prediction of a dependent vari-
able using a single independent (or explanatory)
variable. Multiple Regression is the prediction of a
single dependent variable using two or more inde-
pendent (or explanatory) variables. Data are col-
lected on an independent variable (X) and a depen-
dent or criterion variable (Y) for each individual,
and an equation is computed that depicts a linear
relationship between the two variables.
26
Here is a link to several video tutorials on how to
do a regression analysis is SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+regression+example
Please review this YouTube video for information
about fitting a non-linear regression in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+nonlinear+regression+example
SPSS is offered at no cost to NCU students through
the university. This is available through the
University Services Module in NCUOne.
In simple linear regression, the research question
posits the relationship between two variables. For
example: Does the transformational leadership style
(independent variable) have a direct effect on work-
er productivity (dependent variable)? A researcher
can explore this using simple regression. Another
example of a research question is: What is the
linear relationship that would predict the extent of
physical injury from body strength for elderly wom-
en, and how accurately does this equation predict
the extent of physical injuries?
An index, known as “r-squared”, is obtained in
a regression analysis by squaring the correlation
coefficient. R-squared directly tells us how well we
can predict Y from X. It is also referred to in the
literature as the coefficient of determination, and
is formally defined as the proportion of variation
explained in the dependent variable, Y, by the
explanatory variable, X. R-squared is a measure of
the “goodness of fit.”
Multiple (Linear) Regression Analysis refers to when
there are two or more independent variables used
to predict a dependent variable. An example of
27 Back to Table of Contents
multiple regression might be suggesting that a
leader’s behavioral transparency (X1) and sense of
humor (X2) will lead workers to experience a high-
er level of positive emotions (Y). Here are a number
of tutorials to learn how to fit multiple regression
analyses in SPSS:
https://www.youtube.com/results?search_que-
ry=SPSS+multiple+regression+example
Building a linear regression model is only part of
the process. When using the model in a real-world
application, one should take steps to ensure the
model conforms to the assumptions of linear regres-
sion. There are 9 key assumptions of regression
analysis:
1. The regression model is linear in parameters;
2. The mean of residuals is zero;
3. Homoscedasticity of residuals or equal
variance;
4. Zero (0) correlation of residuals;
5. The X variables and residuals are uncorrelated;
6. The number of observations must be greater
than the number of Xs;
7. The regression model is correctly specified;
8. The independent variables are not highly
correlated with each other; and
9. Normality of residuals.
These assumptions will vary in importance de-
pending on how one intends to make predictions
for individual data points, or if the coefficient is to
be given a causal interpretation. One of the most
important assumptions, which is often overlooked,
is that of validity. This means that the data used
should address the research question seeking to be
answered (Gelman & Hill, 2007).
References and/or Suggested Reading:
Gelman, A., Hill, J. (2007). Data analysis using
regression and multilevel/hierarchical models. Cam-
bridge: Cambridge Press.
Saunders, M., Lewis, P., Thornhill, A. (2015). Re-
search methods for business students (7th ed.) Essex:
Pearson Education Unlimited.
Stanton, J.M. (2017). Galton, Pearson, and the peas:
A brief history of linear regression for statistics in-
structors. Journal of Statistics Education, 9(3), 10-23.
Warner, R.M. (2008). Applied statistics: From bivar-
iate to multivariate techniques. Thousand Oaks, CA:
Sage Publications.
Factor Analysis
Factor analysis may be used to analyze the struc-
ture of interrelation or correlations across a large
set of variables in a dataset (e.g., test scores, test
questions, or questionnaire responses). The pro-
cedure will derive a smaller set of uncorrelated
variables, known as factors. These factors must be
interpreted in order to give meaning to the new
composite measures.
Factor analysis techniques are either exploratory
or confirmatory. Exploratory factor analysis (EFA) is
useful for analyzing structure in the original set of
variables and is used as a variable reduction tech-
nique. This method is appropriate for reducing the
size of datasets with many variables. Confirmatory
factor analysis (CFA) is useful when researchers
have conceptual theories, or prior research which
support preconceived ideas on the actual structure
of the data. In the situation where the researcher
wishes to test hypotheses about how variables
should be grouped from factors, or the number of
factors, a confirmatory approach must be taken to
Back to Table of Contents
assess the level from which the data may meet the
new expected structure. However, in order to be
able to undertake statistical testing in CFA, all of
the variables and factors must have a Multivariate
Normal Distribution. This is a significant limitation
of this approach.
Once the research problem is defined adequately,
the researcher must make the decision as to wheth-
er the factor analysis will be exploratory for iden-
tifying structures through data reduction, or confir-
matory for data summarization. If confirmatory, the
researcher should also decide if structural equation
modeling might be appropriate if it is hypothesized
that a tight fit (or close relationship) may exist in the
data. If exploratory, the researcher should select the
type of factor analysis regarding variables or cas-
es. Cases are comprised of Q-type factor analysis
or cluster analysis, while variables are R-type factor
analysis.
Factor analysis is typically conducted using inter-
val or ratio measured variables, and incorporates
some assumptions regarding testing. A structure
should not exist prior to conducting factor analysis.
Bartlett’s test of sphericity (sig. < .05) can show if
enough correlations exist among variables to pro-
ceed when statistically significant. Measurements
denoting sampling adequacy values must exceed
.50 for both the overall test and each individual
variable. Next, the factor matrix is specified to de-
termine the number of factors to be retained. After-
wards, a rotational method with considerations of
whether the factors should be correlated (oblique)
or uncorrelated (orthogonal) is chosen. Orthogonal
methods include VARIMAX, EQUIMAX, and QUAR-
TIMAX. Oblique methods include Oblimin, Promax,
and Orthoblique. The factor model respecification
will consider whether any variables were deleted,
changing the number of factors. The factor matrix
then undergoes validation with consideration of
28
split/multiple samples, separate analysis for sub-
groups, and identifying influential cases. Once all
of this is completed, the researcher then can make
a selection of surrogate variables, compute factor
scores, and create summated scales.
When selecting factor models, and number of
factors, some best practices may be helpful. Com-
ponent analysis models are appropriate when the
aim is data reduction. The common factor model
is best when there are highly specified theoretical
applications.
This is a YouTube tutorial on how to do an EFA in
SPSS:
https://www.youtube.com/results?search_query=-
exploratory+factor+analysis+in+spss+step+by+step
This is a tutorial on how to do a CFA in SPSS:
https://www.youtube.com/results?search_query=-
confirmatory+factor+analysis+in+spss+step+by+-
step
References and/or Suggested Reading:
Hair, J. F., Black, B., Babin, B., & Anderson, R. E.
(2010). Multivariate data analysis: A global perspec-
tive (7th ed.). Upper Saddle River: Pearson.
Power (Statistical Power)
Power (Statistical Power) is the ability of the statisti-
cal test to detect and reject a null hypothesis when
the null hypothesis is false (and should be reject-
ed). The power of a statistical test is reported as a
probability with values ranging between zero and
1.0. For example, a null hypothesis of no difference
between two groups is rejected. A statistical power
value of 0.8 would be interpreted as an 80% prob-
ability that the null hypothesis is false. A statistical
29 Back to Table of Contents
power of between 0.80 and one (1) is considered
acceptable power for a statistical test.
References and/or Suggested Reading:
Hedberg, E. (2018). The what, why, and when of
power analysis. In Hedberg, E. Introduction to pow-
er analysis: Two-group studies (pp. 1-9). Thousand
Oaks, CA: SAGE Publications, Inc.
Power Analysis
Power Analysis is, technically, the computing of
statistical power (see “Power (Statistical Power)” in
this guide). There are two occasions when a power
analysis should be performed:
1. During research design (a priori power
analysis).
2. After the statistical test has been run (post hoc
power analysis).
There are statistical packages available to perform
these types of power analyses. The most common
statistical package used by NCU dissertation can-
didates is G*Power because it is available over the
internet free of charge, and it is user friendly.
In a dissertation, a student needs to state that the
minimum required sample size has been reached,
and plan early about how to reach this sample size
(the percentage of a sample that will actually re-
spond to a survey or questionnaire is very small).
A student should thus discuss how participants will
be recruited and/or how the data will be obtained
with sufficient detail.
The purpose of an a priori power analysis is to
determine the minimal sample size needed to detect
the relationship of interest and the probability of
rejecting a null hypothesis when the null hypothesis
is false. This helps the researcher determine if their
sampling frame (the group the researcher will be
recruiting from) is large enough so that the research-
er might conceivably recruit enough participants
and support hypothesis testing. An a priori power
analysis does not compute power of the statistical
test because data has not yet been collected and
the statistical test has not yet been run. Instead, the
researcher selects the statistical test to be performed
and enters the pre-determined power, alpha level
(see alpha level), and estimated effect size. Most
software packages include standardized effect size
values based on small, medium, and large catego-
ries. Power values should be set at, or greater than,
0.80, and alpha levels should be set at .05 or lower.
The purpose of a post hoc power analysis (also
referred to as observed power in the literature) is to
determine power of the statistical test based on the
known sample size, known effect size, and known
alpha level. A statistical power of 0.80 or greater
is considered acceptable power for a statistical
Back to Table of Contents
test. Some researchers purport that post hoc power
values are inflated (so it is best to interpret this value
conservatively).
References and/or Suggested Reading:
Cohen, J. (1988). Statistical power analysis for the
behavioral sciences (2nd ed.). New Jersey: Lawrence
Erlbaum.
Gogtay, N. (2010). Principles of sample size calcula-
tion. Indian Journal of Ophthalmology, 58(6), 517-
518.
Universität Düsseldorf. (2014). G*Power 3.1 manual.
[PDF]. Retrieved from: http://www.gpower.hhu.de/fil-
eadmin/redaktion/Fakultaeten/Mathematisch-Natur-
wissenschaftliche_Fakultaet/Psychologie/AAP/gpow-
er/GPowerManual.pdf
G*Power [computer software] available at: http://
www.psychologie.hhu.de/arbeitsgruppen/allge-
meine-psychologie-und-arbeitspsychologie/gpower.html
30
Measuring Validity and Reliability
Validity can be defined as the degree to which
instruments and tools in a research study measure
what they are intended to measure. If study findings
are determined not to be valid, then the results are
essentially meaningless. The tools must measure
what they are intended to measure. Otherwise,
the results will not allow the investigator to answer
the research question(s). In other words, without
validity, the study’s purpose is missed. Validity is
sometimes contextual in that a valid research study
in one circumstance does not necessarily mean that
it is valid in another.
Validity and reliability are two unique concepts.
While validity, defined above, basically notes
whether or not an instrument measures what it
is intended to (e.g., assessing the efficacy of a
teaching style versus a student liking an instructor
or a course), reliability is measured in terms of
the consistency of the results. There can be three
types of consistency that are considered: over time,
across items, and across researchers. Assessing
reliability over time is assessed using a test-retest,
where respondents are measured at one point in
time and then at another point in time. If the results
are correlated (such as over .80), the measure is
considered reliable. Internal consistency measures
the consistency of items in a multi-item measure. The
most common measure on internal consistency is
a statistic known as Cronbach’s coefficient alpha.
Finally, inter-rater reliability assesses the consistency
in the judgment of observers or raters.
In relation to measurement validity, there are four
primary types: face validity, content validity, crite-
rion validity, and construct validity. Each of these
types defines validity from a unique perspective
31 Back to Table of Contents
and evaluates it differently. Face validity is like a
“gut check,” the weakest assessment of validity.
Does the measurement look like it should yield
results as intended? For example, if a researcher in-
tends to study positive emotions, an instrument that
appears to measure positive emotions makes sense.
Content validity is the degree the measure covers
the entire scope of the concept that is being mea-
sured. For instance, if brand loyalty is considered to
be both a behavioral and cognitive phenomenon,
both of these dimensions need to be included in the
measurement. Criterion validity refers to how well
the measure is related to an outcome that may be
classified as concurrent or predictive that refers to
the time sequence. For instance, criterion validity
assesses whether the measure is correlated with
what it is intended to measure, such as a preg-
nancy test predicting pregnancy, or whether an
SAT predicts college performance. Last, construct
validity is a determination of whether or not the
measurement is actually measuring what it is intend-
ed to measure. For instance, does an IQ test really
measure intelligence? Or might it be measuring
educational level?
It is important to ensure there is no confusion be-
tween validity and reliability. It is possible for a
study to be reliable, but not valid. In other words,
reliability is a necessary but not sufficient criteria
for validity.
To conclude, validity is essential to attain in con-
ducting research, especially in the social sciences.
Validity should be considered by researchers as
early as the development of the research questions,
and certainly through study design and implemen-
tation. In order to discover results efficacious to
answer a research question, validity must be con-
trolled as much as is feasible.
References and/or Suggested Reading:
Petty, R. E, Briñol, P., Loersch, C., & McCaslin, M. J.
(2009). The need for cognition. In M. R. Leary & R.
H. Hoyle (Eds.), Handbook of individual differences
in social behaviour. New York, NY: Guilford Press.
Kerlinger, F.N., & Lee, H.B. (2000). Foundations of
behavioral research (4th ed.). Belmont: Wadsworth/
Thomsen Learning.
Trochim, Wiliiam M., Donnelly, James P., & Arora,
Kanika (2016). Research methods: The essential
knowledge base. Boston: Cengage Learning.
Internal/External Validity
There are two main types of validity that are consid-
ered in the design and evaluation of experimental
designs. Internal validity refers to whether or not an
experiment can demonstrate that the effect of an
independent variable can be clearly attributed to
changes in the dependent variable. For example, if
we explore the impact of role mentorship on resil-
ience, we must rule out (as best as possible) the fact
that other variables may influence (i.e., moderate
and/or mediate) this association. External validity
refers to the generalizability of the study results. For
instance, with respect to generalizing to the popula-
tion, a researcher would have better external validity
if the sample was taken randomly from the popula-
tion. In fact, a primary challenge in all research is to
suggest that research findings are generalizable to
populations, settings, products, time periods, etc.
Threats to Validity
Internal validity has been widely written about, and
a number of factors have been identified that can in-
fluence it (Campbell & Stanley, 1963). Investigators
attempt to control for these factors as much as pos-
sible during research to attempt to achieve internal
validity. However, the need for this control can also
impact generalizability.
Back to Table of Contents
These factors include experimental mortality, or the
loss of participants in the comparison groups. This is
especially true during longitudinal experiments. His-
tory is another threat and refers to events that occur
outside of the experiment but during the same mea-
surement periods. Similarly, maturation are changes
that occurs in subjects during the course of the ex-
periment. Subjects may age, or may simply become
hungry or bored during the course of an experiment.
The threat of testing indicates a practice effect in that
repeated applications of a measurement may impact
subsequent data collection.
There are also threats to external validity, or the gen-
eralizing of findings. These include interaction effects
where a pretest might decrease a participant’s sen-
sitivity to an experimental variable. Another threat
is multiple-treatment interference, where participants
are exposed to a series of treatment conditions and
the effects of prior conditions are not ‘erasable.’
Note that there is often a tradeoff between internal
and external validity and the experimental setting (a
lab vs. field experiment). A laboratory experiment is
an artificial setting that allows the researcher better
control over extraneous/potentially confounding
variables. However, the artificiality of an experiment
tends to lessen the external validity since a research-
er wants to be able to generalize to a more realistic
setting. Essentially laboratory vs. field experiments
represent opposite ends of a continuum having to do
with the artificiality of the setting.
References and/or Suggested Reading:
Campbell, D.T., & Stanley, J.C. (1963). Experimental
and quasi-experimental designs for research. Boston:
Houghton Mifflin.
Zikmund, William G., Babin, Barry J., Carr, Jon C.,
& Griffen, Mitch (2013). Business research methods
(9th ed.) Mason: South-Western, Cengage Learning.
32
Selection of Parametric vs. able). However, nonparametric methods are “less
powerful” than parametric methods. The probability
Nonparametric Techniques
that the null hypothesis will be rejected when it is
Most researchers would always opt to use paramet-
false, is less for nonparametric tests as compared
ric statistics to analyze their data, given that most
with parametric tests.
consumers of their research are familiar with these
techniques and, in general, these tests are very
It should be remembered that, with parametric tests,
powerful.
the hypotheses are about population parameters
(e.g. μ = 50 or μ1 = μ2 ). With nonparametric tests,
Many statistical methods (e.g., t-test, correlation,
the null hypothesis is more generalized. For exam-
and regression) are referred to as ‘parametric’ and
ple, in a parametric test the null hypothesis may be
require that the parameters and underlying distri-
that two populations are equal. However, in non-
bution of the data exist. In the t-test, for example,
parametric statistics, this is interpreted as the two
these parameters are the mean and standard devi-
populations being equal in terms of their central
ation. Moreover, the sample must have come from
tendency (which could involve medians).
a univariate Normal distribution. There are various
statistical tests which can be used to assess whether
data are likely to have come from a Normal distri-
bution. These include the Anderson-Darling test, the
Kolmogorov-Smirnov test, and the Shapiro-Wilk test.
Parametric statistics require assumptions to be
made about the format of the data to be analyzed.
Perhaps the most important aspect is when the data
is not normally distributed, such as when the out-
come is an ordinal variable or a rank, then there
are outliers, and the outcome has clear limits of
detection. Parametric tests also involve estimation
of the key parameters of that distribution (e.g., the Nonparametric tests have some definite advantag-
mean or difference in means) from the sample data. es when analyzing variables which are ordinal,
In many cases in the social sciences (including contain outliers, or are measured imprecisely. If
business), these assumptions hold, and even where one wanted to still analyze with parametric meth-
they do not, the data can often be transformed by ods, then major assumptions would have to be
researchers in order to meet the required assump- made about distributions, as well as difficult and
tions. However, there are cases where the assump- error-prone decisions about coding values. Interest-
tions, even with transformed data, do not support ingly enough, many parametric tests perform well
the use of parametric statistical techniques. in non-Normal and skewed distribution environ-
ments--as long as the sample size is large enough.
Nonparametric tests are sometimes referred to as Researchers should always take this into consider-
being distribution-free tests because they are based ation before assuming they need to choose a non-
on fewer assumptions (e.g., they do not assume an parametric test as their only option. However, when
approximate univariate normally distributed vari- sample sizes are relatively small, many statisticians

33 Back to Table of Contents

choose to conduct nonparametric tests which are simpler to conduct and easier to interpret. Below is a
table of parametric tests and their nonparametric counterparts:

METRIC TESTS (MEA RAMETRIC TESTS (ME

RA NS PA DIA
PA ) N NS
NO 1-SAMPLE
)
1-SAMPLE SIGN,
T-TEST 1-SAMPLE
WILCOXON

MANN
2-SAMPLE
WHITNEY
T-TEST TEST

KRUSKAL
ONE-WAY WALLIS,
MOOD’S
ANOVA
MEDIAN
TEST

FACTORIAL FRIEDMAN
DOE TEST

with one factor and one

blocking variable
Source: https://blog.minitab.com

Below are some general guidelines for applying nonparametric statistical tests to data:

If one’s analysis includes two independent samples,
and the data are:
Nominal: consider Chi-square test or Fisher
exact test.
Ordinal: consider Wilcoxon-Mann-Whitney test
or Kolmogorov-Smirnov two-sample test.
If one’s analysis includes matched (or related) sam-
ples, and the data are:
Nominal: consider McNemar change test.
Ordinal: consider Wilcoxon signed ranks test.
If one’s analysis includes three or more independent
samples, and the data are:
Nominal: consider Chi-square test.
Ordinal: consider Kruskal-Wallis one-way
analysis of variance.
If one’s analysis includes measuring relationships,
and the data are:
Nominal: consider a Phi coefficient or kappa
coefficient.
Ordinal: consider a Spearman correlation coef-
ficient or Kendall’s Tau.

Back to Table of Contents 34

Here are some YouTube tutorials for learning how
to work with nonparametric statistics:
https://www.youtube.com/results?search_que-
ry=SPSS+tutorial+nonparametric+statistics
References and/or Suggested Reading:
Box, G. E. (2013). An accidental statistician: The life
and memories of George E. P. Box. Hoboken: Wiley
and Sons.
Lamorte, W., W. (2017). When to use a nonpara-
metric test. Boston University School of Public Health
Best Practice Module, Retrieved from: http://sphweb.
bumc.bu.edu/otlt/mph-modules/bs/bs704_non-
parametric/BS704_Nonparametric2.html
Riegelman, R. (2013). Studying a study and testing a
test. (6 ed.). Baltimore: Lippincott Williams & Wilkins.
Whitley E., & Ball, J. (2002). Statistics review 6:
Nonparametric methods. Critical Care, 6, 509-512.
Retrieved from: https://doi.org/10.1186/cc1820
Presentation of Statistical Results
and Explaining Quantitative
Findings in a Narrative Report
What a researcher discovers is just as important as
how they communicate it to readers. Communicat-
ing data and statistical findings is an essential skill
and an important element in presenting research
findings. If miscommunicated, an audience may be
lost or, at a minimum, bored. Proper presentation,
done correctly, can have an enduring impact on
audiences—both readers and those who attend
presentations or doctoral dissertation defenses.
Data and statistics are left-brained material, in
that they tap into logical and rational information
processing. However, readers and audiences are
35 Back to Table of Contents
more likely to retain right-brain presentation ele-
ments, such as demonstrations, examples, stories,
and analogies. In order to be enduring, statistical
findings should be logical and rational, as well as
memorable.
For example, a study finding that job performance
and job satisfaction share a positive correlation of
r = .31 is rational evidence. Such data could be
bolstered, for example, with other data or research
(e.g., a story of a qualitative prediction by a busi-
ness guru stating that to impact job performance, a
manager might influence job satisfaction).
The 6th edition of the Publication Manual of the
American Psychological Association (APA) devotes
an entire chapter to “Displaying Results” through
proper design and placement of tables and figures
to illustrate findings. In other words, a picture truly
is worth a thousand words if the narrative augments
the displayed evidence. This is true both in a man-
uscript and in a presentation. While in a presenta-
tion, voice may be used to help illustrate the impact
of research findings. The APA manual provides
important formatting and presentation guidelines
for tables and figures so that they are not cluttered
and have the greatest potential for impact. Exam-
ples are also provided for enhancing the visual aids
with a well-composed narrative. An emphasis is
placed on conciseness of the content of each visual,
as well as standard form.
Because the APA understands that most of the
information stemming from a quantitative result is
usually manipulated by a statistical package, the
charts and graphs usually follow a Microsoft Word
document-based construction. This construction is
based on the normal APA rules of formatting, such
as font, size, and spacing. This can be easily found
in the APA 6th edition manual, chapter 5. Most of
the statistical packages will be able to convert these
tables or figures to APA. Even though this is noted,
make sure that everything within a chart or figure
matches the requirements of the correct font, size,
and color for APA when used within the dissertation
process.
The student must make sure that everything has con-
verted correctly and all numbers have been copied
over properly. Everything should also look concise
and clear as it is retrieved from one chart and
re-entered into the dissertation. This may include
the use of the same number of decimal points and
similar cell formatting within a table. Students must
realize that some statistical software packages may
not typically put a demographic in the same sam-
pling frame as a frequency. Therefore, a student
must have enough forethought to work within the
Microsoft Word program to be able to add columns
and rows within table properties.
The most needed explanations for quantitative
methodology consist of: 1) the explanation of the
sample data (descriptive analysis including inde-
pendent and dependent variables); 2) the statistics,
Back to Table of Contents
both in text and in charts, tables, or graphs (using
the correct APA recommended spacing, alignment,
and punctuation marks for those particular statis-
tical tests); and 3) final results, including the null
hypothesis testing with the probability of occur-
rence (p-value). These explanations are normally
displayed within the results section (Chapter 4) of
the dissertation, but they may also cross over into
Chapter 5.
During the explanation of the sample, the pop-
ulation, sample size, type of sampling, power
analysis, instrumentation, and variables should be
explained using measures of central tendency and
tables, as needed (Creswell & Cresswell, 2018).
These explanations of the sampling data should
include measures of central tendency that are ap-
propriate to the data: the range, means, medians,
modes, standard deviations, etc. This can exist in
36
table format to help readers understand the many
variables that are being used. A best practice for
what a researcher enters into a frequency or distri-
bution table would be to add any typology of the
variables. This means that most of the questions on
a questionnaire should exist in a table format.
A researcher should also explain any categories of
numerals listed within a table that are being used
to explain the sample (Elliott & Woodward, 2020).
If a researcher decides to use categories of ages
instead of an open question of age for the sample,
the category and the demographics of that cate-
gory need to be explained, using the frequency or
description for each (e.g., years of age: “young”,
< 18 = 0, 0%; “middle age”, 18.5 - 35.0 = 50,
60%, etc.). Describing tables and charts in words
is important. Many of these tables will occur in the
appendices and be alluded to within the in-text
explanation. An example is: “A total of 188 people
answered the questionnaire. It consisted of only
people over 18. However, these people were load-
ed into two groupings, ‘young’ and ‘middle-aged,’
created by the researcher. Sixty percent of the
sample was… There were more males (100) than
females (88) within the dataset.” Without the expla-
nation of the dataset, the reader would not be able
to understand what happened within the sample,
nor understand the subsequent findings or results
(Adams & Lawrence, 2019).
up will include all the steps that are needed for a
correct, finalized explanation of the findings. Nor-
mally, within this section, a table may be alluded to
and used in an appendix.
Do not forget to state what the hypothesis testing
found, including the effect, and if it was the direc-
tion that was expected (if that was stated in the
hypothesis testing). Remember that a test that may
not have a good ending still has an ending. Also,
not finding the answer that was desired or expect-
ed is still a result (e.g., ‘The one-way ANOVA, F (2,
112) = 2.414, p = 0.101 did not show significant
differences between the age groups as the con-
ceptualization of the theory supported.’). With no
significant finding, the effect size or confidence in-
terval does not need to be reported. Depending on
the tests, it may be that a researcher decides to use
effect size over confidence intervals, or confidence
intervals over effect size, or both. If a researcher
does not find what was expected, the discussion
section is a good place to explain this.
Finally, it is important for a dissertation student to
stay in close communication with his or her Chair as
the results are analyzed and the findings are being
reported.

To be able to have more knowledge of how to con-

duct or write a results section, it is best to read how
other researchers have constructed their own results
sections, and learn how to write a clear and con-
cise results section. Explaining the findings of the
study is imperative. The researcher should cover the
null hypothesis, explain the type of statistical test,
the statistical significance of the testing, confidence
intervals, and the effect size. During this portion,
the written in-text wording of an APA formatted
write-up is necessary. This type of formatted write-
37 Back to Table of Contents
References and/or Suggested Reading:
Adams, K. A. & Lawrence, E. K. (2019). Student
study guide with IBM SPSS Workbook for research
methods, statistics, and applications (2nd ed.). Thou-
sand Oaks, CA: SAGE Publications.
American Psychological Association (2012). Publica-
tion Manual of the American Psychological Associa-
tion (6th ed.). Washington, DC: American Psycholog-
ical Association.

Creswell, J. W. & Creswell, J. D. (2018). Research

design: Qualitative, quantitative, and mixed methods
approaches (5th ed.). Thousand Oaks, CA: SAGE
Publications.

Elliott, A. C. & Woodward, W. A. (2020). Quick

guide to IBM SPSS: Statistical analysis with step-by-
step examples (3rd ed.). Thousand Oaks, CA: SAGE
Publications.

Back to Table of Contents 38