Psychology 27200

Research Methods in I-O Psychology

Part 1

Dr. Franki Kung

Today’s Class
What is Science?
• Five steps in scientific method
From theory to hypothesis
• Four roles of a variable
Study Design
• Difference between correlation and causation
• Three types of research design
• Internal and external validity of research findings
Why should I care about
scientific methods & analyses?
• I-O psychology is a science (Chapter 2)
• I-O Psychology uses a scientist-practitioner model of training
and emphasizes evidence-based research (Chapter 1)
A Job Description
A Job Description
Why should I care about
scientific methods & analyses?
• I-O psychology is a science (Chapter 2, pp.50-53)
• I-O Psychology uses a scientist-practitioner model of training
and emphasizes evidence-based research (Chapter 1, pp. 11-12)

• Research design, data collection, data analysis and

interpretation is a large part of I-O psychologists’ job
What is Science?
• A logical approach to investigation, usually based on a theory,
a hypothesis, or a simple curiosity about an object of interest
• Depends on data
• Goals: use data to describe, explain, and predict events
• Must be communicable, open, and public
• Sets out to disprove theories/hypotheses
• Free from biases or prejudices (disinterestedness)
 Five Steps in Scientific Method
Identify and observe
phenomenon

Generate (or modify) theory

What is a theory?
Develop hypothesis

Design study

Collect & analyze data

 Five Steps in Scientific Method
Identify and observe Theory:
phenomenon • Integrated set of principles
that describe, explain, and
predict observed events
Generate (or modify) theory • Summarizes and organizes
knowledge
• Can be mathematical or
Develop hypothesis verbal

Hypothesis:
Design study
• Research question that is
generated from theory and
can be tested in a data set
Collect & analyze data
 Five Steps in Scientific Method
Identify and observe Theory:
phenomenon • Integrated set of principles
that describe, explain, and
predict observed events
Generate (or modify) theory • Summarizes and organizes
knowledge
• Can be mathematical or
Develop hypothesis verbal

Hypothesis:
Design study
• Research question that is
generated from theory and
can be tested in a dataset
Collect & analyze data
 Five Steps in Scientific Method
Identify and observe
phenomenon

Generate (or modify) theory

Develop hypothesis
Today’s class

Design study
Next class

Collect & analyze data

Today’s Class
What is Science?
• Five steps in scientific method
From theory to hypothesis
• Four roles of a variable
Study Design
• Difference between correlation and causation
• Three types of research design
• Internal and external validity of research findings
 Theory to Hypothesis

Latent
Care for Employee
variables or
employees attitudes
constructs

Observed Good pay, Smiles at work,

variables vacation days, volunteer work hours,
tuition support… trust…

Operationalizations
Developing Hypothesis
• Hypothesis
• A testable statement about the status of a variable or the
relationship among multiple variables

• Using Starbucks’ college achievement plan as an example…

Developing Hypothesis
• The Starbucks example…
• Sample Hypothesis 1: Starbucks’ college tuition support
increases employees’ trust toward the organization.

Tuition support Trust toward Org.

Developing Hypothesis
• The Starbucks example…
• Sample Hypothesis 1: Starbucks’ college tuition support
increases employees’ trust toward the organization.

Tuition support Trust toward Org.

• Sample Hypothesis 2: Employees’ trust toward the organization

increases job performance.
Trust toward Org. Job performance
Developing Hypothesis
• The Starbucks example…

Tuition support Trust toward Org.

Trust toward Org. Job performance

A hypothesis has multiple variables

Variables can play different roles:
1. Independent Variable (IV: Predictor)
2. Dependent Variable (DV: Outcome or Criterion)
Developing Hypothesis
• The Starbucks example…

Tuition support Trust toward Org.

Trust toward Org. Job performance

A hypothesis has multiple variables

Variables can play different roles:
1. Independent Variable (IV: Predictor)
2. Dependent Variable (DV: Outcome or Criterion)
 Developing Hypothesis
• Building an explanation in a hypothesis
Example (combining the two prior hypotheses):
• Starbucks’ college tuition support increases employees’ trust
toward the organization, which in turn increases employee’s job
performance

Tuition support Trust toward Org. Job performance

3rd possible role of a variable: Mediator

 Developing Hypothesis
• Adding contingency into a hypothesis
Example:
• Starbucks’ college tuition support increases employees’ trust
toward the organization, which in turn increases employee’s job
performance. The effect of tuition support on trust should be
particularly strong for students from low-income families.

Tuition support Trust toward Org. Job performance

Family income

4th possible role of a variable: Moderator

Developing Hypothesis
• Hypothesis
• A testable statement about the status of a variable or the
relationship among multiple variables

• Variables
1. Independent Variable (IV: Predictor)
2. Dependent Variable (DV: Outcome or Criterion)
3. Mediator
4. Moderator
Four (Basic) Types of Variables

IV DV

IV DV
Moderator

IV Mediator DV
General Psych Model

Person Behavior/
Thinking/Feeling
(e.g., personality) Performance

Context
(e.g., job situation)
Today’s Class
What is Science?
• Five steps in scientific method
From theory to hypothesis
• Four roles of a variable
Study Design
• Difference between correlation and causation
• Three types of research design
• Internal and external validity of research findings
Activity: Thought Experiment
Let’s say a researcher has observed that:
• Purdue students who take a higher number of Psych course also
report a higher score on agreeableness

Does it mean that Psych course makes students more

agreeable?

What about an observation that:

• The more a worker smokes, the more the worker has interpersonal
problems at work

Does it mean that smoking causes interpersonal problems at

work?
Developing Hypothesis
• Correlation vs. Causation
• Correlation does not imply causation!
• XY
• YX
• Z (Third variable) is the cause for both X and Y
• The correlation between X and Y is completely
coincidental
From Developing Hypothesis to Designing a Study

• Conditions for causation:

• Correlation
• The variables must be related to each other
• Temporal (Time) Order
• “Cause” must precede “Effect”
• Eliminates problem of reverse direction of causation
• Elimination of Alternative Explanations
• Can only be guaranteed in an experiment with random
assignment to conditions
• Eliminates problem with third variable as cause
 Scientific Method
Identify and observe Experimental Design:
phenomenon • Participants are randomly
assigned to different
conditions
Generate (or modify) theory • Study causal relationship
between experimental
treatment and outcome

Develop hypothesis
Non-experimental Design:
• No random assignment
Design study • Study correlational
relationship among
variables
Collect & analyze data
Designing a Study
• Textbook: Three basic types of research design
• Experimental
• Quasi-experimental
• Non-experimental
Research Design
• Experimental design: randomly assign participants to
conditions, vary treatment in each condition, and measure
outcome in each condition
• used to detect cause-effect relationships
Research Design
• Experimental design: randomly assign participants to
conditions, vary treatment in each condition, and measure
outcome in each condition
• used to detect cause-effect relationships
• Two essential elements:
• Random assignment: guaranteeing that subjects are
placed in experimental conditions on the basis of
chance
• Manipulation of variable (Intervention/Treatment)
• Independent variable: the factor that is systematically
manipulated by the researcher.
• Dependent variable: the response to or effect of the
independent variable.
Experimental Design

Participants Random Assignment:

Each member of
sample has equal
probability of being
Independent Variable assigned to each
condition/ treatment.
Experimental
Control Condition
Condition

Dependent Variable
Research Design

• Quasi-experiments: we manipulate the independent variable,

but cannot randomly assign individuals to conditions (e.g., we
cannot assign workers to different manufacturing plants)
• Participants are still assigned to experimental and control groups to
allow a test for between-group differences, but random
assignment is not possible

Common Examples: Placement locations, training groups.

Research Design

• Non-experimental (correlational): include neither the

manipulation of variables nor random assignment
• Survey or observational research
• Primarily examining the relationship of variables involved.
• Because of limited control, it is difficult to infer causation
• Most common type of study in I/O
Experimental vs. Correlational Methods
• In I-O psychology, both experimental and correlational
methods are used to increase individual job performance
and organizational effectiveness.

• What’s the rationale behind:

• Selecting motivated candidates as employees (correlational
relationship)?
• Designing rewarding system to increase everyone’s motivation
(causal relationship)?

• The greatest social benefit will come from I-O psychology

if we combine experimental and correlational methods.
 Scientific Method
Identify and observe Quantitative Methods:
phenomenon • College and analyze data
from tests, rating scales,
questionnaires, and/or
physiological measures that
Generate (or modify) theory yield numerical results.

Develop hypothesis Qualitative Methods:

• College and analyze data
from observations,
Design study interviews, case studies,
diaries or written
documents that include
narrative descriptions of
Collect & analyze data events or processes.
Data Collection
• Quantitative research: involves tests and measures that
yield numerical results
• “How satisfied are you with your job on a scale of 1 to 5?”

• Qualitative research: involves observation, interviews,

case studies, diaries, etc. that produce descriptions of a
phenomenon
• “Tell me what characteristics are important for your satisfaction in
your job.”

• Both methods are important. It is best to collect data

from multiple sources, using multiple methods
(triangulation).
 Scientific Method
Identify and observe
phenomenon

Generate (or modify) theory

Develop hypothesis

Design study

Collect & analyze data

Generating (or Modifying) Theory
• Our findings provide feedback for us to correct errors
and modify our theories:
• Errors of Omission: Relationships exist but are not specified in the
theory.
• Errors of Commission: Relationships do not exist but are specified
in the theory.
• When several theories explain the phenomena in question equally
well, the theory that is the most parsimonious (that is, the one
with the fewest assumptions) is better. (Occam's Razor)
Validity of the Research Findings
• Internal Validity
• Concerns causality
• Higher when stronger evidence exists to rule out alternative
explanations
• Thus, a function of random assignment to conditions/treatments

• External Validity
• Concerns generalizability
• Higher when stronger evidence exists to show the findings generalized
to the entire population of interest
• Thus, a function of having representative sample of study participants, or
random sampling
• Beyond study sample: Time-frame, situational characteristics, and
cultures also matter.

• Note: Do not confuse validity of the findings of a research study with

validity of a measure/scale/questionnaire!!!
 Random Sampling: Each
Population of member of population has
interest equal probability of being in
sample. Increases External
Validity.
Participants Random Assignment:
Each member of
sample has equal
probability of being
Independent Variable assigned to each
condition/ treatment.
Experimental Increases Internal
Control Condition
Condition Validity.

Dependent Variable
Note: Do not confuse random assignment and random sampling!!!
Research Designs & Research Settings

External Validity/Generalizability
Low High

Setting

Artificial Natural

High Lab experiment Field experiment

Internal
Validity/
Quasi-experiment
Control
Low Survey/Questionnaire
Observation
Before next class…
• First Study Group coming up soon (check Brightspace for info)
• Practice Game 1 due Sep 3rd 11:59pm ET
• Remember to use your Purdue Email address!
• HW1 due 11:59pm on Sep 10th 11:59pm
• Quiz 1 begins on Wednesday (Sep 14th) due Sep 17th 11:59pm ET
• Include Lecture 0-2 and Book Chapters 1-2
• Make sure you refer to the syllabus: Class Schedule

Start preparing for the Quiz! 