CHAPTER 2

Research Methods in
Industrial/Organizational Psychology
Inst. Aslı Yalçın
Çankaya University
Psychology Department
Social Scientific Research
Methods
 What qualities make a person an effective manager?
 Ask others? (get conflicting answers)
 Knowledge of tasks?
 Relationship with subordinates?
 Situation?
 Observe a good manager?
 How do you know who is a good manager?
 How will you determine the characteristics ?
 Scientific research methods do not rely on hunches or
beliefs, but on the systematic collection & analysis of
data
Social Scientific Research
Methods
 To determine the characteristics of a successful
work group manager;
 Define the criteria  Success
(successful- well-liked? successful- group productivity?)
 Measurement  Accurate and precise
(distinction btw. unsuccessful & successful managers can be made)
 Isolate the specific characteristics related to criteria 
(K, A, Ss or personality that make a successful manager)
Social Scientific Research
Methods
 Social scientific research methodology  a set of
procedures that allows us to investigate the HOWs and
WHYs of human behavior and to predict when certain
behavior will or will not occur.

 Social scientific research methods enable an I/O

psychologist to study a specific issue objectively.

 Objectivity is the unbiased approach to observation and

interpretations of behavior.
 Social Scientific Research Methods

 The goals of science; to describe, explain, & predict

phenomena.
 I/O psychology;
 Describe ( e.g. describing the production levels of a company, the rates of
employee turnover)
 Explain (e.g. employee turnover rates high bec. of dissatisfaction with the
payments)
 Predict work beh. (e.g. use scores from tests to predict which
employee would be the best candidate)
 As an applied science, I/O psychology also has the goal of
attempting to control or alter behavior to obtain desired
outcomes. ( e.g. İmplement a program to increase job satisfaction)
Social Scientific Research
Methods
1. The first step in conducting research is to specify the
problem or issue to be studied.
( rel. btw. worker job satisfaction & loyalty to organization)

II. The second step is to take those elements the

researcher intends to measure and develop
hypotheses.

 Variables are the elements measured in research

investigations. (e.g. Job satisfaction, loyalty to organization,
turnover, tenure)
 Hypotheses are statements about the supposed
relationships between or among variables.
 Social Scientific Research
Methods
 A theory  organization of beliefs into a representation of the
factors that affect behavior.

III. The third step; selecting the research design.

 depends upon several factors: research setting, the degree of control the
researcher has, the research questions, and the variables. (e.g.
observation, survey, experiment)

IV. The fourth step; data collection.

 Sampling is the selection of a representative group from a
larger population for study.
It is impossible to study all members of population!
( generalize the results obtained from sample  to population)
Social Scientific Research Methods
 Random sampling refers to the selection of research
participants from a population so that each individual has an
equal probability of being chosen.
(table of random numbers or a computer program )

 Stratified sampling is the selection of research participants

based on categories representing important distinguishing
characteristics of a population.

 Randomly select a specified number of people in a way that

research sample mirrors the actual breakdown of these groups
in the total population.
 Examples: Male and female, manager/non-manager as different stratas.
 40% female, 60% male in whole population; 25% manager, 75% non
manager it should be the same in your sample
 Social Scientific Research
Methods
 V. The fifth step in the research process is analysis of
data.
 Once the data are collected, they can then be analyzed.
 In most cases, quantitative data are analyzed using
statistical analysis. (also qualitative data)
 Statistics are used to describe data & to test hypotheses.

 Simple or complex statistics  Only the relationships

among the variables or analyses showing causality
 Social Scientific Research
Methods
 VI. The final step of the research process is the
interpretation of the results.
 The researcher draws conclusions about the meaning of
the findings and their relevance to actual work behavior,
as well as limitations of the current study and directions
for future research investigations.
 Example: Effects on work productivity of two managerial styles
(directive and non-directive/participative) in a manufacturing
company.
 Conclusion: directive style lead more productive groups
 Limitation: May only apply to managers of factory work groups;
not to managers of salespersons.
Major Research Designs
 Experimental Design: research design characterized by a
high degree of control over the research setting to allow for
the determination of cause-and-effect relationships among
variables.
 Experiments can be conducted in both labs. & fields.
 Independent variable (IV): the variable that is manipulated by the
researcher
 Dependent variable (DV) is the variable that is acted upon by the
independent variable (the outcome variable).
 No elements except the IV are allowed to vary.
(Any other variable that can affect the DV other than IV should be kept
constant or controlled)
 Statistically or by design
 The primary advantage of the experimental method 
Allows us to determine cause-and-effect relationships among
variables.
Major Research Designs
 treatment group: the group that is subjected to the change in the
independent variable
 control group: the group that receives no treatment or irrelevant
treatment.
 extraneous (confounding) variables: variables other than the
independent variable that may influence the dependent variable.

 Hold all extraneous variables constant!

 Ex: Hawthorne studies  attention of researchers

 Random assignment: assigning subjects to groups to control the

effects of extraneous variables.
 Easier in lab settings.
Major Research Designs
 Disadvantages of experimental design:
 Artificiality  A situation quite different from actual work
setting
 Generalizability problem (advantage of field experiment)

 Experimental Design Example:

 Effectiveness of a training program for salespersons
 100 employees randomly assigned to receive customer training & 100 employees
randomly assigned to no-customer training condition.
 Following the training program; look at the change in customer spending
 IV: The training program
 DV: customer spending
 Experimental Group: received training
 Control Group: received no training
Major Research Designs
 Quasi-experiment
 In many cases, a researcher does not have the control over
the situation needed to run a true experiment.
 Quasi-experiments involve comparison of pre-existing
groups, where random assignment of participants to
groups is not possible.

Differences from a true experiment:

 Random assignment and/or
 Manipulation of independent variable

Example: The job of a bank teller is redesigned in one branch of a

bank but not in another branch. After three months, customer
satisfaction found to be higher at the branch where the job
redesigned took place.
Major Research Designs
 The Correlational Design: examines the relationship between variables
as they occur naturally.
 The researcher observes two variables & measures their statistical
association with each other.
 No manipulation of variable!

 The main advantage: easier to implement in a particular setting,

including the workplace.
 Disadvantage: cause-effect rel. can not be drawn
 Reverse causality problem
e.g. organizational Satisfaction- purchasing organizational stock

 Third variable problem

e.g. employees with tenure
 
organizational satisfaction – purchasing organizational stock
Major Research Designs
1 2 3 4 5 6 7

1. İlişki Odaklı Liderlik 1

2. İş Odaklı Liderlik .01 1

3. Kişisel Yakınlık .49** -.01 1

4. Lidere Yönelik Etkinlik Algısı .58** .14 .35* 1

5. Lidere Duyulan Güven .42** .20 .43** .61** 1

6. İş Doyumu .49** .23 .36** .59** .54** 1

7. İş Performansı .01 .36* .12 .21 .18 .42** 1

Major Research Designs
 Meta-analysis
 Different research investigations of the same topic may reach
inconsistent (or contradictory) results.

 So, what conclusion?

 Meta-analysis; technique that allows results from several different
research studies to be combined &analyzed & summarized (at least
20 studies).

 Statistical procedure that combines the results of many independent

research findings on a single topic
 used to identify moderating variables
 estimate true relationship
 Measures effect size of findings
 Uses archival data
Major Research Designs
 Meta-analysis
 Meta-analyses rely on indicators of effect size & the number of
participants in each of the independent studies.

 Effect size: an estimate of the magnitude of the relationship

btw. X & Y (in corr. design ) or effect of IV on DV (in
experimental design) found in a research investigation.

 Example: Job Satisfaction – Absenteeism

 Example: One meta-analysis confirmed that more physically

demanding jobs were related to workers becoming stressed &
burnout.
Major Research Designs
 Case study
 Depth view of past events using observations, interviews &
archival records
 Qualitative descriptions of behavior.
 may provide rich, descriptive info. about work beh.s and
settings.
 Ex: In one case study; a psychologist found that company picnics &
other social activities increased employees’ loyalty to organization.
 Benefits:
 Detailed account of why particular event occurred
 Disadvantages:
 Little generalizability (anectodal)!
 No hypothesis testing, no cause-effect rel.s
Measurement of Variables
 Research variables are operationalized, or clearly defined

Abstract  Concrete

 Operational definition: a working definition of what the variable

is & how a variable will be manipulated/ measured in a study.

Ex: safe driving beh: wearing a seat belt, using a turn signal & coming to a
full stop at an intersection

 Two basic methods of measurement in I&O psychology

 Observation
 Self-Report
Measurement of Variables
 Observation: the researcher directly & systematically
observes certain beh.s
Ex: supervisory beh.s- demonstrating work techniques to subordinates,
giving direct orders.

 Obtrusive observation: research observation in which the

presence of the observer is known to the participants.
Disadvantage: the part. may behave diff.ly ( Hawthorne effect)

 Unobtrusive observation: observation in which the

presence of the observer is not known to the participants.
Disadvantage: ethical concerns about protecting the privacy of the part.s
Measurement of Variables
 Self-Report Techniques: measurement methods that rely on
research participants’ reports of their own behaviors or
attitudes.
 Interview: How do feel right now?.
 Survey: How often do you feel angry?
Disadvantage: distortion of the responses ( workers may give socially
desirable answers)
Interpreting and Using
Research Results
 Two key issues concerning measurement of variables:
 Reliability: consistency of the measure

 Test-retest reliability: people who take the test more than one
time should get the same scores.
Ex: the job applicant should get similar scores on the same tests taken
at two different points of time.

 Inter-rater reliability: the extent to which different people agree

on the characteristics they are measuring.

Ex: 1. I/O psychologist- worker’s beh.s  job satisfaction

2. I/O psychologist- worker’s beh.s  loyaty to organization

o
Interpreting and Using
Research Results
 Two key issues concerning measurement of variables:
 Validity: extent to which a measure assesses what it claims to
measure.
Ex: a job performance test should really test an employee’s job performance

 Internal validity: the extent to which extraneous/ confounding

variables are removed.
the extent to which cause&effect conclusions can be drawn.
 External validity: the extent to which the results can be generalized to
the different settings, different people.

 Reliability is necessary but not sufficient for validity.

Ethical Issues in Research and Practice in
I/O Psychology
 The American Psychology Association (APA) lists several
core principles that should guide the ethical conduct of
researchers in psychology, including I/O psychology.

 Do no harm & strive for the benefit of participants

 Be honest & accurate in science, teaching and practice of psychology
 Respect the rights of people to privacy and confidentiality.

 Informed Consent: a research participant is fully informed of the nature,

purpose, duration and procedures of the study and ensure that the part. has
the right to not participate. (confıdentiality & anonymity of the responses)
 Debriefing: part.s should be fully debriefed & ensure that no harm has
been caused.
Statistical Analyses of Research Data
 The science & the practice of I/O psychology require a
throughknowledge of research methods and statistics.

 Two types of statistics:

 Descriptive Statistics
 Inferential Statistics

 Descriptive Statistics: numerical characteristics of the nature of

the data set.
 Arithmetical formulas for summarizing & describing research
data

 What is the avarage tenure in your sample?

 How many female employees? How many male employees?
Statistical Analyses of Research Data
o Three major types of descriptive statistics;
1. Frequency Distribution: A descriptive statistical technique that presents data in a
useful format, arranging the scores by category
In the form of bar graph or histogram
Statistical Analyses of Research Data
o Three major types of descriptive statistics;
II. Central Tendency: where the group tends to cluster
 Mean, Median, Mode
EXAMPLE
EXAMPLE
Data:
Data:1,1,1,1,1,2,2,3,3,3,3,5,5
1,1,1,1,1,2,2,3,3,3,3,5,5
Mean
Mean ==2.38;
2.38;Median
Median==2;2;Mode
Mode==11
 Mean
 Arithmetic average
 Median
 The score at the midpoint of a statistical distribution
 Mode
 The most frequently obtained score in the distribution of data
 Statistical Analyses of Research Data
o Three major types of descriptive statistics;
III. Variability: How scores varies among the part.s
 Range - distance between highest and lowest score
 (Range = High score – Low score)
 Range = 17 – 5 = 12

 Standard Deviation – average distance from the mean

 S= Σ(x
(less– susceptible
xm)2 / n – to
1 extreme scores!)
(sample)

S = (5-9) 2 + (6-9) 2 + (6-9) 2 + (8-9) 2 + (9-9) 2 + (10-9) 2 + (11-9) 2 + (17-9) 2 / 7

S = 3.85
 Statistics: The Language of Research
Range vs. Standard Deviation
mean

(
Sample Distributions
 Normal Distribution
 Bell-shaped with most scores falling toward the middle with few
at high and low extremes
 Mean, median and mode are equal

 Skewed Distribution
 Asymmetrical with more scores falling closer to high or low
extremes
 Median is most useful
measure of central tendency
because it is less affected
by extreme scores
The Normal Distribution of IQ
 Inferential Statistics
 Methods for analyzing data that express relationships (e.g.,
differences between groups) in terms of probabilities
 Statistical significance
 The level of confidence we have in the results of an experiment based
on probability values
 Probability
 The idea that a difference between the means of experimental and
control groups could have occurred by chance
 If P=.01, a difference would occur by chance only 1 time
out of 100.

E.g. A safety training program reduced the accident rates?

Control group  no training
Treatment group  safety training
Whether the differences btw. groups meaningful?
Statistical Analyses of Experimental
Method Data
 T-test: A statistical test for comparing the means of two
groups
 whether a process or treatment has an effect ?
 whether two groups are different from one another?
IVCat. DV  Con.

 Analysis of Variance (ANOVA): Differences among

more than two groups on a single variable
IVCat. DV  Con.

 Multivariate Analysis of Variance (MANOVA):

Examines data from multiple groups with multiple DVs.
 whether there are any differences between groups on more than one
dependent variable?
IVCat. DV  Con.
Statistical Analyses of Experimental
Method Data
Statistical Analysis of Correlational Methods

Correlation Coefficient Multiple Regression

 Multiple Regression
 Effects of one variable over another (used for prediction)

Yi = ß0 + ß1Xi1 + ß2Xi2 (Y = a + bX)

 Regression Variables
- Predictor Variable (X) – measure used to predict an
outcome (similar to independent variable)
E.g. selection test scores, years of experience, education level

- Criterion Variable (Y) – outcome to be predicted

E.g. work performance, turnover, sales, absenteeism, promotion, etc.
IVCon. DV  Con.
Statistical Analysis of Correlational Methods

 Multiple Regression
Statistical Analyses of Experimental
Method Data
 Multiple Regression
 Main effects

 Interaction effects  When the effect of one IV on

the dependent variable differs depending on the level of
the second independent variable.

E.g: Leadership context (military, business, politics) X

Perceived leadership style (ROL, TOL)

 effectiveness (Goncu, Kastendieck, & Johnson, 2009)