Research, Methodology 3 Christ

8-1
Unit 3 : Data Collection, measurement and scaling
Hours: 10
■Qualitative methods of data collection,

■attitude measurement and scaling-
■types of measurement scale,
■attitude,
■ classification of scales –
■Single Item v/s Multi Item Scales,
■Comparative v/s Non-Comparative scales,
■Continuous Rating Scales;
■Criteria for Good Measurement,
■questionnaire design
8-2
Basic Measurement Issues
Measurement is the process of assigning numbers

or labels to objects, persons, states, or events in
accordance with specific rules to represent
quantities or qualities of attributes.
We do not measure specific objects, persons, etc.,
we measure attributes or features that define
them.
Ex., What defines the person Brent Wren? What is
a student’s level of education? How customer
oriented is our company?
Overriding Goal: To provide a valid and reliable
description or enumeration of the person, objects,
issue, etc.
8-3
Measurement Process
1. Define concepts to be measured

2. Define attributes of the concepts
3. Select scale of measurement (data type)
4. Generate Items/Questions
■ Wording
■ Response format
5. Layout and design questionnaire
6. Pretest and refine
8-4
Basic Measurement Issues

■ Concept or Construct
■ A generalized idea about a class of objects,
attributes, occurrences, or processes
■ Concrete – demographics, traffic patterns,
purchase quantity
■ Abstract – loyalty, personality, satisfaction,
leadership
■ Attribute
■ A single characteristic or fundamental feature of
an object, person, situation, or issue
■ Often measure multiple attributes
8-5
1) Overview
2) Measurement and Scaling
3) Primary Scales of Measurement
i. Nominal Scale
ii. Ordinal Scale
iii. Interval Scale
iv. Ratio Scale
4) A Comparison of Scaling Techniques
8-6
Chapter Outline
5) Comparative Scaling Techniques
i. Paired Comparison
ii. Rank Order Scaling
iii. Constant Sum Scaling
iv. Q-Sort and Other Procedures
6) Verbal Protocols
7) International Marketing Research
8) Ethics in Marketing Research
8-7
Measurement and Scaling

Measurement means assigning numbers or other
symbols to characteristics of objects according to
certain prespecified rules.
■ One-to-one correspondence between the numbers
and the characteristics being measured.

■ The rules for assigning numbers should be
standardized and applied uniformly.

■ Rules must not change over objects or time.
8-8
Measurement and Scaling

Scaling involves creating a continuum upon which
measured objects are located.
Consider an attitude scale from 1 to 100. Each

respondent is assigned a number from 1 to 100, with
1 = Extremely Unfavorable, and 100 = Extremely
Favorable. Measurement is the actual assignment of
a number from 1 to 100 to each respondent. Scaling
is the process of placing the respondents on a
continuum with respect to their attitude toward
department stores.
8-9
Primary Scales of Measurement

Scale
Nominal Numbers Finish
Assigned
7 8 3
to Runners
Ordinal Rank Order Finish

of Winners
Third Second First
place place place
Interval Performance
Rating on a 8.2 9.1 9.6
0 to 10 Scale
Ratio Time to
15.2 14.1 13.4
Finish, in
Seconds
8-10
Nominal Scale
■ The numbers serve only as labels or tags for

identifying and classifying objects.
■ When used for identification, there is a strict
one-to-one correspondence between the numbers
and the objects.
■ The numbers do not reflect the amount of the
characteristic possessed by the objects.
■ The only permissible operation on the numbers in a
nominal scale is counting.
■ Only a limited number of statistics, all of which are
based on frequency counts, are permissible, e.g.,
percentages, and mode.
8-11
Illustration of Primary Scales of Measurement
Nominal Ordinal Interval Ratio

Scale Scale Scale Scale
Preference Preference
$ spent last
No. Store Rankings Ratings 3 months
1-7 11-17
1. Lord & Taylor
2. Macy’s
3. Kmart
4. Rich’s
5. J.C. Penney
6. Neiman Marcus
7. Target
8. Saks Fifth Avenue
9. Sears
10.Wal-Mart
8-12
Ordinal Scale
■ A ranking scale in which numbers are assigned to
objects to indicate the relative extent to which the
objects possess some characteristic.
■ Can determine whether an object has more or less of
a characteristic than some other object, but not how
much more or less.
■ Any series of numbers can be assigned that
preserves the ordered relationships between the
objects.
■ In addition to the counting operation allowable for
nominal scale data, ordinal scales permit the use of
statistics based on centiles, e.g., percentile, quartile,
median.
8-13
Interval Scale
■ Numerically equal distances on the scale represent
equal values in the characteristic being measured.
■ It permits comparison of the differences between
objects.
■ The location of the zero point is not fixed. Both the
zero point and the units of measurement are
arbitrary.
■ Any positive linear transformation of the form y = a
+ bx will preserve the properties of the scale.
■ It is meaningful to take ratios of scale values.
■ Statistical techniques that may be used include all of
those that can be applied to nominal and ordinal
data, and in addition the arithmetic mean, standard
deviation, and other statistics commonly used in
marketing research.
8-14
Ratio Scale
■ Possesses all the properties of the nominal, ordinal,

and interval scales.
■ It has an absolute zero point.
■ It is meaningful to compute ratios of scale values.
■ Only proportionate transformations of the form y =
bx, where b is a positive constant, are allowed.
■ All statistical techniques can be applied to ratio data.
8-15

8-16
Criteria for Scale Selection
■ Understanding of the questions

■ Discriminatory power of scale descriptors
■ Balanced versus unbalanced scales
■ Forced or non-forced choice scales
■ Desired measure of central tendency and
dispersion
8-17
Central Tendency and Dispersion
■ Central Tendency ■ Dispersion

■ Mode ■ Frequency distribution
■ Median ■ Range
■ Mean ■ Standard deviations

8-18
Central Tendency and Dispersion
■ If a nominal scale is used, analysis of raw data can only

be done using modes and frequency distributions
■ If ordinal scales are used, analysis of raw data can be
done using medians and ranges (plus modes and frequency
distributions)
■ If interval or ratio scales are used, analysis of raw data
can be done through the use of sample means and
estimated standard deviations as the sample statistic (plus
the above)
8-19
A Classification of Scaling Techniques
Scaling Techniques
Comparative Noncomparative
Scales Scales
Paired Rank Constant Q-Sort and Continuous Itemized

Comparison Order Sum Other Rating Scales Rating Scales
Procedures
Semantic Stapel
Likert
Differential
8-20
A Comparison of Scaling Techniques

■ Comparative scales involve the direct comparison of stimulus
objects. Comparative scale data must be interpreted in relative terms
and have only ordinal or rank order properties.
■ Comparative Rating Scales

■ format that requires a judgment comparing one object, person, or
concept against another on the scale
■ In noncomparative scales, each object is scaled independently of

the others in the stimulus set. The resulting data are generally
assumed to be interval or ratio scaled.
■ Noncomparative Rating Scales

■ format that requires a judgment without reference to another
object, person, or concept
8-21
A COMPARATIVE RATING SCALE
A COMPARATIVE RATING SCALE ASKS RESPONDENTS TO RATE A

CONCEPT BY COMPARING IT WITH A BENCHMARK.
Please indicate how the amount of authority in your present position

compares with the amount of authority that would be ideal for this
position.
TOO MUCH ABOUT RIGHT TOO LITTLE

8-22
Relative Advantages of Comparative Scales

■ Small differences between stimulus objects can be
detected.
■ Same known reference points for all respondents.
■ Easily understood and can be applied.
■ Involve fewer theoretical assumptions.
■ Tend to reduce halo or carryover effects from one
judgment to another.
Relative Disadvantages of Comparative Scales
■ Ordinal nature of the data
■ Inability to generalize beyond the stimulus objects
scaled.
8-23
Comparative Rating Scales

8-24
Comparative Scaling Techniques
Paired Comparison Scaling
■ A respondent is presented with two objects and

asked to select one according to some criterion.
■ The data obtained are ordinal in nature.
■ Paired comparison scaling is the most widely used
comparative scaling technique.
■ With n brands, [n(n - 1) /2] paired comparisons are
required
■ Under the assumption of transitivity, it is possible to
convert paired comparison data to a rank order.
8-25
Obtaining Shampoo Preferences
Using Paired Comparisons
Instructions: We are going to present you with ten pairs of

shampoo brands. For each pair, please indicate which one of the
two brands of shampoo you would prefer for personal use.
Recording Form:
a
A 1 in a particular box means that the brand in that column was preferred
over the brand in the corresponding row. A 0 means that the row brand was
preferred over the column brand. bThe number of times a brand was preferred
is obtained by summing the 1s in each column.
8-26
Rank Order Scaling
■ Respondents are presented with several objects

simultaneously and asked to order or rank them
according to some criterion.
■ It is possible that the respondent may dislike the
brand ranked 1 in an absolute sense.
■ Furthermore, rank order scaling also results in ordinal
data.
■ Only (n - 1) scaling decisions need be made in rank
order scaling.
8-27
Preference for Toothpaste Brands
Using Rank Order Scaling
Instructions: Rank the various brands of toothpaste in order

of preference. Begin by picking out the one brand that you like
most and assign it a number 1. Then find the second most
preferred brand and assign it a number 2. Continue this
procedure until you have ranked all the brands of toothpaste
in order of preference. The least preferred brand should be
assigned a rank of 10.
No two brands should receive the same rank number.
The criterion of preference is entirely up to you. There is no
right or wrong answer. Just try to be consistent.
8-28
Preference for Toothpaste Brands
Using Rank Order Scaling
Form
Brand Rank Order
1. Crest _________
2. Colgate _________
3. Aim _________
4. Gleem _________
5. Macleans _________
6. Ultra Brite _________

7. Close Up _________
8. Pepsodent _________
9. Plus White _________
10. Stripe _________
8-29
Constant Sum Scaling
■ Respondents allocate a constant sum of units, such

as 100 points to attributes of a product to reflect
their importance.
■ If an attribute is unimportant, the respondent assigns
it zero points.
■ If an attribute is twice as important as some other
attribute, it receives twice as many points.
■ The sum of all the points is 100. Hence, the name of
the scale.
8-30
Importance of Bathing Soap Attributes
Using a Constant Sum Scale
Instructions
On the next slide, there are eight attributes of
bathing soaps. Please allocate 100 points among
the attributes so that your allocation reflects the
relative importance you attach to each attribute.
The more points an attribute receives, the more
important the attribute is. If an attribute is not at
all important, assign it zero points. If an attribute is
twice as important as some other attribute, it
should receive twice as many points.
8-31
Importance of Bathing Soap Attributes
Using a Constant Sum Scale
Form
Average Responses of Three Segments
Attribute Segment I Segment II Segment III
1. Mildness
2. Lather
3. Shrinkage
4. Price
5. Fragrance
6. Packaging
7. Moisturizing
8. Cleaning Power
Sum
Attitude Measurement
ATTITUDE
AN ENDURING DISPOSITION TO
CONSISTENTLY RESPOND IN A
GIVEN MANNER
8-34
Attitudes as Hypothetical Constructs
● The term hypothetical construct is

used to describe a variable that is
not directly observable, but is
measurable by an indirect means
such as verbal expression or overt
behavior - attitudes are considered
to be such variables.
8-35
THREE COMPONENTS OF AN
ATTITUDE
■ AFFECTIVE
■ COGNITIVE
■ BEHAVIORAL
AFFECTIVE
THE FEELINGS OR EMOTIONS

TOWARD AN OBJECT
8-37
COGNITIVE
■ KNOWLEDGE AND BELIEFS

8-38
BEHAVIORAL
■ PREDISPOSITION TO ACTION
■ INTENTIONS
■ BEHAVIORAL EXPECTATIONS
8-39
MEASURING ATTITUDES
■ RANKING
■ RATING
■ SORTING
■ CHOICE
8-40
The Attitude Measuring Process
RANKING - Rank order preference

RATING - Estimates magnitude of a
characteristic
SORTING - Arrange or classify concepts
CHOICE - Selection of preferred
alternative
8-41
● Ranking tasks require that the

respondent rank order a small
number of objects in overall
performance on the basis of
some characteristic or stimulus.
8-42
● Rating asks the respondent to

estimate the magnitude of a
characteristic, or quality, that an
object possesses. The respondent’s
position on a scale(s) is where he or
she would rate an object.
8-43
● Sorting might present the respondent with

several concepts typed on cards and
require that the respondent arrange the
cards into a number of piles or otherwise
classify the concepts.
● Choice between two or more alternatives
is another type of attitude measurement -
it is assumed that the chosen object is
preferred over the other.
8-44
● Physiological measures of attitudes

provide a means of measuring attitudes
without verbally questioning the
respondent. For example, galvanic skin
responses, blood pressure
measurement, etc.
8-45
Simple Attitude Scaling

● In its most basic form, attitude scaling
requires that an individual agree with a
statement or respond to a single
question. This type of self-rating scale
merely classifies respondents into one
of two categories.
8-46
SIMPLIFIED SCALING EXAMPLE
THE PRESIDENT SHOULD RUN FOR RE-ELECTION

_______ AGREE ______ DISAGREE
8-47
Category Scales
● A category scale is a more sensitive
measure than a scale having only two
response categories - it provides more
information.
8-48
EXAMPLE OF CATEGORY SCALE

How important were the following in your decision to visit San
Diego? (check one for each item)
VERY SOMEWHAT NOT TOO
IMPORTANT IMPORTANT IMPORTANT
CLIMATE ___________ ___________ ___________

COST OF TRAVEL ___________ ___________ ___________
FAMILY ORIENTED ___________ ___________ ___________
EDUCATIONAL
/HISTORICAL ASPECTS _________ ___________ ___________
FAMILIARITY WITH
AREA ___________ ___________ ___________
8-49
Method of Summated Ratings:

The Likert Scale
● An extremely popular means for measuring

attitudes. Respondents indicate their own
attitudes by checking how strongly they
agree or disagree with statements.
● Response alternatives: “strongly agree”,
“agree”, “uncertain”, “disagree”, and
“strongly disagree”.
8-50
Likert Scale
A likert scale is
an ordinal scale format
that asks respondents
to indicate the extent to which
they agree or disagree with a series of
mental or behavioral belief
statements about a given object
8-51
LIKERT SCALE FOR MEASURING
ATTITUDES TOWARD TENNIS
It is more fun to play a tough, competitive
tennis match than to play an easy one.
___Strongly Agree
___Agree
___Not Sure
___Disagree
___Strongly Disagree
8-52
LIKERT SCALE FOR MEASURING ATTITUDES

TOWARD TENNIS Cont.
There is really no such thing as a tennis

stroke that cannot be mastered.
___Strongly Agree
___Agree
___Not Sure
___Disagree
8-53
LIKERT SCALE FOR MEASURING

ATTITUDES TOWARD TENNIS
Cont.
Playing tennis is a great way to exercise.

___Strongly Agree
___Agree
___Not Sure
___Disagree
8-54
Semantic Differential Scale
A semantic differential scale is

unique bipolar ordinal scale format
that captures a person’s attitudes
and/or feelings about a given
object
8-55
Semantic Differential
● A series of seven-point bipolar rating
scales. Bipolar adjectives, such as
“good” and “bad”, anchor both ends (or
poles) of the scale.
8-56
● A weight is assigned to each position on

the rating scale. Traditionally, scores
are 7, 6, 5, 4, 3, 2, 1, or +3, +2, +1, 0,
-1, -2, -3.
8-57
SEMANTIC DIFFERENTIAL SCALES

FOR MEASURING ATTITUDES
TOWARD TENNIS
Exciting ___ : ___ : ___ : ___ : ___ : ___ : ___ :
Calm
Interesting ___ : ___ : ___ : ___ : ___ : ___ : ___ :
Dull
Simple ___ : ___ : ___ : ___ : ___ : ___ : ___ Complex
Passive ___ : ___ : ___ : ___ : ___ : ___ : ___

Active
8-58
Behavioral Intention Scale

A behavioral intention scale is
a special type of rating scale
designed
to capture the likelihood that
people
will demonstrate some type of
predictable behavior intent
toward purchasing an object or
service
in a future time frame
8-59
Numerical Scales
● Numerical scales have numbers as
response options, rather than “semantic
space’ or verbal descriptions, to identify
categories (response positions).
8-60
Stapel Scales
● Modern versions of the Staple scale place a
single adjective as a substitute for the
semantic differential when it is difficult to
create pairs of bipolar adjectives.
● The advantage and disadvantages of a
Stapel scale, as well as the results, are very
similar to those for a semantic differential.
However, the stapel scale tends to be
easier to conduct and administer.
8-61
A Stapel Scale for Measuring a
Store’s Image
Department
Store Name
+3
+2
+1
Wide Selection
-1
-2
-3
8-62
Select a plus number for words that you think

describe the store accurately. The more
accurately you think the work describes the
store, the larger the plus number you should
choose. Select a minus number for words
you think do not describe the store
accurately. The less accurately you think the
word describes the store, the large the minus
number you should choose, therefore, you
can select any number from +3 for words
that you think are very accurate all the way
to -3 for words that you think are very
inaccurate.
8-63
Paired Comparisons:
In paired comparisons the respondents are

presented with two objects at a time and
asked to pick the one they prefer. Ranking
objects with respect to one attribute is not
difficult if only a few products are compared,
but as the number of items increases, the
number of comparisons increases
geometrically (n*(n -1)/2). If the number of
comparisons is too great, respondents may
fatigue and no longer carefully discriminate
among them.
8-64
Rank Order Scales

■ Rank the following in order of preference, 1
being the most preferred and 3 the least:
■ Brand A _________
■ Brand B _________
■ Brand C _________
8-65
Rank Order Scales

● easy to understand and use, ordinal
scales, comparative data, number
and kinds of alternatives could be a
problem and difficult to know how
far apart.
8-66
Constant Sum Scale

■ Divide 100 points among each of the
following brands according to your preference
for the brand:
■ Brand A _________
■ Brand B _________
■ Brand C _________
8-67
Constant Sum Scale

Magnitude of each alternative known
Equal value can be indicated.
Too many can confuse

8-68
Graphic Rating Scales

● A graphic rating scale presents
respondents with a graphic continuum.
8-69
Graphic Rating Scale Stressing
Pictorial Visual Communications
3 2 1
Very Very
Good Poor
8-70
Graphic Rating Scale

8-71
Graphic Rating Scales
● easy to construct and simple to

use, can discern fine distinctions,
but not very reliable, can be
treated as interval data.
8-72
Purchase Intent Scale

● How likely are you to buy this product
in the next six months.
Will definitely buy ____
Will probably buy ____
Will probably not buy ____
Will definitely not buy ____
● Can be weighted to get probable
market share.
8-73
MONADIC RATING SCALE
A MONADIC RATING SCALE ASKS ABOUT A SINGLE CONCEPT
Now that you’ve had your automobile for about 1 year, please tell us
how satisfied you are with its engine power and pickup.
Completely Very Fairly Well Somewhat Very

Satisfied Satisfied Satisfied Dissatisfied Dissatisfied
8-74
A UNBALANCED SCALE
A UNBALANCED SCALE HAS MORE RESPONSES DISTRIBUTED AT

ONE END OF THE SCALE.
How satisfied are you with the bookstore in the Student Union?
Neither Satisfied Quite Very

Satisfied Nor Dissatisfied Dissatisfied Dissatisfied
Reliability and Validity
Reliability
When a Measurement Procedure yields
consistent scores when the phenomenon
being measured is not changing.
Degree to which scores are free of
“measurement error”
Consistency of measurement
VALIDITY
• The extent to which measures indicate what
they are intended to measure.
• The match between the conceptual
definition and the operational definition.
RELATIONSHIP BETWEEN
RELIABILITY AND VALIDITY
• Necessary but not sufficient

• Reliability is a prerequisite for
measurement validity
• One needs reliability, but it’s not
enough
Example
• Measuring height with reliable bathroom
scale
• Measuring “aggression” with observer
agreement by observing a kid hitting a Bobo
doll
Types of Reliability
Measurement
• 1. Stability Reliability
• 2. Equivalence Reliability
Stability Reliability
• Test-retest
• SAME TEST – DIFFERENT TIMES
• Testing phenomenon at two different
times;
• The degree to which the two
measurements of “Sam Ting,” using same
measure, are related to one another
• Only works if phenomenon is unchanging
Example of Stability
• Administering same questionnaire at 2
different times
• Re-examining client before deciding on
intervention strategy.
• Running “trial” twice (e.. g. errors in tennis
serving)
Notes on Stability Reliability
• When ratings are by an observer rather
than the subjects themselves, this is called
Intraobserver Reliability or Intrarater
Reliability.
• Answers about the past are less reliable
when they are very specific, because the
questions may exceed the subjects’ capacity
to remember accurately.
Equivalence Reliability
1. Inter-item (split ½)
2. Parallel forms [Different types of
measures]
3. Interobserver Agreement
-Is every observer scoring the same ?
1. Inter-item Reliability
• (Internal consistency): The association of

answers to a set of questions designed to
measure the same concept.
2. Parallel forms of Reliability
• Split ½ (inter-item)
• Different types of measures
• Inter-observer Reliability
– Is everyone measuring the same thing ?
• Different measures – same time
3.Interobserver Reliability
• Correspondence between measures

made by different observers.
Types of Validity
• 1. Content Validity
– Face Validity
– Sampling Validity (content validity)
• 2. Empirical Validity
– Concurrent Validity
– Predictive Validity
• 3. Construct Validity
Face Validity
• confidence gained from careful inspection of
a concept to see if it’s appropriate “on its
face;”
• In our [collective] intersubjective, informed
judgment, have we measured what we want
to measure?
• (N.B. use of good judgment)
• Rosenberg’s self esteem scale (SES)
questions:
Content validity
• Also called “sampling validity”
• establishes that the measure covers the full
range of the concept’s meaning, i.e., covers
all dimensions of a concept
• N.B depends on “good “ judgment
• EG. Earlier SES scale in class
• Authoritarian personality questions from
Walizer & Wienir
EMPIRICAL Validity
• Establishes that the results from one
measure match those obtained with a
more direct or already validated measure
of the same phenomenon (the “criterion”)
• Includes
– Concurrent
– Predictive
Concurrent Validity
• Validity exists when a measure yields scores that
are closely related to scores on a criterion
measured at the same time
• Does the new instrument correlate highly with
an old measure of the same concept that we
assume (judge) to be valid? (use of “good”
judgment)
• EG. Aronson’s doodle measure of achievement
motivation.
• Act vs. SAT
Predictive Validity
• Exits when a measure is validated by
predicting scores on a criterion measured in
the future
• Are future events which we judge to be a
result of the concept we’re measuring
anticipated [predicted] by the scores we’re
attempting to validate
• Use of “good” judgment
Examples of Predictive Validity
• Bronson screening test for “at risk”
parenting followed up by interviewing and
observing family members and school staff
later
• Sat / ACT scores and later college
“performance” (grades)
• Grades are “judged” to be measured validly
What’s a Construct?
• Multidimensional concept
– SES
– Industrialization
• Fuzzy concept / hard to define
– Ego strength
– Love
• Concept build out of other concepts
– Force=mass * acceleration
Construct validity
• : established by showing that a measure is
(1) related to a variety of other measures
as specified in a theory, used when no
clear criterion exists for validation
purposes (2) that the operationalization
has a set of interrelated items and (3) that
the operationalization has not included
separate concepts
Construct validity
• Check the intercorrelation of items used to
measure construct judged to be valid
• Use theory to predict a relationship and use
a judged to be valid measure of the other
variable then check for relationship
• Demonstrate that your measure isn’t related
to judged to be valid measures of unrelated
concepts
Convergent Validity
• Convergent validity: achieved when one
measure of a concept is associated with
different types of measures in the same
concept (this relies on the same type of
logic as measurement triangulation)
• Measures intercorrelated
Discriminant Validity
• Discriminant validity: scores on the
measure to be validated are compared to
scores on measures of different but related
concepts and discriminant validity is
achieved if the measure to be validated is
NOT strongly associated with the
measures of different concepts
• Measure not related to unrelated concepts
Difference between reliability and
validity
• Reliability: the degree to which a measurement
procedure produces similar outcomes when it is
repeated.
• E.g., gender, birthplace, mother’s
name—should be the same always—
• Validity: tests for determining whether a

measure is measuring the concept that the
researcher thinks is being measured,
• i.e., “Am I measuring what I think I am
measuring”?
SAMPLING RESEARCH
SAMPLING…
⦿ The process of selecting a number of individuals
for a study in such a way that the individuals
represent the larger group from which they were
selected
⦿ .. the representatives selected for a study
whose characteristics exemplify the larger
group from which they were selected
⦿ Population…
⦿ …the larger group from which individuals are
selected to participate in a study
THE PURPOSE FOR SAMPLING…
⦿ To gather data about the population in order to
make an inference that can be generalized to
the population
THE SAMPLING PROCESS…
POPULATION
INFERENCE
SAMPLE
REGARDING THE SAMPLE…
POPULATION (N)
IS THE SAMPLE
REPRESENTATIVE?
SAMPLE (n)
REGARDING THE INFERENCE…
POPULATION (N)
INFERENCE
IS THE
INFERENCE
SAMPLE (n)
GENERALIZABLE?
STEPS IN SAMPLING...
1. Define population (N) to be sampled

2. Determine sample size (n)
3. Control for bias and error
4. Select sample
1. DEFINE POPULATION TO BE
SAMPLED...
⦿ Identify the group of interest and its

characteristics to which the findings of the
study will be generalized
…called the “target” population

(the ideal selection)
…oftentimes the “accessible” or
“available” population must be
used (the realistic selection)
2. DETERMINE THE SAMPLE SIZE...
⦿ The size of the sample influences both the

representativeness of the sample and the
statistical analysis of the data
…larger samples are more likely

to detect a difference between
different groups
…smaller samples are more likely
not to be representative
RULES OF THUMB FOR DETERMINING
THE SAMPLE SIZE...
1. The larger the population size, the

smaller the percentage of the
population required to get a
representative sample
2. For smaller samples (N ‹ 100), there is
little point in sampling. Survey the
entire population.
3. If the population size is around 500
(give or take 100), 50% should be
sampled.
4. If the population size is around 1500,
20% should be sampled.
5. Beyond a certain point (N = 5000),
the population size is almost
irrelevant and a sample size of 400
may be adequate.
3. CONTROL FOR SAMPLING BIAS AND
ERROR...
⦿ Be aware of the sources of sampling bias and
identify how to avoid it
● Decide whether the bias is so severe

that the results of the study will be
seriously affected
● In the final report, document
awareness of bias, rationale for
proceeding, and potential effects
4. SELECT THE SAMPLE...
⦿ A process by which the researcher attempts to
ensure that the sample is representative of the
population from which it is to be selected
…requires identifying the sampling

method that will be used
BASICS OF SAMPLING
Samples offer many benefits:
⦿ A sample is a
⦿ Save costs: Less expensive to study
“part of a whole the sample than the population.
to show what the
⦿ Save time: Less time needed to study
rest is like”.
the sample than the population .
⦿ Sampling helps to
⦿ Accuracy: Since sampling is done
determine the
corresponding with care and studies are conducted
value of the by skilled and qualified interviewers,
population and the results are expected to be
plays a vital role accurate.
in marketing ⦿ Destructive nature of elements: For
research. some elements, sampling is the way to
test, since tests destroy the element
itself.
BASICS OF SAMPLING
Limitations of Sampling Sampling Process

Developin
⦿ Demands more rigid g
control in undertaking Defining a
sample operation. the sampling
population Frame
⦿ Minority and smallness in
number of sub-groups Specifying
often render study to be Sample Determini
suspected. Method ng
⦿ Accuracy level may be Sample
affected when data is Size
subjected to weighing.
⦿ Sample results are good
approximations at best. SELECTING THE SAMPLE
Sampling: Step 1 Sampling: Step 2
Defining the Universe Establishing the Sampling
Frame
⦿ Universe or population is the
whole mass under study. ⦿ A sample frame is the list of all
⦿ How to define a universe: elements in the population
(such as telephone directories,
◼ What constitutes the units of
electoral registers, club
analysis (HDB apartments)?
membership etc.) from which
◼ What are the sampling units
the samples are drawn.
(HDB apartments occupied in
the last three months)? ⦿ A sample frame which does not
fully represent an intended
◼ What is the specific designation
population will result in frame
of the units to be covered (HDB
error and affect the degree of
in town area)?
reliability of sample result.
◼ What time period does the data
refer to (December 31, 1995)
STEP - 3
DETERMINATION OF SAMPLE SIZE
⦿ Sample size may be determined by using:
◼ Subjective methods (less sophisticated methods)
The rule of thumb approach: eg. 5% of population
Conventional approach: eg. Average of sample sizes of similar
other studies;
Cost basis approach: The number that can be studied with the
available funds;
◼ Statistical formulae (more sophisticated methods)
Confidence interval approach.
STEP 4:
SPECIFYING THE SAMPLING METHOD
⦿ Probability Sampling
◼ Every element in the target population or universe [sampling
frame] has equal probability of being chosen in the sample
for the survey being conducted.
◼ Scientific, operationally convenient and simple in theory.
◼ Results may be generalized.
⦿ Non-Probability Sampling
◼ Every element in the universe [sampling frame] does not
have equal probability of being chosen in the sample.
◼ Operationally convenient and simple in theory.
◼ Results may not be generalized.
PROBABILITY SAMPLING
Four types of probability sampling
⦿ Appropriate for ⦿ Appropriate for

homogeneous population heterogeneous
◼ Simple random sampling population
Requires the use of a random ◼ Stratified sampling
number table. Use of random number table
◼ Systematic sampling may be necessary
Requires the sample frame ◼ Cluster sampling
only, Use of random number table
No random number table is may be necessary
necessary
NON-PROBABILITY SAMPLING
⦿ Four types of non-probability sampling
techniques
◼ Very simple types, based on subjective criteria
Convenient sampling
Judgmental sampling
◼ More systematic and formal
Quota sampling
◼ Special type
Snowball Sampling
RANDOM SAMPLING METHODS...
1. Simple random sampling

2. Stratified sampling
3. Cluster sampling
4. Systematic sampling
1. Simple random sampling: the process
of selecting a sample that allows
individual in the defined population to
have an equal and independent
chance of being selected for the
sample
STEPS IN RANDOM SAMPLING...
1. Identify and define the population.
2. Determine the desired sample size.
3. List all members of the population.
4. Assign all individuals on the list a consecutive number from
zero to the required number. Each individual must have the
same number of digits as each other individual.
5. Select an arbitrary number in the table of random numbers.
6. For the selected number, look only at the number of digits
assigned to each population member.
7. If the number corresponds to the number assigned to any of
the individuals in the population, then that individual is
included in the sample.
8. Go to the next number in the column and repeat step #7 until
the desired number of individuals has been selected for the
sample.
advantages…
…easy to conduct
…strategy requires minimum knowledge of the population to be
sampled
disadvantages…
…need names of all population members
…may over- represent or under- estimate sample members
…there is difficulty in reaching all selected in the sample
SIMPLE RANDOM SAMPLING
⦿ Also called random 20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
sampling
1 37 75 10 49 98 66 03 86 34 80 98 44 22 22 45 83 53 86 23
⦿ Simplest method of 51
probability 2 50 91 56 41 52 82 98 11 57 96 27 10 27 16 35 34 47 01 36
08
sampling
3 99 14 23 50 21 01 03 25 79 07 80 54 55 41 12 15 15 03 68
56
4 70 72 01 00 33 25 19 16 23 58 03 78 47 43 77 88 15 02 55
67
5 18 46 06 49 47 32 58 08 75 29 63 66 89 09 22 35 97 74 30
80
6 65 76 34 11 33 60 95 03 53 72 06 78 28 14 51 78 76 45 26
45
Need to use 7 83 76 95 25 70 60 13 32 52 11 87 38 49 01 82 84 99 02 64
Random 00
8 58 90 07 84 20 98 57 93 36 65 10 71 83 93 42 46 34 61 44
Number Table 01
9 54 74 67 11 15 78 21 96 43 14 11 22 74 17 02 54 51 78 76
76
10 56 81 92 73 40 07 20 05 26 63 57 86 48 51 59 15 46 09 75
64
11 34 99 06 21 22 38 22 32 85 26 37 00 62 27 74 46 02 61 59
81
12 02 26 92 27 95 87 59 38 18 30 95 38 36 78 23 20 19 65 48
2. Stratified sampling: the process of
selecting a sample that allows
identified subgroups in the defined
population to be represented in the
same proportion that they exist in the
population
STEPS IN STRATIFIED SAMPLING...

3. Identify the variable and subgroups (strata) for which you
want to guarantee appropriate, equal representation.
4. Classify all members of the population as members of one
identified subgroup.
5. Randomly select, using a table of random numbers) an
“appropriate” number of individuals from each of the
subgroups, appropriate meaning an equal number of
individuals
advantages…
…more precise sample
…can be used for both proportions and stratification sampling
…sample represents the desired strata
disadvantages…
…need names of all population members
…there is difficulty in reaching all selected in the sample
…researcher must have names of all populations
3. Cluster sampling: the process of
randomly selecting intact groups,
not individuals, within the defined
population sharing similar
characteristics
CLUSTER SAMPLING
⦿ Is a type of sampling in which clusters or groups of

elements are sampled at the same time.
⦿ Such a procedure is economic, and it retains the
characteristics of probability sampling.
⦿ A two-step-process:
◼ Step 1- Defined population is divided into number of mutually
exclusive and collectively exhaustive subgroups or clusters;
◼ Step 2- Select an independent simple random sample of
clusters.
⦿ One special type of cluster sampling is called area sampling, where
pieces of geographical areas are selected.
STEPS IN CLUSTER SAMPLING…
3. Identify and define a logical cluster.
4. List all clusters (or obtain a list) that make up the population of clusters.
5. Estimate the average number of population members per cluster.
6. Determine the number of clusters needed by dividing the sample size by
the estimated size of a cluster.
7. Randomly select the needed number of clusters by using a table of
random numbers.
8. Include in your study all population members in each selected cluster.
advantages…
…efficient
…researcher doesn’t need names of all population members
…reduces travel to site
…useful for educational research
disadvantages…
…fewer sampling points make it less like that the
sample is representative
4. Systematic sampling: the
process of selecting individuals
within the defined population
from a list by taking every Kth
name.
SYSTEMATIC SAMPLING
STEPS IN SYSTEMATIC SAMPLING...
3. Obtain a list of the population.
4. Determine what K is equal to by dividing the size of the
population by the desired sample size.
5. Start at some random place in the population list. Close you
eyes and point your finger to a name.
6. Starting at that point, take every Kth name on the list until the
desired sample size is reached.
7. If the end of the list is reached before the desired sample is
reached, go back to the top of the list.
advantages…
…sample selection is simple
disadvantages…
…all members of the population do not
have an equal chance of being selected
…the Kth person may be related to a
periodical order in the population list,
producing unrepresentativeness in the
sample
NON-PROBABILITY SAMPLES
⦿ Convenience sampling
◼ Drawn at the convenience of the researcher. Common in exploratory
research. Does not lead to any conclusion.
⦿ Judgmental sampling
◼ Sampling based on some judgment, gut-feelings or experience of the
researcher. Common in commercial marketing research projects. If inference
drawing is not necessary, these samples are quite useful.
⦿ Quota sampling
◼ An extension of judgmental sampling. It is something like a two-stage
judgmental sampling. Quite difficult to draw.
⦿ Snowball sampling
◼ Used in studies involving respondents who are rare to find. To start with, the
researcher compiles a short list of sample units from various sources. Each of
these respondents are contacted to provide names of other probable
respondents.
SAMPLING VS NON-SAMPLING ERRORS
Sampling Error [SE] Non-sampling Error [NSE]
Very small sample Size
Larger sample size
Still larger sample
Complete census
CHOOSING PROBABILITY VS. NON-PROBABILITY
SAMPLING
Probability Evaluation Criteria Non-probability

sampling sampling
Conclusive Nature of research Exploratory
Larger sampling Relative magnitude Larger

non-sampling
errors sampling vs. error
non-sampling error
High Population variability Low

[Heterogeneous] [Homogeneous]
Favorable Statistical Considerations Unfavorable
High Sophistication Needed Low
Relatively Longer Time Relatively shorter
High Budget Needed Low

MISTAKES TO BE CONSCIOUS
OF...
1. Sampling error
2. Sampling bias
…which threaten to render a study’s

findings invalid
Sampling error…
…the chance and random variation in
variables that occurs when any
sample is selected from the
population
…sampling error is to be expected

…to avoid sampling error, a census of
the entire population must be taken
…to control for sampling error,
researchers use various sampling
methods
Sampling bias…
…nonrandom differences, generally the fault
of the researcher, which cause the sample
is over-represent individuals or groups
within the population and which lead to
invalid findings
…sources of sampling bias include the

use of volunteers and available
groups
APPROACHES TO QUANTITATIVE
SAMPLING...
1. Random: allows a procedure

governed by chance to select the
sample; controls for sampling bias
2. Nonrandom (“nonprobability”): does
not have random sampling at any
state of the sample selection;
increases probability of sampling bias
NONRANDOM SAMPLING METHODS...
1. Convenience sampling
2. Purposive sampling
3. Quota sampling
1. Convenience sampling: the process
of including whoever happens to be
available at the time
…called “accidental” or “haphazard”
sampling
disadvantages…
…difficulty in determining how much
of the effect (dependent variable)
results from the cause (independent
variable)
2. Purposive sampling: the process whereby the
researcher selects a sample based on experience
or knowledge of the group to be sampled
…called “judgment” sampling
disadvantages…
…because samples need to be small and many
potential participants are unwilling to undergo the
demands of participation, most qualitative
research samples are purposive
3. Quota sampling: the process whereby
a researcher gathers data from
individuals possessing identified
characteristics and quotas
disadvantages…
…people who are less accessible (more
difficult to contact, more reluctant to
participate) are under-represented
APPROACHES TO QUALITATIVE
SAMPLING...
…qualitative research is characterized by in-depth
inquiry, immersion in a setting, emphasis on
context, concern with participants’ perspectives,
and description of a single setting, not
generalization to many settings
…potential for inaccuracy in the researcher’s
criteria and resulting sample selections
…representativeness is secondary to the quality of
the participants’ ability to provide the desired
information about self and setting
1. Intensity sampling: selecting participants who
permit study of different levels of the research
topic
2. Homogeneous sampling: selecting participants
who are very similar in experience, perspective, or
outlook
3. Criterion sampling: selecting all cases that meet
some pre-defined characteristic
4. Snowball sampling: selecting a few individuals
who can identify other individuals who can identify
still other individuals who might be good
participants for a study
5. Random purposive sampling: with a small sample,
selecting by random means participants who were
purposively selected and are too numerous to
include all in the study
Sampling Process
Define
Population
Determine Sampling
Frame
Determine Sampling
Procedure
Probability Sampling Non-Probability Sampling

Type of Procedure Type of Procedure
Simple Random Sampling Convenience
Stratified Sampling Judgmental
Cluster Sampling Quota
Determine
Appropriate
Sample Size
Execute Sampling
Design
COPYRIGHT TERRY L.
Addressing Non-Sampling Error
How do We Get People to Complete Our Surveys
and Provide us with Good Information ??
Non-Respo
nse
Error
Non-Sampli
ng
Response
Error
or
Field
Error
MAJOR CONCLUSION
Sampling Error is Very Minor When
Compared
to Non-Sampling Error
Sampling Error - 5 % of Total Error
Non-Sampling Error - 95% of Total Error
Source: Assael and
COPYRIGHT TERRY L. Keon, JM.
Sources of Non-Sampling Error
INADEQUATE
SAMPLING
NON- FRAME
COVERAGE NOT AT HOM
NON- CAN’T LOCAT
SAMPLING CONTACT
NON- EXPLICIT
ERROR RESPONSE SENSITIVE
TOTAL NON- REFUSA TOPIC
OVERBURDENIN
ERROR OBSERVATION LS G
IMPLICIT
LACK OF
NON-SAMPLING FIELD DATA INTERVIEW MOTIVATIO
COLLECTION N
ERROR BIAS
OBSERVATION
ERROR
OFFICE EDITING
PROCESSING CODING
ANALYSES
COPYRIGHT TERRY L.
Research design – sampling
● Chapter 7 in Babbie & Mouton (2001)

● How we select from an infinite number of
observations we could possibly make
● Why do we sample?
● Size of the population
● Cost of obtaining elements
● Convenience and accessibility of elements
● (in his study on suicide, Durkheim had relatives who held
high governmental positions, who were able to provide
access to French statistical records on suicides)
Research design - sampling 156

How do we decide what to observe?
● This decision should be a matter of deliberate

choice rather than chance.
● Representativeness – a small sample of
individuals from a population must contain
essentially the same variations that exist in the
population
● BUT: limited to those characteristics that are
relevant to the substantive interests of the
study, not ALL aggregate characteristics

Key concepts
● Element
● The unit about which information is collected
● Typically the elements are people
● But look at the section on “unit of analysis”
again: any of them could be elements
(schools, universities, corporations, etc.)

Population
● All the potential study elements, as

defined
● Careful specification of the population

Study population
● Almost impossible to guarantee that every

element meeting your definition of “the
population” has a chance to be selected
into the sample.
● Thus the “study population” will be
somewhat smaller than “the population”

Sampling unit
● Typically the sampling units are the same

as the elements and probably the units of
analysis
● (We are not going to look into more
complex sampling units)

Sampling frame
● The actual list of sampling units (or

elements).
● e.g. if you want to study “Students at the
University of Cape Town”, there is a list of
such sampling units (but there are a
number of definition issues to be resolved
here)

Sample
● A subset of a population selected to estimate the

behaviour or characteristics of the population
● O/head p. 169
● Importance of sampling properly
● A sample exists to represent its parent
population
● We must know what the actual parent
population is, otherwise we draw false
conclusions
● e.g. if we sample only women, we cannot safely make claims
about men
Deciding who to choose
● Basically two sampling strategies

available:
● Probability sampling – each member of the
population has a certain probability to be
selected into the sample
● Non-probability sampling – members selected
not according to logic of probability (or
mathematical rules), but by other means (e.g.
convenience, or access)

Non-probability sampling
● Sometimes it is not possible to get the

kind of information about populations
required for probability sampling
● When the sampling frame is not known
● Complicates and limits statistical analyses
● Often well-suited for qualitative research,
where distribution of characteristics is not
important
Convenience sample
● Rely on available respondents

● Most convenient method
● Risky; exercise caution

Purposive sampling
● Select the sample on the basis of

knowledge of the population: your own
knowledge, or use expert judges to
identify candidates to select
● Typically used for very rare populations,

such as deviant cases

Snowball sampling
● Typically used in qualitative research

● When members of a population are
difficult to locate, for covert
sub-populations, non-cooperative groups
● Recruit one respondent, who identifies
others, who identify others,….
● Primarily used for exploratory purposes

Quota sampling
● A stratified convenience sampling strategy

● Begins with a table that describes the characteristics of
the target population
● e.g. the composition of postgraduate students at UCT in terms of
faculty, race, and gender
● Then select on a convenience basis, postgraduate
students in the same proportions regarding faculty, race,
and gender than in the population
● Of course, the quota frame (the proportions in the table)
must be accurate
● And biases may be introduced when selecting elements
to study
Probability sampling
● Bias introduced in non-probability sampling

● The sample then not representative of the population
● Probability sampling
● Typically produce more representative samples
● Allow us to estimate the accuracy or
representativeness of the sample
● Random sampling is the key – each element has an
equal chance of being selected
● Random sampling offers access to probability theory,
through which we can estimate how representative
our sample is
Probability sampling theory
● Sampling distribution – possible samples to be

drawn
● Sampling error
● The degree to which the sample characteristics
approximate the characteristics of the population
● The smaller the sample, the greater the sampling
error
● The larger the sample, the smaller the sampling error
- but only for probability sampling plans

Estimating the true mean
● Note how the notion of a sampling

distribution relies on the “Central
Tendency Theorem”, that you will
recognize from the statistics part of the
course
● Standard error and standard deviation
● Inferences
● Confidence levels

Populations and sampling frames
● In countries like South Africa, less-than-perfect

conditions regarding sampling frames
● Either the information is not available, or if it is, it is
subject to error (parts of it are missing)
● Remember: we use sampling frames as reflecting the
population; ideally, the population also is the time frame
– but almost never the case
● So problems when there is a disparity between a
population and the sampling frame we use

Example
● A good example is using telephone

directories as sampling frames
● Or the difficulties with finding a complete
list of all primary health care facilities in
the country
● And think about the risks in using
municipal service records as a sampling
frame for “the residents of Cape Town”
Three kinds of probability sampling
strategies
● 1 Simple random sampling
● Establish a sampling frame (a list, e.g. of all
the company’s customers, or all UCT
students)
● Assign a single number to each element in the
list
● Use random numbers to select the elements

2 Systematic sampling
● Usually more efficient than SRS

● Establish a sampling frame
● Select the first element at random
● Then select every nth element in the list, until you have
the required number of respondents
● e.g. with a population of 300, if we want a sample of
10, choose every 30th element
● Keep an eye out for peculiar arrangements in the
sampling frame

3 Stratified sampling
● Sampling error reduced by

● 1. using a large sample
● 2. a homogeneous population
● Stratified sampling based on 2.

Stratified sampling
● Modifies random sampling and systematic sampling, to

obtain a greater degree of representativeness
● Organize the population into homogeneous subsets, then
sample randomly within each one
● e.g. for university students, stratify according to seniority
and gender
● Stratification ensures equal proportions of people having
the relevant characteristics are selected into your sample
● Depends on what variables are available to stratify on

Research, Methodology 3 Christ

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Research, Methodology 3 Christ

Uploaded by

Copyright:

Available Formats

8-1

Unit 3 : Data Collection, measurement and scaling

■Qualitative methods of data collection,

■types of measurement scale,

■Single Item v/s Multi Item Scales,

■Comparative v/s Non-Comparative scales,

■Continuous Rating Scales;

■Criteria for Good Measurement,

Basic Measurement Issues

Measurement is the process of assigning numbers

1. Define concepts to be measured

Basic Measurement Issues

Measurement and Scaling

and the characteristics being measured.

standardized and applied uniformly.

Measurement and Scaling

Consider an attitude scale from 1 to 100. Each

Primary Scales of Measurement

Ordinal Rank Order Finish

■ The numbers serve only as labels or tags for

Illustration of Primary Scales of Measurement

Nominal Ordinal Interval Ratio

■ Possesses all the properties of the nominal, ordinal,

Primary Scales of Measurement

Criteria for Scale Selection

■ Understanding of the questions

Central Tendency and Dispersion

■ Central Tendency ■ Dispersion

■ Mean ■ Standard deviations

Central Tendency and Dispersion

■ If a nominal scale is used, analysis of raw data can only

A Classification of Scaling Techniques

Paired Rank Constant Q-Sort and Continuous Itemized

A Comparison of Scaling Techniques

■ Comparative Rating Scales

■ In noncomparative scales, each object is scaled independently of

■ Noncomparative Rating Scales

A COMPARATIVE RATING SCALE

A COMPARATIVE RATING SCALE ASKS RESPONDENTS TO RATE A

Please indicate how the amount of authority in your present position

TOO MUCH ABOUT RIGHT TOO LITTLE

Relative Advantages of Comparative Scales

Comparative Rating Scales

■ A respondent is presented with two objects and

Instructions: We are going to present you with ten pairs of

■ Respondents are presented with several objects

Instructions: Rank the various brands of toothpaste in order

6. Ultra Brite _________

■ Respondents allocate a constant sum of units, such

Attitudes as Hypothetical Constructs

● The term hypothetical construct is

THE FEELINGS OR EMOTIONS

■ KNOWLEDGE AND BELIEFS

The Attitude Measuring Process

RANKING - Rank order preference

● Ranking tasks require that the

● Rating asks the respondent to

● Sorting might present the respondent with

● Physiological measures of attitudes

Simple Attitude Scaling

SIMPLIFIED SCALING EXAMPLE

THE PRESIDENT SHOULD RUN FOR RE-ELECTION

EXAMPLE OF CATEGORY SCALE

CLIMATE _____ _ _______

Passive _ : _ : _ : _ : _ : _ : ___