Session 4

Descriptive Research: Survey/Observation

Causal Research: Experiment
 Roadmap
Marketing Research Data

Secondary Data Primary Data

Qualitative Data Quantitative Data

Descriptive Causal

Survey Observational and Experimental

Data Other Data Data
 Methods of Obtaining Quantitative Data in
Descriptive Research
Figure 7.3 Methods of O
btaining Quantitative D
ata in Descriptive Resea
rch

Quantitative Descriptive
Research

SURVEY OBSERVATION
Information Obtained Information Obtained
by Questioning Respondents by Observing Behavior or
Phenomena
 Survey Method
• Obtaining information based on questioning respondents
• Involves sampling a large number of people and asking the
m a series of fixed-response questions
• Questions regarding to behavior, intentions, attitudes, awar
eness, motivations, demographics and lifestyle characteristi
cs can be asked
• Advantages: reliability, simplicity, easy to administer
• Disadvantages: unwilling or unable to response, cannot me
asure subconscious nature of respondent’s motive
 Criteria for Evaluating
Survey Methods
• 1. Task factors
– Complexity of the questionnaire
– Use of physical stimuli (e.g., product, ads)
– Quantity of data or length of questionnaire
– Response rate (i.e., % of attempted interviews that are co
mpleted)
– Sample control: the ability to reach the desirable sample e
ffectively and efficiently

5
 Criteria for Evaluating Survey Methods
• 2. Situational factors
– Control of the data collection environment
– Potential for interviewer bias (i.e., the error due to the interviewer n
ot following the correct interviewing procedures; interviewer’s facial
expressions, intonations, the way they ask the questions)
– Control of field-force (e.g. whether the interviewers can be supervis
ed from a central location)
– Speed: time taken for administering the survey
– Cost (of administering the survey and collecting the data)

• 3. Respondent factors
– Perceived anonymity of the respondents’ identities
– Social desirability: the tendency of the respondents to give answers
that may not be accurate but that may be desirable from a social sta
ndpoint.
6
 Classification of Survey Methods

Survey Methods

Telephone Personal Mail Electronic

Traditional In-Home Mail E-Mail

Telephone
Mall Intercept Internet
Computer-
Assisted Computer-
Telephone Assisted
Interviewing Personal
(CATI) Interviewing
(CAPI)
 1. Telephone Method
• Traditional telephone interviews
– Use a paper questionnaire and records the res
ponses with a pencil
– Field service supervisors can closely monitor th
e telephone conversation

• Computer-assisted telephone interviewing (CATI)

– The computer dials the telephone number to be called
– Interviewer reads the questions posed on the computer
screen and record answers directly into the computer
– The computer has built-in logic to ensure data accuracy
Relative Advantages of Different Survey Methods

Method Advantages Disadvantages

Telephone - Fast
- High sample control
- Good control of field force
- Good response rate
- Moderate cost
- No use of physical stimuli
- Limited to simple questions
- Quantity of data is low
 2. Personal Method
• Personal in-home interviews
– Respondents are interviewed face-to-face at
homes

• Mall/Street intercept personal interviews

– Stopping potential respondents in a public
area, screening them for appropriateness
and inviting him or her to complete the
interview in a convenient place.
• Computer-assisted personal interviewing (CAPI)
– Respondent sits in front of a computer and answers a
questionnaire on the screen. An interviewer is present
to serve as a host and to guide the respondent as
needed.
Relative Advantages of Different Survey Methods
Method Advantages
Personal In- - Complex questions can be asked
Home - Good for physical stimuli
- Very good sample control
- High quantity of data
- Very good response rate
- Longer interviews can be done
Mall-intercept - Complex questions can be asked
- Very good for physical stimuli
- Very good control of environment
- Very good response rate
CAPI - Complex questions can be asked
- Very good for physical stimuli
- Very good control of environment
- Very good response rate
- Low potential for interviewer bias
Disadvantages
- Low control of field force
- High social desirability
- Potential for interviewer bias
- Most expensive
- Some samples (e.g. high-
crime areas) may be difficult to
access
- May take long to collect data
- High social desirability
- Potential for interviewer bias
- Quantity of data is moderate
- High cost
- High social desirability
- Quantity of data is moderate
- High cost
 3. Mail Method
• Mail interviews
– Send “cold” mail survey to individuals who
meet a specified demographic profile but wi
thout pre-contact them
• Through mailing lists (usually from company database
, directory or those purchase from syndicated companies)
• Package: outgoing envelope, cover letter, questi
onnaire, return envelope and incentives
Relative Advantages of Different Survey Methods
Method Advantages Disadvantages
Mail - No field force problems - Limited to simple questions
- No interviewer bias - Low sample control for cold mail
- Moderate/High quantity of data - No control of environment
- Social desirability is low - Low response rate for cold mail
- Low cost - Low speed
 4. Electronic Method
• Email surveys
– the questionnaire is emailed to respondents
• Internet surveys
– the questionnaire is posted on a web site that is s
elf-administrated by the respondent
Relative Advantages of Different Survey Methods
Method Advantages Disadvantages
Electronic: - Low cost - Moderate quantity of data
E-mail - No interviewer bias - Low sample control
- High speed - No control of environment
- Social desirability is low - Low response rate
- Contact hard-to-reach - Security concerns
respondents
Electronic: - Visual appeal and interactivity - Moderate quantity of data
Internet - No interviewer bias - Low sample control
- Low cost - No control of environment
- Social desirability is low - Low response rate
- Very high speed
- Personalized, flexible questioning
- Contact hard-to-reach
- Respondents
 Methods of Improving Response Rates

Prior Other
Incentives Follow-up
Notification Facilitators

Monetary Nonmonetary

Prepaid Promised
 Methods of Obtaining Quantitative Data in
Descriptive Research
Figure 7.3 Methods of O
btaining Quantitative D
ata in Descriptive Resea
rch

Quantitative Descriptive
Research

SURVEY OBSERVATION
Information Obtained Information Obtained
by Questioning Respondents by Observing Behavior or
Phenomena
 Observation Methods
• The recording of behavioral patterns of people, objects,
and events in a systematic manner to obtain information
about the phenomenon of interest.
• The observer does not question or communicate with th
e people being observed.
 How to obverse?
– Mechanical observation
• Use of electronic device to record behavior

– On-site camera (still, motion picture,

video), optical scanner in supermarkets, the

Nielsen peoplemeter

– Personal observation
• A trained observer collects the data by
recording behavior exactly as it occurs
• No attempt to control or manipulate the
phenomenon being observed
Data Collection for Observation Methods

Watching
Watching

Listening
Listening

Touching
Touching

Smelling
Smelling

Reading
Reading
 What can be observed?
• Physical movements
• Verbal behavior
• Expressive behavior and physiological reaction
s
• Temporal patterns
• Physical objects
• And more…
 Examples of what can be observed
In-store At home On the road On the Internet
Arrival time in store TV viewing patterns Use of public Comments made on
transport review sites
Shopper’s movement Family purchasing Driving routes taken Brands mentioned in
pattern in store behavior blogs and on
Facebook
Time spent Brands of products Behavior while driving Links made to
shopping/queuing owned (e.g., use of car seat corporate or celebrity
belts, mobile phones, sites
etc.)
Behavior/expressions Children’s behavior Products used in
when queuing with a new toy videos
Interaction with Clothes worn in videos
service personnel
Products purchased Opinions expressed
Payment methods Trends discussed
used
 Chat:
What can NOT be observed?
• The report summariz
es the experience of
a mystery shopper w
ho pretended to be a
potential bank custo
mer

• Source: Marketing Re
search Kit for Dummi
es
Structured Observational Report for retailer

Record #:________
 Female  Male
First soft drink can picked up for examination:
 Pepsi  Coca-Cola  7-up  Sprite  Other: ______
Total # cans picked up for examination, any brand: _________

Brand selected (leave blank if none selected)

 Pepsi  Coca-Cola  7-up  Sprite  Other: ______

Time (in front of soft drink shelves): ________ seconds

 Relative Advantages of Observation
• Measuring actual behavior (rather than self-report of intende
d or preferred behavior)
• Respondents participations are not required (i.e. minimize no
n-response errors)
• No reporting bias, and potential bias caused by the interviewe
r
• Appropriate for collecting behavior that people are unware or
unable to communicate
• May cost less and faster
 Relative Disadvantages of Observation
• Attitudes, motivations and values are hardly measured
• Behaviors occurring infrequently or spanning a long period of
time are too expensive to use this method
• Interviewers may overlook other important issues
• Sometimes invade privacy

It is best to view observation as a complement to survey methods, rather than as being

in competition with them
 Roadmap
Marketing Research Data

Secondary Data Primary Data

Qualitative Data Quantitative Data

Descriptive Causal

Survey Observational and Experimental

Data Other Data Data
 The Value of Causal Research
• While descriptive research is effective in identifying co-variati
on between variables (e.g. blue packages outsell red ones, co
nsumption rate varies by education level, overall satisfaction r
ate due to several factors), it cannot truly indicated causality
(e.g. color causes sales, education causes consumption).

• Causal research is in need to determine “what causes consum

ers to act that way”
– E.g. does 10% decrease in price cause an increase in profit compared t
o no price change?
 Experimentation
• Causal Research
– To study whether a change in one marketing variable (IV, c
ause X) produce a change in another variable (DV, an effect
Y)

– Experiment is the main method in causal research

• Involve the manipulation of one or more independent variables (I
V) and to measure their effect on one or more dependent variable
(DV), while controlling the influence of outside or extraneous varia
bles.
Concept of Causality
 Concept of Causality
• “X causes Y “ means that the occurrence of X increases the pr
obability of the occurrence of Y

• Examples

“decline in price causes increase in sales” – when t

he price was lowered, the sales are more likely to i
ncrease
 Conditions for causality
1. Concomitant variation
– Both IV (a cause, X) and DV (an effect, Y) should occur together or vary
together as predicted by the hypothesis under consideration
• May vary in a positive (i.e. the cause and effect both increase or de
crease together) or inverse relationship (i.e. one increases while th
e other decreases) => an association exists between IV and DV
2. Appropriate time order of occurrence of variables
– IV (a cause, X) should occur before or simultaneously with the DV (an e
ffect, Y); it should cannot occur afterwards
3. Absence of other possible causal factors
– Ensure that other possible causal factors (extraneous variables) are not
the source of influence (i.e. the presence of other possible factors must
be controlled)
In store
Purchase
adverting

The combination of all THREE conditions does not guarantee a causal relationship. But, an
accumulation of consistent evidence increases the confidence that a causal relationship exists.
 Experimentation
• Terminology of experimentation
– Test Units
• Are individuals, organizations, or other entities whose response to IV or trea
tments is being studied.
– Independent Variable (IV) (or experimental treatment) Symbol: X
• Variables that are manipulated by the researcher and whose effects are mea
sured and compared (i.e. the cause)
– Treatment levels
• The different levels of manipulation of IV
– Dependent Variable (DV) Symbol: O
• Variables that measured the effects of the IV on the test units (i.e. the effect,
Y)
– Extraneous Variable
• All variables other than the IV that affect the response of the test units e.g.,
store size, store location, and competitive effort
 Experimentation
• Terminology of experimentation
– Experimental Group (EG)
• The group exposed to the manipulated IV
– Control Group (CG)
• The group which is not exposed to the IV manipulation, thus provi
des a point of comparison when examining the effects of these ma
nipulated on the DV
– Randomization Symbol: R
• The random assignment of test units or groups to separate treatm
ents (use to control extraneous variables)
– Matching
• Comparing test units on a set of key background variables before a
ssigning them to the treatment conditions (use to control extraneo
us variables)
 Example
Will consumer buy more with in-store audio advertising?

An field experiment
was conducted to
investigate if in-store
audio advertising will
lead to purchase in a
supermarket.
 Validity in Experimentation
• Internal validity (Does X -> Y in this test?)
– Whether the manipulation of the independent variables or
treatments actually caused the observed effects on the de
pendent variables
– Internal validity is threatened when the influence of extran
eous variables are mixed with the independent variables

e.g. To claim internal validity, the researchers have to

show that price and other extraneous variables affecting
the sales were kept the same in both the experimental and
control group.
-> Increase in sales of the products in the experimental
group supermarkets was in fact due to (caused by) the in-
store audio advertising.
 Validity in Experimentation
• External validity (Does “X -> Y” happen in cases other than this
test?)
– Whether the cause-and-effect relationships found in
the experiment remain the same when replicated in
a larger population
• Can the results of the experiment be generalized to other peopl
e, settings and time?

e.g. To claim external validity, the

researchers have to show the sample of
20 supermarkets was representative, and
thus the results are generalizable to ALL
supermarkets.
 A Classification of Experimental Designs
• Preexperimental Design: Designs that offer little or
no control over extraneous factors.
– One-Shot Case Study
– One Group Pretest-Posttest

• True Experimental Design : Research using an exper

imental group and a control group, to which test uni
ts are randomly assigned.
– Pretest-Posttest Control Group
– Posttest-Only Control Group
 Pepsi’s Ad Research

Shall Pepsi continue its traditional strategy

of using celebrities in its ad?

42
 Pre-experimental Designs:
1. One-Shot Case Study

EG: X 01

• A single group of test units (e.g., consumers) is expo

sed to a treatment (e.g., watching the ad)
• A single measurement on the dependent variable
(e.g., attitude towards Pepsi) is taken

43
 Pre-experimental Designs:
1. One-Shot Case Study

Pepsi tests the effectiveness of new ad with celebrities:

• A group of consumers watched the ad
• These consumers rated their attitude to the brand

Rating of
Name
brand (O1) EG: X 01
Ann 2

Mary 5
Celebrity effect?
compare O1 with a
John 3
normal score (e.g., 3)
Peter 4

…
…
Mean 3.5

44
 Pre-experimental Designs:
2. One-Group Pretest-Posttest Design

EG: 01 X 02

• A single group of test units is measured twice.

• The treatment effect is computed as
• TE=02 – 01

45
 Pre-experimental Designs:
2. One-Group Pretest-Posttest Design
Pepsi tests the effectiveness of new ad with celebrities:
• A group of consumers first rated their attitude to the pepsi
• After a month, these consumers watch the new ad
• They rated their attitude to the brand again

Rating of Rating of
Name
brand (O1) brand (O2)
Ann 4 2 EG: 01 X 02
Mary 5 5
John 3 3 extraneous variables
Peter 5 4 are largely
… uncontrolled
…
Mean 4.25 3.5
46
 True Experimental Designs:
3. Pretest-Posttest Control Group Design

EG: R 01 X 02
CG: R 03 04

• Test units are randomly assigned to either the experiment

al or the control group.
• A pretreatment measure is taken on each group.
• The treatment effect is measured as:
• TE=(02 - 01) - (04 - 03).
• Selection bias is eliminated by randomization.

47
 True Experimental Designs:
3. Pretest-Posttest Control Group Design
Pepsi tests the effectiveness of new ad with celebrities:
• Consumers are randomly assigned to two groups
• Both groups rate their attitude toward the brand
• After a month, only one group watched the new ad
• Both groups rated again their attitude towards the brand
Rating of Rating of
Name
brand (O1) brand (O2)
Group 1 Ann 2 3
(watch new Mary 4 3
People are ad)
Peter 3 4
EG: R 01 X 02
randomly
assigned CG: R 03 04
to two Name
Rating of Rating of
brand (O3) brand (O4)
groups Group 2
(control John 5 5
group) Kitty 4 5
… 48
 True Experimental Designs:
4. Posttest-Only Control Group Design

EG : R X 01
CG : R 02

• The treatment effect is obtained by:

• TE = 01 - 02
• Except for pre-measurement, the implementation of this des
ign is very similar to that of the pretest-posttest control grou
p design.

49
 True Experimental Designs:
4. Posttest-Only Control Group Design

Pepsi tests the effectiveness of new ad with celebrities:

• Consumers are randomly assigned to two groups
• Then, only one group watched the new ad
• Both groups rated their attitude towards the brand

Rating of
Name
brand (O1)
Group 1 Ann 2
(watch new Mary 4
ad)
People are Peter 3 EG : R X 01
randomly
assigned to Rating of CG : R 02
Name
two groups Group 2 brand (O2)
(control John 5
group) Kitty 4
… 50
 Experimental Setting

• An artificial setting for experimentation in

which the research constructs the desired
conditions.

• Tests conducted outside the laboratory in an

actual environment, such as a marketplace.
• The researcher has much less control over
external environment.
 Laboratory versus Field Experiments
Factor Laboratory Field
Environment Artificial Realistic
Control High Low
Reactive error High Low
Demand artifacts High Low
Internal validity High Low
External validity Low High
Time Short Long
Number of units Small Large
Ease of implementation High Low
Cost Low High

Remarks: Demand artifacts refers to a phenomenon in which the respondents

attempt to guess the purpose of the experiment and modifying their responses
accordingly.
 Limitations of experimentation

• Time consuming
• Expensive
• Difficult to administer (e.g., maybe impossi
ble to control for the effects of extraneous vari
ables)
• The end!