Class Session #9 & 10

Marketing Research
 Summary for Last Class
• Some major influences on consumers
– Culture
• Individualism vs. collectivism, etc.
– Social Groups
• Reference groups
• Psychographics
– Psychological/contextual factors
• Attention, learning, memory
• 5-step consumer decision process
• The continuum of “involvement” in decision
making
Major influence on consumers
 Consumer Decision-making Process
1) Need Recognition

2) Information Search

3) Evaluation of
Alternatives

4) Purchase

5) Post-purchase
behavior
 Today’s Agenda
• Topics of marketing research

• Types of data: secondary vs primary

• Types of data collection: qualitative vs

quantitative

• Survey Designs

• Quantitative methods: surveys, lab

experiments, field experiments, natural
experiments
La La Land Box Office

74th Golden 89th Academy

Globe Awards Awards Nominations
Domino’s Pizza
Annual Revenues
 Domino’s Pizza

Recession

The Pizza Turnaround

Success of House of Cards

Data analysis: people who likes David Finch also likes Kevin
Spacey.

Big Data analytics: avoid scenes where people pause or

drop the series.
The Netflix Prize
 Textbook’s Definition
• “The process of planning, collecting, and
analyzing data relevant to a marketing
decision.”

• Goals: describe, diagnose, predict ->

– better decisions,
– tracking problems,
– keep customers, and
– understanding the market
 Some Specific Questions for
Marketing Research
• Marketing environment
– What are current economic conditions and trends?
• Demand analysis
– What consumer need-gaps exist?
– How big is the market?
• Competitive intelligence
– How does a competitor’s product portfolio compare?
– How does a competitor react to our marketing efforts?
• Marketing Mix
– How can I optimize my product portfolio?
– What will happen if I raise my price by x%?
– Which online ads and channels are most
impactful?
– Which ad should I run and how can I improve it?
In real world, it’s even more complicated
Using celebrity:
• Pros
– People like them --> people will like
your brand more
– Builds trust and credibility
– Enter new market
• Cons
– They are human…they can make
mistakes

Or just to stay on the

safe side, like Geico
Using celebrities
The worst could happen

OJ.Simpson & Hertz

20 years(1975-1994)
endorsment
 Should I use celebrity
endorsements?
• How many more customers can endorsers bring?

• What’s the probability that they will make mistakes

during the endorsement?

• How bad it will be for me if they make mistakes?

• How long will it last?

• And many more…

 Steps of
Marketing Research
1. Plan the research design(is secondary
data enough?)
2. *Decide on sampling procedure(if
necessary)
3. Collect data
4. Analyze data
5. Report
6. Follow up
 Step 0: Identify the Problem
• What do you want to achieve, what actions
will you take?
• How will you define success for a project?
• Not “we need to run a focus group” or “we
need numbers justifying a new product”

• “SEO”
• “Google Adwards”
 Step 1: Plan Research Design

• Examine constraints

• Decide methods to collect data

 Constraints: When not to do
Marketing Research

• Already have enough info

• Lack of resources (e.g., budget)
• Costs outweigh benefits
• Opportunity has passed
• Decision has been made
 Step 1: Plan Research Design
• What do you want to know?
 Qualitative & Quantitative
Methods of Data Collection
Qualitative (e.g., focus group) Quantitative (e.g., survey)

Discovery/exploration of new ideas Validation/confirmation of estimates,

facts, key relationships
Preliminary understanding of key
variables
Deep understanding of (hidden)
motivations
Open-ended, unstructured format Mostly structured format

Small, representative samples; need Large, varied samples

researcher skill
Descriptive, interpretive analysis Statistical, descriptive analysis
 Step 1: Plan Research Design

• Usual starting point: Secondary data

– Gathered for some purpose other than the
one on hand
– Usually don’t have all the variables you want

• Internal or external source

– e.g., CRM database, sales records
 Secondary Data: Tv Ratings Ranked on Households
Week Ending Feb. 5
# PROGRAMS Net HOUSEHOLDS PERSONS 2+
Rtg% Shr (000) # Rtg% (000)
1FOX SUPER BOWL LI FOX 45.3 70 53,650 1 36.9 111,317

2FOX SUPER BOWL FOX 26.1 45 30,936 2 20.2 61,081

POST GAME

3BIG BANG THEORY, CBS 8.9 16 10,589 3 4.9 14,659

THE

4SUPERIOR DONUTS CBS 6.5 11 7,684 4 3.5 10,539

5BLUE BLOODS CBS 6.3 12 7,467 5 3.4 10,130

6HAWAII FIVE-0 CBS 6.1 11 7,193 6 3.3 9,812

7GREY’S ANATOMY ABC 5.6 10 6,645 8 2.8 8,499

8MOM CBS 5.4 9 6,436 7 2.9 8,708

9BIG BANG CBS 5.3 9 6,222 8 2.8 8,435

THEORY,THE-8P:SP

10CRIMINAL MINDS CBS 4.7 8 5,622 10 2.5 7,456

 Secondary Data:
Movie Ticket Sales
Title (click to Weekend Theater Budget
TW LW Studio % Change Average Total Gross Week #
view) Gross Count / Change *
1 1 Split Uni. $14,424,195 -43.8% 3,373 +174 $4,276 $98,540,660 $9 3

2 N Rings Par. $13,002,632 - 2,931 - $4,436 $13,002,632 $25 1

3 2 A Dog's Purpose Uni. $10,509,170 -42.3% 3,178 +119 $3,307 $32,610,435 $22 2

4 3 Hidden Figures Fox $10,189,588 -27.2% 3,401 +50 $2,996 $119,491,683 $25 7

5 5 La La Land LG/S $7,372,066 -39.7% 3,236 +100 $2,278 $118,228,990 $30 9

Resident Evil:
6 4 The Final SGem $4,701,302 -65.4% 3,104 - $1,515 $22,053,077 $40 2
Chapter

7 7 Sing Uni. $4,052,190 -36.8% 2,293 -409 $1,767 $262,878,670 $75 7

xXx: The Return

8 6 Par. $3,875,479 -54.9% 2,478 -1,173 $1,564 $40,200,366 $85 3
of Xander Cage

The Space
9 N STX $3,775,596 - 2,812 - $1,343 $3,775,596 $30 1
Between Us

10 14 Lion Wein. $3,760,097 +60.1% 1,405 +830 $2,676 $24,465,771 - 11

February 3-5 2017, weekend estimates

 Secondary Data: Questionnaires
Top 2 Box (%) Male 18-34 Male 35+ Female 18-34 Female 35+
(n=105) (n=50) (n=67) (n=63)
C D E F
My eating habits have changed for the 88 D 71 83 86
better
Because of my lifestyle, I eat a lot on the 74 64 61 63
run
I consider myself more self-aware than I 84 78 86 78
used to be
When it comes to food, I'm not willing to 60 60 48 59
compromise taste even for something that
is healthy
I'm somewhat skeptical of processed food 73 64 81 D 72
When it comes to food, I consider myself 82 75 72 84
to be open-minded and adventurous

Eating better is a reasonable alternative to 74 DEF 44 51 44

exercising more

I believe what I eat today will have a 89 81 88 95 D

significant impact on the quality of my life
in the future
I try to watch what I eat 86 75 86 87
When it comes to my health, I trust nature 73 65 68 71
more than science
Source: IPSOS Past 6 Month User Omnibus – Sep 2010
CDEF Significantly higher at 95%
Census
 Secondary Data
• Real-time customer responses
– e.g., Amazon recommendations

– Bank call center

 Secondary Data
Google Flu Trends

Google Ads
 Some other internal sources of
secondary data
• Sales invoices
– customer demographics
– shipment points
– wtp, etc.
• Accounts receivable reports
– purchase records
– customer as % of sales
– profit margin
– items returned, etc.
• Credit applications
– customer biographies, etc.
• Warranty registrations
– sales volume
– Zip codes
– reasons for returns, etc.
• Exit interviews, mail-order forms, customer letters, cash
register receipts, etc., etc.
 Secondary Data & Privacy

• Data brokers
– Demographics
– Life events
– Pay stubs
– Credit Card Statement
Correlation
Traps in data analysis : Causality
Traps in data analysis :
endogeneity
 Primary Data
Examples:
Research Participation

Eye tracking

Survey
– Collecting new data to answer the
question(s) at hand
– Cost more/Answer question more
directly/accurately
– Many times impossible to gather
Primary or Secondary?
 Primary data collection
• Qualitative Methods
– Depth interviews
– Focus groups

• Quantitative Methods
– Surveys
– Observational
– Lab Experiments
– Field Experiments
– Natural Experiments
Qualitative Methods: Examples
• Depth interviews
– “Anthropological” one-on-one study
– Interviewer speaks very little
– Best representation possible
– Example outcome: before vs. after
• Focus groups
– Moderator leads discussion with group
 Quantitative Methods:
Observational
• Observational methods
– In-person vs. mechanical
– E.g., Airport terminals
 Quantitative Methods:
Experiments
• Experimental & control groups
-- Double Blind design in medical research

• Manipulate independent variable across

Groups
-- placebo vs. real medicine
-- only 1 variable change between groups
-- keep everything else constant across
groups
Lab Experiment
• As long as participants know
they are in a study, it’s a “lab”
study. It doesn’t necessarily
proceed in a lab.

• Pro: accurate à
everything else is strictly
controlled
• Con: is the result
generalizable?
 Field Experiment Example
• Examine an intervention in the real world.

• Research question: Impact of price on revenue & profits

• Vary prices in different markets (A=$4,
B=$5, etc.)
• Compare data across price levels
 Hypothetical Pricing Experiment for
McDonalds
Column 1 2 3 4 5

Total cost
($300,000
Fixed Cost
Total + $2.00 Total
Unit Revenue Unit Profits
Demand at (Col. 1 x Variable (Col. 3 –
Market Unit Price Price Col. 2) Cost) Col. 4)
A $4 200,000 $800,000 700,000 $100,000

B $5 150,000 $750,000 600,000 $150,000

C $6 100,000 $600,000 500,000 $100,000

D $7 50,000 $350,000 400,000 ($50,000)

 “Natural” experiment
• Control conditions are determined by nature or by
other factors outside the control of the researchers.
 “Natural” experiment
• April 30 – July 13: 449; #4,709 on Amazon

• July 14: JK Rowling revealed

• July 15: #1 on Amazon; 228,000

Natural experiment
India’s new banknote
 Natural experiment
COVID-19 & WFH

• Offices are very expensive

-- WeWork
-- WeWork Disaster

• Without COVID:
--Volunteers? Self-selected.
--Mandatory for all?
-- Worst outcome is unbearable.
-- Fail at the run-in period.
 Key Differences Between
Qualitative & “Quantitative”
Qualitative “Quantitative”

Discovery/exploration of new ideas Validation/confirmation of estimates,

facts, key relationships
Preliminary understanding of key
variables
Deep understanding of (hidden)
motivations
Open-ended, unstructured format Mostly structured format

Small, representative samples; need Large, varied samples

researcher skill
Interpretive analysis Statistical, descriptive analysis
 Step 3: Decide on Sampling

• Probability: simple random,

stratified(divide subjects into groups before
using simple random)

• Nonprobability: convenience,
snowball
 The Survey Design Process
1) Plan What to Measure

2) Draft the Questionnaire

3) Pretest and Correct

Problems
• Note
– Get data that you can’t get from the rich
secondary data you already have
– There’s no perfect survey design
– All surveys are biased…
 Survey Design Step 1:
Plan what/how to Measure
• Ensure that information is relevant to
research objectives.
– For the survey, ask “What information do we
need to know to make a decision?”
– For each question, ask “How will this
information be used?”
• When in doubt, keep it short.
• In many ways, a survey is like a
conversation.
 Survey Design Step 2
Draft the Questions
• Question Content
– Make questions clear and concise
– Choose language to match respondents and
research goals
• Question Format
– Open-ended vs. closed questions
– Response scales
• Overall Layout
– Easy to read
– Order and grouping of questions
 Typical Survey Layout
Location Type Example (for potential DVR
customers)
Instructions: Start State purpose of survey, Thank you for helping us. We are
of questionnaire response requirements, interested in understanding how
deadlines, etc. Don’t make you choose electronic equipment.
it sound like homework. Your answers are completely
confidential.
Starting questions Broad, general questions Do you own a DVR?
Next few questions Simple, direct questions What brands of DVR did you
consider when you bought it?
Major portion of Focused questions, may be What attributes did you consider
questionnaire increasingly complex when you purchased your DVR?
Rank the following of a DVR
based on their importance to
you…
Last few questions Personal questions that What was your household income
may seem sensitive last year?
The Process of Editing a Question
1. Shouldn’t there be changes in the amount of
violence shown on TV?
• Yes
• No
– (problem: leading, begs for an affirmative answer)
2. Should there be changes in the amount of
violence shown on TV?
• Yes
• No
– (what kind of changes?)
3. Should there be less violence shown on TV?
• Yes
• No
– (ambiguous: fewer episodes of violence, or less extreme
incidents? –and- hidden assumption: what if respondent believes
more violence is better?)
 The Process of Editing a Question
4. How strictly should depictions of shootings and assaults on
network TV during prime-time hours (8 to 11 pm, Monday
to Thursday) be controlled?
• More strictly
• As they are now
• Less strictly
– (double-barreled question)
5. How strictly should depictions of shootings on network TV
during prime-time hours (8 to 11 pm, Monday to Thursday)
be controlled?
• More strictly
• As they are now
• Less strictly
– (too wordy)
6. Depictions of shootings on network TV during prime-time
hours (8 to 11 pm, Monday to Thursday) should be:
controlled more strictly, controlled as they are now,
controlled less strictly
 Questionnaire Format:
Response Scales
• Reported daily tv consumption
Version 1 % choosing Version 2 % choosing
Up to ½ hour 7.4 Up to 2½ hours 62.5
½ hour to 1 hour 17.7 2½ hours to 3 hours 23.4
1 hour to 1½ hour 26.5 3 hours to 3½ hours 7.8
1½ hour to 2 hours 14.7 3½ hours to 4 hours 4.7
2 hours to 2½ hours 17.7 4 hours to 4½ hours 1.6
More than 2½ hours 16.2 More than 4½ hours 0.0
 Questionnaire Format:
Response Scales

• Most important
– Keep it constant (thus comparable*)

*the most important for a survey, because surveys never reveal the public opinion
 Survey Layout
• Question order
– Think about possible dependencies

• How satisfied are you right now with your life as a

whole?
• How satisfied are you right now with your marriage?
• r = .32

• Reverse order: r = .67

• Lead-in instruction: “Next we will ask you two

questions related to your well-being, one about your
marriage and one about your life as a whole.”
• r = .18
 Survey Design Step 3:
Pretest & Correct Problems

• Give to small sample

• Look at results, ask direct questions
• Revise if needed
 Survey Example

Please indicate the importance of the following attributes to

you in selecting a new car:
Very Very
Important Unimportant
Mileage 7 6 5 4 3 2 1
Repair cost 7 6 5 4 3 2 1
Price 7 6 5 4 3 2 1
 Attributes of a new car
• Average mileage
– 10 mpg more than industry average
– Same as industry average
– 10 mpg less than industry average
• Average maintanance cost
– $100 / yr
– $500 / yr
– $1000 / yr
• Average price
– $18,000
– $25,000
– $32,000
 Conjoint Analysis:
• Ask people to make trade-offs, thus avoid a
lot of problems surveys have(inconsistency in
using scales, straightlining, etc)

• Predict how consumers make decisions when

facing complex choices

• Widely used in marketing, government,

psychology studies

• “A better version” of survey, so has all the

problems that survey has
 Conjoint Analysis:
Rankings of feature combinations
Car Buyer #1 Price: Price: Price:
Mileage $18,000 $25,000 $32,000
10 mpg less 6 7 9
Average 3 5 8
10 mpg more 1 2 4

Car Buyer #2 Price: Price: Price:

Mileage $18,000 $25,000 $32,000
10 mpg less 4 8 9
Average 2 5 7
10 mpg more 1 3 6
 Conjoint Analysis:
Ranking specific combinations
Concept Mileage Repair Cost Price
1 10 less $500 $18,000
2 average $100 $25,000
3 average $100 $32,000
4 10 more $500 $32,000
5 10 more $1000 $25,000
 Conjoint Analysis: JJ Lin Concert

• Research question: How to raise ticket prices

without hurting attendance?

• Attributes of multi-concert ticket packages:

– # concerts: 3, 6, 10
– Ticket price/seat/concert: $100, $180, $350
– Promo item: back stage meetings, autographed
photos, free food in concerts, JJ Lin hat or
jersey, $20 off local restaurant
 Conjoint Analysis
• Useful when product or service can be
decomposed into attributes
– Mutual Fund = Past Returns + Fees + Brand
Name + Online Access
– Commercial Aircraft = Capacity + Max Range +
Fuel Efficiency + Price + Cost of Service Contract
+…..
– Fashion clothing? Wedding dress?
• Transparent
• Predicts preference, not necessarily
purchase
 Low-cost Market Research
• Visit the competitor’s store

• Become a customer

• Invest in the competitor

• Talk to your/their customers

• Browse government records

 Big Data
• Previous known as “junk data”

• Not (necessarily) about the size, but about

the nature of the data
 Big Data
• What makes a player keep playing an
online game?
• Number of friends in the
game

• Achievements/Goals in the
game

• Nice graphics

• New features being added

regularly

How about what the gamer typed in the game?

 Big Data
• Traditional data analytics sees majority
(say 80%) of the whole picture at a
relatively low cost

• Big data analytics gain some more

understanding(say 90%) of what’s really
happening with a higher cost

• Better computing equipment & better

algorithms made it possible & cost efficient
to do big data analytics
 Big Data Methods(For Fun)

• MCMC(Markov chain Monte Carlo)

• Machine Learning

• Deep Learning

• Neural Networks

But there’s no explanation for why it works

when you use these big data methods.
 Chess:
40^40 possibilities

Go:
200^211 possibilities
Sample finding by (recent) research

• Why are ratings for great products ever-

decreasing?
 Today’s class:
Marketing Research
• Types of data: secondary vs primary

• Types of data collection: qualitative vs

quantitative

• Survey Designs

• Conjoint Analysis

• Quantitative methods: lab experiments, field

experiments, natural experiments

• Big data(what it is)