How Online Display Ads Work

Lessons from 432 Field Experiments

Garrett Johnson
Assistant Professor
Questrom School of Business garjoh.com
Boston University
*Based on joint work with Randall Lewis & Elmar Nubbemeyer garjoh_canuck
432 online display ad experiments on
Google AdWords
● 431 diverse advertisers
● 4M users on average
● Videos & banners
● Metrics: Site visits & conversions
● Predicted Ghost Ad metholodogy*
○ "Brand Lift" & "Conversion Lift" products at Google

*Johnson, Lewis, & Nubbemeyer (2017 JMR)

Example campaign*

*Hypothetical advertiser. Not in sample.

Brand lift metrics (unobserved)
Observe site visits

*Hypothetical advertiser. Not in sample.

Observe conversions

*Hypothetical advertiser. Not in sample.

Q1: How well do display ads work?
 200 Visits
Visits Lift (%)

100

-100

Experiment
16.6%
Median lift in visits
 Visits
Conversions
Conversions Lift (%)

Experiment
 8.1%
Median lift in conversions
Q2: What is the carryover effect of
display ads?
Ad carryover example
During After
Campaign Campaign
Treatment
1200 visits 1000 visits
Carryover
LiftAfter/LiftDuring
Control
1000 visits 900 visits =100/200
=50%

Lift
200 visits 100 visits
Negative carryover example
During After
Campaign Campaign
Treatment
1200 visits 850 visits
Carryover
LiftAfter/LiftDuring
Control
1000 visits 900 visits =-50/200
=-25%

Lift
200 visits -50 visits
 3%
Median carryover for visits
(4 weeks)
Carryover effect by advertiser (visits)

+ / −
58% 32%
Q3: How do the incremental effects
of display ads propagate down
the marketing funnel?
Translating proxy outcomes
Proxy outcomes
survey responses, clicks, ad engagements, online search volume, site visits

Sales
Translating proxy outcomes
Proxy outcomes
survey responses, clicks, ad engagements, online search volume, site visits

Sales
Translating funnel outcomes
Site visits

Conversions
Marketing Funnel:
Baseline vs. Incremental
Site visits

ental
ne
Baseli
Increm
Conversions
Should we expect incremental visitors to
convert at same rate?
Site visits

Conversions
Should we expect incremental visitors to
convert at same rate?
Site visits Site visits

Conversions Conversions
How does lift translate down the
funnel?
10% lift

tal
ne
men
Baseli
10% liftIncre
How does lift translate down the
funnel?
10% lift ntal
10% lift

tal
ne

ne
men
eme
Baseli

Baseli
Incre
Incr

5% lift 10% lift

How does lift translate down the
funnel?
10% lift ntal
10% lift 10% lift

ntal
tal
ne

ne

ne
men
eme

Increme
Baseli

Baseli

Baseli
Incre
Incr

5% lift 10% lift 20% lift

% Lift decreases down funnel
10% lift ntal

Incremental users are

ne
eme
Baseli

worse prospects
Incr

5-7% lift