Professional Documents
Culture Documents
Measure To Grow: Drive Double-Digit Growth by Measuring Marketing Right
Measure To Grow: Drive Double-Digit Growth by Measuring Marketing Right
TO GROW
Drive double-digit growth by
measuring marketing right
NOVEMBER 2021
Boston Consulting Group partners with leaders in business and society to tackle their most important challenges
and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963.
Today, we help clients with total transformation—inspiring complex change, enabling organizations to grow, building
competitive advantage, and driving bottom-line impact.
To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry
and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge
management consulting along with technology and design, corporate and digital ventures—and business purpose.
We work in a uniquely collaborative model across the firm and throughout all levels of the client organization,
generating results that allow our clients to thrive.
Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook
launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further
empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences
like augmented and virtual reality to help build the next evolution in social technology.
2 MEASURE TO GROW
Presenting the industry perspective
Digital marketing is the fastest growing channel, and already accounts for 30%+ of total marketing spends
in India. Marketing measurement capabilities and approach, however, have not kept pace. Moreover, there is
limited consensus on the best approach to measure impact of marketing investments on business outcomes.
Significant value is being left on the table and there is opportunity to unlock multi-fold growth through
robust measurement.
BCG, Meta and key stakeholders across the ecosystem have come together to present an authoritative view
on best in class marketing measurement that can become the launchpad for growth.
We are thankful to the leading measurement specialists, who provide solutions across the globe and in India, for
their valuable inputs in developing this report.
Cartesian
Consulting
We have also gathered learnings from 18 leading Indian digital-first organizations for a holistic view of
measurement maturity in India and current best practices.
We are grateful to these organizations and their leadership for their invaluable contribution.
3
Digital marketing Significant value
is the new can be unlocked by
mainstream measuring right
AT A
digital-first organizations
GLANCE
70% of marketing
spends on digital
25% Gross Margin
4 MEASURE TO GROW
Unlock growth Global shift Indian organizations
by measuring towards have started on the
“Incrementality” “Incrementality” journey
are still
Incremental While using basic
lift methodologies
eBay Airbnb 90%+ for key strategic
Netflix Domino’s decisions
Booking.com Walmart
Expedia Salesforce have started
Control Test Viber
10-20% measuring
incrementality
5
CONTENTS
TABLE OF
6 MEASURE TO GROW
08 16 32
Value left on Maximizing Getting there
Corner
Practitioners’
the table growth by Measurement
Why measurement measuring right maturity in India and
matters – now The industry- the way forward
recommended
approach
7
01
VALUE LEFT ON THE TABLE
WHY MEASUREMENT MATTERS – NOW
8 MEASURE TO GROW
VALUE LEFT ON THE TABLE 9
Digital marketing spends are exploding
Share of marketing spends by channel1 (%) Share of digital in total marketing spends4 (%)
90%
75%
40% 39% 40% 39%
70%
TV
65%
1. Dentsu Digital Advertising in India 2021 2. Estimated 3. Forecasted 4. BCG Measurement Maturity Assessment of leading digital-first organizations in India-2021
10 MEASURE TO GROW
But significant value is being left on the table
Measurement practices have not kept pace CXOs can precisely forecast impact
Only 9% on ROMI from change in spends3
with digital adoption
• Large and growing digital marketing budgets are have high-quality data that
inaccurately measured Only 5% enables key business decisions3
1. BCG - Meta research and analysis 2. Nielsen Compass Benchmarks 3. BCG global marketing expert survey (N=100) 4. BCG Measurement Maturity
Assessment of leading digital-first organizations in India - 2021
1. Gartner, “3 Steps for Marketers to Prepare for a Cookieless World”, July 2, 2021
12 MEASURE TO GROW
Move to best-in-class measurement is the launchpad for growth
Organizations
that drive focus on
measurement as
a strategic priority ₹
10x RETURN ON AD SPENDS
see significant
value across key
metrics
Note: The figures above represent the maximum increase realized for respective metrics; impact is applicable to business driven by digital performance marketing
Source: BCG Experience with leading Indian and global organizations across Financial Services, Media, Technology, Retail
14 MEASURE TO GROW
IN COLLABORATION BEST LEADING INDIAN
WITH PERFORMANCE PRACTICES ORGANIZATIONS
MARKETING FROM GLOBAL ACROSS INDUSTRIES
MEASUREMENT LEADERS ACCOUNTING FOR INR
SPECIALISTS 2K CR DIGITAL SPEND
Cartesian
Consulting
16 MEASURE TO GROW
MAXIMIZING GROWTH BY MEASURING RIGHT 17
Keeping up with the consumer
Key questions/decisions organizations face today, driven by complex customer journeys
1. Lines Of Business
18 MEASURE TO GROW
Searching for answers
in a complex measurement ecosystem
Measurement specialists
Firms offering niche capabilities:
measurement analytics, Divergent
experimentation, etc. recommendations
• Different answers from
each part of the ecosystem
Web App
measurement measurement Need for clarity on
Desktop & mobile Mobile app methods and metrics
site analytics analytic partners:
Adjust, AppsFlyer, • Limited consensus on the
tools: GA,
Branch, Singular best methods and metrics
Omniture, etc.
etc.
Organization
(In-house teams)
2020 MEASURE
MAXIMIZETO
GROWTH
GROW BY MEASURING RIGHT
Unpacking the metrics
Campaign
Methodology Pre-post tests Rules-based Multi Touch Marketing Mix User- Geo-
attribution Attribution (MTA) Models (MMM) based based
– Statistical lift test lift test
Approach • Compare movement • Pre-defined rules to • Statistical model • Regression-based • Compares impact across
in KPIs before & assign credit based on to allocate credit to model to isolate geographies/ user sets using test
after a campaign, touch point(s) in the different channels impact of multiple & control groups; measures the
crediting positive path to conversion on pathway to factors (media, real (incremental) impact of ads
movement to purchase promotions, market • Requires an experimental design
marketing • Data-driven dynamics) driving where a subset of population or
• Typically used by approach to sales lift geo unit is held out from ads, to
in-house teams to understand • Measures impact compare impact
optimize campaign impact of multiple of both online and
inputs channels offline media
Industry • Results biased by • High ease of use - • Provides real- • Measures • Enables an understanding
factors other than Used by 90%+ orgs. time and granular incremental of the incremental impact
view ads (seasonality, etc.) Provided by MMPs & output, can be outcomes of marketing interventions
• Industry publisher consoles deployed at high • Requires • Does not provide real time
recommendation: • Can over-report frequency considerable results
minimize use impact of ads for high • Dependent on investment;
intent audiences signals; weight Recommended for
• Critical to guard allocation can be a marketing spends >
against click fraud black box INR 50 Cr
Pros & cons
Ease of use
Speed to insight
Signal-loss resilience
Provide
incremental output
Best Average Head to the Practitioner’s Corner where we bust some common myths regarding these methods
22 MEASURE TO GROW
No silver bullet – a mix of Strategic
methodologies needed decisions
Media
mix - channels,
geographies, LOBs
THE CODE
INDUSTRY
RECOMMENDED
APPROACH FOR Think incrementality - make it a key Only incremental output
KEY MARKETING part of the organization’s
credited to marketing
intervention
DECISIONS measurement DNA
Incrementality 20%+
• Incrementality is the ‘Incremental’ impact
delivered by marketing activities i.e., Impact
that would not have happened but for the
marketing intervention
Control Test
Operational
High- 2 Deploy User/ • Use for high investment activities Traditional ML-based
impact Geo based Lift (new channels, large campaigns, Geo tests Geo tests
new market launches, etc.) Complete media Controlled
decisions tests to define • Ideal test duration: 4-6 weeks black-out in media testing in
e.g., right digital channel • For Geo tests, use ML-based control geography synthetic control
digital mix mix based on synthetic control groups to avoid
incrementality going completely media dark in
control geography
For details on how incrementality can uncover additional information for the optimum media mix, head to the Practitioner’s Corner
1. Marketing Mix Model
24 MEASURE TO GROW
Case Study
Org context Online fashion major was seeking the ideal media mix between digital and offline channels
Developed and deployed MMM to define media mix based on incremental impact; then
Approach calibrated the model with Lift Experiments
2.5x 3.5x
Realized
Higher ROAS higher ROAS Attribution based
Allocated budgets across channels based on incrementality, and continued to run user-
Implication based lift tests to validate the model output
MMM helped us bring cross-media elements into a single measurement platform and
understand their individual contributions to incremental traffic. This helped us optimize our
Org speak overall media investments
- Head, Marketing Strategy & Analytics
Operational
Optimizations (campaign,
channel, creatives…)
0.8x
Optimal 44 Optimize • Run user lift test to identify best
campaign campaigns for campaign settings (e.g., bidding strategy,
incrementality attribution window)
settings to
• Use the settings that delivered
maximize higher incremental outcomes for
impact future campaigns Basis Basis Lift
Attribution tests
Identifying 55 Use A/B tests to • Run A/B tests with variants of creative/ • Calibration factor of 0.8x
optimal identify optimal copy to be tested, all other settings kept identified via lift tests
the same • Applied to output from
campaign campaign inputs • Creative/ad copy with higher outcome attribution model
inputs (CTR, CVR etc.) to be scaled up for performance
(creatives, ad • Avoid Pre-Post tests; high bias due measurement of
copy etc.) to seasonality future campaigns
26 MEASURE TO GROW
Case Study
Industry leaders in India are calibrating attribution models with lift tests
and optimizing for incremental outcomes
Leading Ed-tech player
calibrated attribution with user- Leading Travel player optimized
based lift tests campaigns for incrementality
The organization sought to identify the higher-ROI
Industry leading player sought to ascertain the true
optimization approach between:
Org context impact of activities on a particular marketing
• 7 day click (approach 1)
channel on app installs
• Or 7 day click + 1 day view (approach 2)
Identified the ‘calibrating factor’ for attribution Ran user-based lift test across both approaches,
Approach models through user-based lift tests and compared the incremental installs & purchases
Higher installs seen from Channel X basis lift tests Higher installs and purchases seen basis approach 2
1.25x 1.7x 5x
Results
Basis attribution Approach 1
Basis incremental lift Approach 2
Installs Installs Purchases
Cartesian
Consulting
NO SILVER BULLET -
A MIX OF METHODOLOGIES NEEDED
Minimize use of
• Pre-post tests
• Define media mix (offline vs. online) based
to decide
on incrementality, enabled by MMM MMM
media mix
• Calibrate model with lift tests
Strategic • Only click/
decisions • Deploy Lift tests to measure incremental
touch-based
metrics
Media mix - channels, impact of key decisions, such as digital Lift tests to define
geographies, LOBs spend mix, investment in new channel etc. media mix
• Pre-post tests
Operational to optimize
decisions • Optimize campaigns for incrementality Lift tests campaigns,
Optimization - campaigns, comparing
channels, creatives, etc. results
across time
• Use A/B tests to identify optimal
A/B tests
campaign inputs e.g., creatives, ad copy
28 MEASURE TO GROW
Hear it from the experts
When it comes to measurement, many With incrementality, marketers can Make the most of your marketing
different methodologies are available answer critical questions about their budget by linking marketing
to marketers; which ones to use, when marketing performance – questions performance to actual business
to use them, how to use them – these like “How do I know the real impact outcomes. We advise our clients to
are the decisions that separate great of my marketing investments?” and focus on all parts of the marketing
organizations from merely good ones “Which campaigns really move the funnel and have an optimal cadence
needle?” of strategic and tactical solutions to
inform marketing decisions
We are seeing a clear shift in the It is important that a methodology Marketers should take help from AI/
measurement approach of mature provides granularity in measurement, ML and use advanced algorithmic
organizations; they are now looking to both at the user level and in the attribution models, which will give
calibrate their MMM & MTA models sequence of interactions. This enables them direction on providing adequate
with lift tests nuanced decision making and deeper credit to channels, without any bias
insight generation
Incrementality is one form of measurement that ensures that we actually understand what
our actions are, and because we know what our actions are we can affect our actions to 15-25%
improve our business. − Stephan McBride, Director of advertising science (Netflix1) gross margin
impact5
Randomized control experiments are the gold standard for marketing measurement, Despite
the challenges, the appropriate use of randomized controlled experiments should continue to
grow in marketing. − Matthew Taylor, Effectiveness specialist (Google2)
25%
increase in LTV5
Smart reallocation of wasteful marketing spend and optimization to incrementality will
generate a 2-3x increase in incremental ROI and drive continuous growth of overall sales. 6x
− Vadim Tsemekhman, Director of product management (Walmart3) higher
conversion rate5
If you want to gain a competitive advantage, your firm should build an experimentation
capability and master the science of conducting online tests.
− Steven Thomke, Professor of business administration, (Harvard Business School4) 35-40%
higher ad
performance6
1. Meta, “Unlock Business Growth with incrementality measurement”, January 2018 2. Google, “Three Grand Challenges”, June 2019 3. Association of National
Advertisers, US, Meeting presentation, October 2020 4. Harvard Business Review, “The surprising power of online experiments”, Sep-Oct 2017 5. BCG experience 6.
Harvard Business Review, “Marketers Underuse Ad Experiments. That’s a Big Mistake.”, October 28, 2020
Note: The impact on business metrics is applicable only to business driven by digital performance marketing
Source: BCG Experience with leading Indian and global organizations across Financial Services, Media, Technology, Retail
30 MEASURE TO GROW
Case Study
Incrementality in action
Leading global sports retailer achieved $60M higher profit by adopting Incrementality
Context Revenue growth had slowed, even as costs were on the rise
$50-60Mn $500Mn
Impact
incremental inefficient
annualized marketing spend
net profit identified
32 MEASURE TO GROW
GETTING THERE 33
Holistic measurement maturity assessment of leading
digital-first organizations
On the journey
• Basic measurement models deployed
INR 2k cr 3 levels of • Key skills in place, developing tech & data
digital spend maturity
First steps
• Minimal focus on marketing measurement
• Minimal to no specialist skills, tech or data
34 MEASURE TO GROW
Early days, but promising signs
Most Indian organizations still taking first steps – but some are well on the way
Measurement capabilities ... but pockets of excellence Learnings from those pushing
still nascent… across industries the envelope
Overall philosophy and vision
making adequate investment
use only rules- 30% in measurement
• Data-driven measurement insights
drive marketing decisions
based attribution
90%+ for key strategic Adoption of advanced measurement techniques
decisions Metrics captured
deploy deploy • Focus on incremental metrics, in
20%+ MMM 10%+ Lift tests addition to click based metrics
Methodologies deployed
• Have started using Experimental/
Pockets of excellence Statistical methodologies to guide
across industries key business decisions
Pushing the envelope • Use of calibration to augment efficacy
Players across FS, eCom,
Ed-tech, Media driving of attribution
excellence in measurement
Organization capabilities
• Dedicated team with new-age
specialist roles for measurement;
reporting to CMO
On the journey
• Top-down culture of ‘Test & Learn’
GETTING THERE 35
Technology, data and organization
Mature organizations have every base covered
Measurement specialists can augment capabilities, head to the Practitioner’s Corner for details
1. % of top three that fulfill criteria 2. % of bottom three which fulfill criteria 3. Number of specialist skills being more than or equal to 5
Source: BCG Measurement Maturity Assessment of leading digital first organizations in India - 2021
36 MEASURE TO GROW
Building measurement maturity – in five steps
Impact
Think
incrementality
Mix of 60%
methods Sales
Measure Deploy the right uplift
incremental impact methods based
of all marketing on the decision
interventions hierarchy; calibrate
with lift tests
10x
ROAS
GETTING THERE 37
PRACTITIONERS’
CORNER
38 MEASURE TO GROW
Myth-busting
Incremental lift tests give results in • Sophisticated measurement solutions available globally can
batch mode, and not immediately provide incrementality results in just 24 hours1
PRACTITIONERS’ CORNER 39
The additional information that can be uncovered by
measuring incrementality
Conversions recorded Overlap of conversions
Why you need to Incremental conversions
1 basis last-click attribution 2 from the two channels 3
start measuring driven by channels
for Channels A & B with organic conversions
“incrementality” for
optimum media mix
ORGANIC ORGANIC ORGANIC
Possibility of under/
over reporting true Channel A
impact from last click
measurement due to:
• Clicks lost due to URL
stripping or click fraud
• Complex journeys
across devices /channels Channel B
• If a customer converts
after a view, without
clicking
Higher last-click based Lesser overlap with organic Incremental conversions:
conversions for Channel A conversions for Channel B Higher for Channel B
Source: Inputs from measurement specialists and industry experts; BCG analysis
40 MEASURE TO GROW
Sub-optimal MTA models often deployed
• Marketers generally use the default MTA option available in
publisher tools
• Inertia to explore sophisticated MTA models with multiple
capabilities
PRACTITIONERS’ CORNER 41
Capabilities can be fast-tracked and augmented through collaboration with
measurement specialists
WEB APP
Measurement
specialists
Key methodologies MTA Attribution models, Incrementality-based lift tests
supported
• Algorithmic attribution across multiple touchpoints • Visibility of customer’s end-to-end mobile journey
• Customer journey analytics to understand across multiple touchpoints
touchpoints/ drop off points across the journey • Advanced models for attribution across multiple
• Cohort analysis of customers with shared touchpoints
characteristics • Profiling customers and defining audience
• Real-time segmentation of customers using segments to enable the creation of test and
ML-based algorithms control groups
• Experimentation and real-time analytics of test
results
Key offerings • Sophisticated, multi-layered, mobile ad fraud
prevention and detection
“For understanding how well different marketing “With the help of AppsFlyer’s incrementality solution,
campaigns are doing, we go to the Adobe Analytics we were able to uncover critical insights and achieve
suite. It’s our eyes and ears to our customers’ journeys better and faster results. It helped us scale up rapidly
and experiences. It enables us to make the right and effectively.”
Success stories decisions on time.”
Senior Director, Analytics Head, Growth Marketing
(E-commerce leader) (Gaming leader)
42 MEASURE TO GROW
CONSULTING
Cartesian Consulting
• Dynamic calibration of MMM • Segmentation & selection of • Algorithmic modeling for data • One-stop shop for MMM -
with real-time data identical customer groups for driven attribution to multiple – data integration, modelling,
• Forecasting & simulation of tests using techniques such as touchpoints visualization and prediction
marketing mix using scenario look-alike modelling • Online platform for scenario • Automated online scenario
simulation • Execution of multiple tests while planning, forecasting & planning, simulation and
• DIY predictive analytics measuring isolated impact– optimization using MMM dynamic mix optimization
capability to measure ‘lift on lift’ • Audience profiling through
with cloud-based, AI-driven with MMM
• Attribution modeling using comprehensive data on
solution advanced algorithms such as interactions across multiple • ML-based geo selection and
• Synthetic geo selection for lift Markov chain channels - mobile, TV, print analysis of test results
tests and predictive analysis • Customized analytics through & online
basis results advanced analytical techniques • Experimentation
such as genetic algorithms through lift tests
Cutting Edge Offering: Cutting Edge Offering: Cutting Edge Offering: Cutting Edge Offering:
Always-on (real-time) ROMI Dynamic budget allocation through Proprietary calibration technology Live MMM with daily insights for
insights & forecasting using MMM ‘Multi-armed Bandit’ algorithm for for MMMs; MTA in cookie-less world analysis and optimization
calibrating MMM with test results with Nielsen ID system for customer
identification
“Building MMM with Analytic Edge “Using Cartesian’s NoC (Net “The attributed metrics, enabled “With RainMan, we have driven
has helped us assess our marketing of Control) Lift measurement by Nielsen’s MTA solution, are effective media solutions and pricing
performance and optimize media methodology, we’ve been able to our source of truth. We can easily strategies. They enabled us to move
mix real-time. It enables us to make identify 8% increment in sales from identify channels or tactics being beyond the basic questions of ‘how
better decisions in an agile way.” specific marketing efforts, which under or over-reported by last paid much and how to invest effectively
were otherwise hard to attribute click, plan budget allocation more in media.’ It has helped the business
Head, Digital Analytics impact to. This ability really helps effectively, and know which levers to to deploy the analytical solution into
(Global computer hardware leader) us in sharpening our allocations and pull to drive site traffic and sales.” transformational action.”
focus on what is delivering results.”
Head of Marketing Senior Digital Marketing leader Head, Business Science
(Global fast-food chain) (Global travel & tourism leader) (Consumer goods leader)
Note: Select examples
Source: Measurement partners; BCG research
PRACTITIONERS’ CORNER 43
NOTE TO THE READER
AUTHORS
Shaveen Garg Pallavi Malani Vikash Jain Nimisha Jain Suhail Bhansali
Managing Director Managing Director Managing Director Managing Director Project Leader
and Partner and Partner and Senior Partner and Partner
BCG, Delhi BCG, Mumbai BCG, Delhi BCG, Delhi BCG, Mumbai
Core member of the Deep expertise in Leads the Technology, Leads the Expertise in
Technology, Media & Digital Marketing Media & Telecom Marketing, Sales Digital Marketing
Telecom practice practice in India and and Pricing practice
co-leads BCG’s growth in Asia Pacific
tech efforts.
garg.shaveen@bcg.com malani.pallavi@bcg.com jain.vikash@bcg.com jain.nimisha@bcg.com bhansali.suhail@bcg.com
44 MEASURE TO GROW
ACKNOWLEDGMENTS
This study was undertaken by Boston Consulting Group (BCG) in association with Meta.
We thank David Ratajczak - Managing Director and Senior Partner in the Atlanta office of the Boston Consulting Group, and Neal Rich
- Partner and Director, Marketing in the Denver office of the Boston Consulting Group, for their contributions towards the development
of this report.
We are thankful to Ankit Shrivastava, Navnoor Kaur and Chirag Singhal for their assistance in writing this report.
We are thankful to Jasmin Pithawala for managing the marketing process as well as Jamshed Daruwalla, Saroj Singh, Vivek Thakur, and
Akshata Thakur for their contribution towards design and production of this report.
To find the latest BCG content and register to receive e-alerts on this topic or others, please visit bcg.com.
Follow Boston Consunting Group on Facebook and Twitter.
11/21
45
46 MEASURE TO GROW