Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Syllabus MKTG 6230 - Driving Marketing Performance: Measure, Analyze,

Profit

Schedule: Thursdays 7:30-9:30PM, starting January 21st 2021

Office Hours: Thursdays by appointment: I will be in my office before class (e.g. 6.30-
7.15 pm). Outside, the best way to reach me is via e-mail (k.pauwels@northeastern.edu).

Text book:
“It’s not the Size of the Data – It’s How You Use It: Smarter marketing with analytics and
dashboards”, Koen Pauwels, AMACOM, 2014

Course Cases and Readings:

Harvard link to the Harvard cases: https://hbsp.harvard.edu/import/786815

Most readings are 4-page summaries of scientific research from the ‘Truly Accountable
Marketing’ Special Issue of the Marketing Intelligence Review, for free at link:
http://issuu.com/gfkmir/docs/komplett_eng_accountable_marketing?
e=15611816/12576717

My own cases and links to other readings are on Canvas. Please go to

https://canvas.northeastern.edu and make sure you are enrolled in this class.

Course Description
Your company is going to spend millions on a new marketing or strategic initiative,
but how will you know if and when it is working? Sooner or later, you will be asked to
either evaluate or defend proposed initiatives for which a ‘back-of-the-envelop’ Net
Present Value calculation just won’t do. Marketing performance measurement and
feedback systems enable managers to take smarter risks by assessing experimental
projects and forecasting the profit potential of bigger, bolder initiatives. In this respect, it
brings together several key business areas, including marketing, strategy and finance.
This course enables you to assess and evaluate the profit impact of marketing
expenditures and to develop ‘market dashboards’ that summarize marketing productivity
and suggest steps for performance improvement in marketing strategy and tactics. We
will review and practice tools developed by high performance marketers such as Amazon,
Avaya, Vanguard, Johnson & Johnson, Google, General Electric, Diageo, Harrah’s,
Capital One, General Mills, Unisys, HSBC, Dupont, Gillette and Starbucks. Through
several real life cases and readings, we apply these tools to a variety of industries and
market actions. In this course, you will:
 Learn appropriate & relevant ways to measure Return on Marketing Investment
 Enhance your ability to propose and defend marketing budget (re)allocations
 Benchmark your approaches against firms leading in marketing effectiveness
 Sharpen your skills of communicating accountability throughout the company
 Blend marketing intelligence and financial language in choosing the best metrics
 Integrate online and offline insights for improved marketing effectiveness
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Learning Objectives
This course will give you the opportunity to build skills in how to:
1) build a system for measurement and improvement of return on marketing
investment
2) interpret and critically evaluate the expected benefits, costs and risks for proposed
offline and online marketing activities
3) benchmark your approaches against firms leading in return on marketing
investment
4) communicate marketing accountability throughout the organization

Teaching philosophy
I strongly believe in the experiential learning approach which builds on your input
and business experience to share and refine key concepts and tools from my experience.
In particular, you will (1) propose marketing initiatives to your colleagues, (2) evaluate
the marketing initiatives of your colleagues and (3) defend your point of view in written
and oral form. Moreover, your input in the syllabus is strongly encouraged, and you have
a choice in terms of (1) who to work within a team, (2) which topic to pick for your final
project, and (3) when to spend most of your time effort regarding class case discussions.
Half of our class sessions will be devoted to discussing cases (in italics in the
course outline). The remaining sessions are participative lectures to introduce key
concepts and tools to address issues in marketing productivity. In any session, you are
strongly encouraged to ask questions. Asking the right questions and framing them
appropriately are an integral part of the learning process. Likewise, deciding which
information matters to you is crucial given the data overload managers face today. For
one, choosing the relevant data from a case requires some thought. For another, the
reading assignments include some articles that appeared in a leading academic marketing
journal. You are not expected to follow the technical details of model estimation and data
measurement, but should be able to draw appropriate conclusions from abstract,
introduction, results discussion and relevant tables and figures.

Optional Reading and Resources

Given the high managerial relevance of the topic, several recent books and websites
cover (Marketing) Productivity, Accountability and Dashboards. Please share your own
experience with me and the class – I consider the following very useful:

Books:
 ”The Big Book of Dashboards: Visualizing Your Data using Real-World Business
scenarios”, by Steve Wexley, Jeffrey Shaffer and Andy Cotgreave, Wiley, 2017.
 “Your Gut Is Still Not Smarter Than Your Head. How Disciplined, Fact-Based
Marketing Can Drive Extraordinary Growth and Profits”, by Kevin Clancy and
Peter Krieg, 2007
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Websites:
 https://visionedgemarketing.com/customer-case-studies/ is dedicated to ‘Creating
a Performance Driven Marketing Organization’ and has many case studies to
choose from,
 http://www.marketingprofs.com/ has several articles and dozens of user questions
and answers (in the ‘Forum’ section) on Research/Metrics.
 My blog: https://analyticdashboards.wordpress.com and website notsizedata.com

Course Requirements
During the term, students’ individual and group work will be evaluated based on their
performance in each of the following activities:

Class participation (individual) 25%

Assignments (4, individual, 5% on each) 20%
One Self-selected Case Presentation (individual) 15%
Final Project (individual or team – let me know by week 4) 40%:
February 4: Project choice (0.5 page: company, issue, team)
March 4: Detailed Project description (2-3 pages)

April 22: Full report presentation (15-30 slides)

Classroom Participation:

Effective class participation is judged by the instructor based on the following:

1. Is the participant prepared? Do comments show evidence of analysis of the

method or case? Do comments add to our understanding of the situation? Does
the participant go beyond simple repetition of facts without analysis and
conclusions? Do comments show an understanding of application?
2. Is the participant a good receiver? When reacting to colleagues’ proposals, does
the participants give constructive critique? Are the points made relevant to the
discussion? Are they linked to the comments of others? Is the participant willing
to interact with other class members?
3. Is the participant an effective communicator? Are concepts presented in a
concise and convincing fashion?

In other words, you will be evaluated on how well you respond to the questions and how
effectively you take into account the comments and analyses of your classmates. To give
you an idea of how I grade classroom participation out of 10:

1. 10/10 for someone who is an outstanding leader in class discussion - always

prepared with readings and cases, frequently initiates stimulating and thought-
provoking questions, often brings outside articles and examples that illustrate
course concepts, and serves as a role model building on the comments of other
students. Shows up on time and has perfect attendance (one miss is fine as long as
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

you emailed me ahead of time).

2. 8/10 for a consistent contributor to class discussion who responds to instructor
questions and comments. Rarely late and e-mailed professor ahead of time for any
missed classes.
3. 6/10 - Occasional contribute when called upon but never volunteer answers/
comments/insights on your own.
4. 4/10 - Someone who regularly attends class but does not take part in class
discussions (i.e. simply attending every class).
5. 2/10 – Being regularly late or absent from class without prior approval
6. 0/10 Completely avoids any attempt to contribute to the classroom.

Class Conduct Policy

■ My primary goal for this semester is to create a learning environment where I can
provide a top-quality, safe, and healthy educational experience to all students. As
we are all in this together, we must work together to make smart health decisions.
Accordingly, if you are not feeling well or may have been exposed to someone
who is sick, please make the safe choice to attend class virtually. To simplify this
decision, I will do my absolute best to ensure students have equal opportunity to
participate and equal access to course materials regardless of whether you attend
in-person or virtually.
■ If students are feeling well and decide to attend class in-person, they must
properly wear a face mask (fully covering their mouth and nose) and observe safe
social distancing requirements in accordance with our University’s guidelines
(https://news.northeastern.edu/coronavirus/reopening/workplace-safety-
protocols).
■ To make sure masks can be worn properly at all times, we will have to enact a no
eating/drinking policy in the classroom this semester. I understand this might be
tough given your schedule, so if you need to eat/drink during a class session,
please leave the classroom and find a safe place to do so. Feel free to return to the
classroom once you are done.
■ As we are all still learning these new procedures, all students and the professor
should feel empowered to voice concerns about any non-compliance with safety
measures. After one warning, any student who does not comply with these safe
guidelines will be kindly asked to leave the physical classroom space immediately
to ensure the safety of others.
■ Because failure to stay safe in the classroom puts yourself and others at risk,
[enter your own enforcement consequence if you choose to include this bullet].
■ Thank you all for helping me create a “culture of care” in our classroom so we
can “protect the pack” and all stay healthy and safe in class!

Class Recording Policy

In order to create an accessible learning experience, this class will be recorded for the
benefit of all students, including those who may have difficulties accessing all
synchronous materials. By participating verbally in any live session, you grant consent to
be recorded for the purposes of posting the video lecture recording on Canvas for this
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

class section only. If you have any concerns about being recorded, please discuss them
with me immediately so that we can find a resolution.

Absence or Late Arrival

If you know you have to miss / be late for a class, you need to email me for prior
approval. To get class participation credit for a class in which a case is discussed, please
email me your personal answer to the case questions in a 2-3 page write-up. If no case is
due, email me a 1 page summary of the reading

If you have an urgent, unforeseen reason for absence or late arrival, please email me as
soon as you can. I am understanding.

Use of wireless devices in class: Students are expected to act professionally in the
classroom. Therefore, laptops, tablets and smart phones are permitted for use during class
time only for taking notes or as specified by the professor as a part of class activities –
including spreadsheet calculation and online quizzes. Use of chat programs, web surfing,
and other non-class related activities devices are strictly prohibited. Failure to adhere to
this policy will jeopardize a student’s class participation grade.

Assignments:

Due by email before noon on the day of class

Format:
 12 point font
 Times New Roman
 1 inch margins on top, bottom, left and right margins
 Double spaced

Late: Assignments Will Not Be Accepted without Prior Approval

If you know you are going to be absent we can work something out. Assignments not
received on the date due will not be accepted, unless prior approval has been obtained
from the instructor. For exceptions, see the ‘urgent, unforeseen’ clause above.
Self-selected Case Proposals
Half of our class sessions will be devoted to case discussions (in italics in the course
outline). For most cases, you will prepare as usual; i.e. know and understand the case, be
ready to answer the case questions in class and to evaluate choice options.

For 1 case you select amongst the 6 cases, your preparation will go further. In particular,
you will need to argue for your proposed decision with both a 5-10 minute oral
presentation based on a 3-10 slide presentation. This presentation is due at noon of the
day of class by email to instructor, so I can plan the session for optimal learning.
Indeed, one of your peers will do the same, but argue for another decision option; as may
happen in a business meeting. You will receive feedback from your colleagues and from
me, and may use this feedback to revise your final hand-in of slides, which I will grade.
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

To match supply and demand, I need you to indicate 2 cases you would prefer. Which
cases to pick? Ideally, at least one that relates closely to your own question for your final
project or an industry that you like (e.g. if you want to improve online marketing metrics,
you may prefer the Mednet.com (session 3) or the eSig case (session 9), if you like to
cook or eat, Culinarian (session 5) or Endless Crab Feast (session 8) is for you and snacks
are the topic for week 6).

Final project
You are encouraged, from the very beginning, to choose a specific project you are
interested in developing. This will typically involve your own (or a fellow student’s or
friend’s) company/industry. You may select any company or business unit: it is not
necessary to obtain “inside access”: use of publicly available information is perfectly
acceptable if you can obtain the necessary data. You may develop this final project by
yourself or in your group of at most 3 students. To help you manage your time and give
you feedback, I am putting several milestones in the syllabus:
Feb 4: Project choice (0.5 page, simply listing the organization, its issue, your team)
March 4: Detailed Project description (2-3 pages): having interviewed at least 1
organization employee, what do you think is the underlying problem or opportunity for
this organization? Which of the course concepts and tools are relevant to it? What exactly
can you do in this semester to help the organization address it? How will you get data,
from interviews, from company sources, from the web?
April 22: Full report presentation deck due (15-30 slides)

Your final project consists of a presentation deck of 15-30 slides, which:

1) describes the company and industry environment and history
2) clearly states the question you want to answer (e.g. what is the sales effect of a
price promotion? Has TV or print advertising a higher ROI? Which metrics best
forecast performance in a market dashboard?)
3) make an original effort to address the question, eg by gathering opinions of
managers and/or customers, by analyzing data, ….

Moreover, you will present a 7-minute version of your project on the last day of the
course to your fellow students (subset of 5-10 slides with a focus on the issues and how
you dealt with them, not the company itself, which can remain anonymous). This project
will help you think about the course concepts and tools in a very concrete manner,
relevant to a business of your own interest. As I have to grade this project, I of course
guarantee full confidentiality regarding the company involved.

In case you choose to do this in a team, each team member is expected to contribute
fully to the project. You will evaluate each other's participation at the end of the
semester. In utopia, everyone puts in equal work. If this doesn’t work, you will have the
chance to let me know; both those who slack off and those who put in extra effort.

Which project to choose? First, you or a team member may already be interested in a
brand currently on the market. You could have access to the decision makers (eg through
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

a coop, a friend or family connection) or not, in which case you can combine publicly
available data with interviews (eg one team studied the Domino’s turnaround). Second,
you may want to propose the marketing plan for a new market offering, eg an
entrepreneurial venture from IDEA (https://www.northeastern.edu/idea/) Third, you can
get inspired by the guest speakers. Your final option is to recommend the best type of
content and/or social media posting plans to promote the book http://notsizedata.com/ and
linked blog ‘Smarter Marketing for Better Results’
https://analyticdashboards.wordpress.com/ - in which case I can guarantee you full access
to decision maker and performance metrics  . In any case, you will have to go beyond
what the brand is currently doing to propose a change in at least 1 part of their marketing.

Academic Standards and Academic Honesty: As members of a learning community,

students are expected to ensure that their conduct helps to create an atmosphere
conducive to learning and the interchange of knowledge, and are committed to academic
honesty. It is expected that students will behave in accordance with the Graduate Student
Honor Code. In particular, plagiarism and cheating will be penalized. Please do not
assume that your ideas in this course have to be your own. Collaboration in this course is
encouraged, except during exams or individual papers. The key to academic honesty is
that such collaboration requires acknowledgment. You are expected to read the Graduate
Student Honor Code: http://www.damore-mckim.northeastern.edu/academic-
programs/graduate-programs/current-students. Pages 40-42, in particular, apply to
students in this course. Please note that it states that one of the things that should not be
done is handing in the same paper [or significant parts thereof] or more than one course
without the explicit permission of the instructor.

What Can You Do?

 Understand that you are responsible for the quality of all work you submit
 Communicate with me if you do not understand class or assignment expectations
 Give credit to (cite) all sources you use. For specific citation info, see:
www.lib.neu.edu/online-research/help/citations-foodnotes

In your preparation for the cases, you should not go beyond the case in your quest for
information. The case provides all the company facts that should be used; and you are
requested to put yourself in the place of the decision maker which only had the
information provided in the case at the time s/he had to make a decision. Under no
circumstances should the company be called, knowledgeable sources interviewed, or
library/on-line searches conducted.

Resource Help

Please get in touch with me if you have any questions regarding assignments or cases.
You can also contact me just to talk. If you have problems (particularly with the written
and oral assignments), tutorial help as well as English Language help/editing is available
through the following resources:
 English Language Center- 406 Ell, x2455
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

 International Student Center- 403 Ell, x3451

 Disabilities Resource Center- 20 Dodge, x2675
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Course Outline
The course schedule follows the textbook’s organization and consists of 3 parts:
1. How to plan your marketing for business results? (Chapters 1-3, weeks 1-2)
2. How to measure offline and online marketing performance? (Chapters 4-10, wks 2-9)
3. How to use metrics to improve marketing performance? (Chapters 11-14, wks 10-14)

After the schedule, you will see the details for each session, including due assignments.

Winter Schedule MKTG6230

Class Date Topics and CASES Reading Due
1 Jan 21 Syllabus/Course Intro CH 1-2 Assignment 1
Aligning marketing metrics with strategy
2 Jan 28 Start with the Vision CH 3, ‘Long-Time Mednet.xls
CASE: Mednet.com Advertising ROI’ Project
choice
3 Feb 4 Measuring Return on Marketing CH 4,5, ‘Closing Gap Culinarian.xls
Investment CASE: Culinarian Cookware Marketing & Finance’ Project
choice
4 Feb 11 Building your database of KPIs CH 6,7 ‘Regression: A Assignment 2
review for managers’
5 Feb 18 Selecting leading KPIs CH 8, Assignment 3
CASE: KPIs for a Snack product ‘Mindset Metrics’
6 Feb 25 Marketing effectiveness and Attribution ‘Comcast Spotlight
CASE: Artea (A): Designing Targeting brings accountability’
7 March Attribution with Customer-level data ‘Algorithmic Bias in Project
4 CASE: Artea (B): Customer-level Data Marketing ’ Description
8 Mar 11 Cohort analysis and lifetime value CH 9 ‘The 6 metrics
CASE: eSig Conversion Funnel your boss cares about’
9 Mar 18 Marketing Performance Management in CH10, Winning
Emerging Markets Hearts, Minds, Sales
10 Mar 25 Design, Risk and what-if-analyses CH 11, “Unnecessary
CASE: Endless Crab Feast risk in your plan”
11 April 1 Acting on Performance Insights CH 12,13, “Theorem Assignment 4
CASE: Atrium Budget Allocation for Optimizing the
Advertising Budget
12 April 8 Time to work on project to get feedback Work on Final Project
13 April Culture of Accountability CH 14, Conclusion Final Project
15 PRESENTATION of your final project ‘From Metrics to Presentation
by half of the teams Action’ deck due
14 April 22 PRESENTATION of your final project Final Project
by the remaining teams due
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Part I: How to plan your marketing for business results

Session 1: Aligning marketing metrics with business strategy

The first step in any measurement system is to ensure your marketing objectives are
aligned with the overall business strategy of your company. This enables senior
management and the marketing department to track business results with a few key
performance indicators. Such systems, in companies as diverse as Amazon, Avaya,
Google, JP Morgan, Lenovo, Samsung and Vanguard, have been shown to increase firm
performance and allowing sufficient time for managers to react if business is not heading
in the right direction. We will review and practice tools developed by high performance
marketers across industries. Next, we will practice how to achieve such results in your
organization.

Readings : Text Book, introduction and chapters 1-2

Assignment 1: Take the ‘Free Organization Diagnosis’ Test at the end of Chapter 1
(online at http://notsizedata.com/diagnosis-is-your-organization-suffering-from/ )
and write half a page on how your organization could best use this course’s insights.
Email it to kpauwels@northeastern.edu BEFORE CLASS and have it ready during class.

Session 2: Start with the Vision and organize for marketing accountability

In this session, we discuss how to uncover the strategic vision for your organization and
explicitly link it to your marketing measurement system. The case
presentation and reaction will be our key into that discussion.

Readings: a) Text Book, chapter 3

b) "The Long-Term ROI of TV Advertising in a Digital World”, Raimund Wildner and
Guido Modenbach, Marketing Intelligence Review, 7 (1), 2015, pages 55-60.
c) https://analyticdashboards.wordpress.com/2018/07/14/do-you-measure-what-matters/

Case: Mednet.com confronts “Click-Through” Competition

In January 2007, Mednet.com is a five-year-old website that delivers scientific-based
health information free to online visitors. While the site operations conservatively in this
government-regulated market, its business model relies on advertising income, primarily
from pharmaceutical companies. Mednet competes for advertising dollars with large
search engines, category-specific sites and clinical trial sites. Large search engines charge
for “results” or “click-throughs”, while other sites, including Mednet.com, charge for
impressions. Faced with fierce competition for an important customer; Windham
Pharmaceuticals, Mednet is forced to defend key elements of its business model.
Addressing the following questions may help you offer such defense:
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

a) What specific customer behaviors determine whether or not a business model

produces the results that an advertiser wants? What are the best metrics for
measuring these?
b) What is the best argument Heather Yates can make to justify charging Windham
Pharmaceuticals for impressions instead of click-throughs? Does she have
acceptable alternatives?
c) What value to the site visitor does a general interest site (such as Marvel’s search
site) versus a niche site (such as Cholesterol.com) provide? Which is the more
defensible business model?

Each student fills out the spreadsheet ‘Mednetcalculation.xls’. provided by instructor

The selected presenters take the role of Mednet and prepare a 3-7 slide presentation to
convince the other students, which take the role of Windham Pharmaceuticals. Please
email sheets, presentations (kpauwels@northeastern.edu) before noon of the class day.

Part II: How to measure offline and online marketing performance?

Session 3: Measuring Return on Marketing Investment

Starting with the actual measurement challenges, we first discuss and use
economic/accounting metrics of Net Returns and Return on Marketing Investment. We
apply this to the ‘simplest’ puzzle in marketing performance measurement: the profit
impact of short-term price promotion with only short-term revenues and costs. This case
helps us to understand the challenges and issues with applying formulas from other areas
in marketing performance measurement.

Readings: Text Book, Chapters 4 and 5

“Measuring the ROI of Consumer Insight’ BCG 2018
"Closing the Gap Between Marketing and Finance: the Link to Driving Wise Marketing
Investment”, David Reibstein, Marketing Intelligence Review, 7 (1), 2015, 23-27.
Watch: Marketing ROI Mike Hanssens’ MASB video

Case: Culinarian Cookware: Pondering Price Promotion

In 2006, senior executives at Culinarian Cookware were debating the merits of price
promotions for the company's premium cookware products. The VP of Marketing,
Donald Janus, and Senior Sales Manager, Victoria Brown, had different views.

Discussion Questions:

1. Describe consumer behavior in the cookware market. How is cookware bought?

How is it sold? What are the implications for Culinarian’s marketing strategy?
2. What are Culinarian’s strengths and weaknesses?
3. Was the 2004 promotion profitable? Calculate the profitability using the consultant’s
logic and then calculate profitability using Brown’s logic. How would YOU calculate
profitability?
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

4. Should Culinarian run a 2007 price promotion? If so, what would be the specifics of
such a promotion (e.g. product scope, discount rate, timing, communication)? If not,
what other types of (sales) promotion would you propose (e.g. gift with purchase,
manufacturer rebate, sweepstakes, product placement, advertising,…)?

Each student calculates the promotion profitability by filling out the spreadsheet
‘culinarian.xls’, provided by instructor. The selected students also prepare a 3-7 slide
presentation to convince the other students. Selected students are free to choose the
role of Victoria Brown, Donald Janus, the time series consultants or another executive
at Culinarian – depending on which point of view your own opinion is closest to. Both
the spreadsheets and the presentations are due by email (kpauwels@northeastern.edu)
before noon of the class day.

Session 4: Building your database of potential Key Performance Indicators (KPIs)

Moving beyond measuring short-term profitability, we move to metrics that aim to

measure long-term performance. Next we build our database with such potential Key
Performance Indicators and look at summary statistics that indicate their usefulness in an
analytic dashboard. Privacy issues come into play for certain customer metrics.

Readings: Textbook, chapters 6 and 7

‘What is Correlation?’ https://towardsdatascience.com/what-is-correlation-975ea899aaed
A refresher on regression analysis: https://hbr.org/2015/11/a-refresher-on-regression-analysis
If you prefer video on regression: https://www.youtube.com/watch?v=k_OB1tWX9PM

Assignment 2: For your own business, or a chosen organization you have (had) contact
with (including Northeastern University ), generate at least 25 potential KPIs. Discuss
in 1 page whether they cover all major business components, measure both short-term
and long-term performance and represent key ways in which the business can succeed or
fail. Bonus points for going through the ‘Wheel of Discovery’ for 3 KPIs (see Chapter 7)
OR discussing privacy issues in collecting these metrics (see the cartoons in
https://marketoonist.com/2017/03/one-to-one-marketing.html)

Session 5: Selecting Leading Key Performance Indicators

After ensuring we have generated a comprehensive set of metrics, we now have to select
those that matter most and lead, instead of lag, actual performance. To this end, we dive
into regression analysis for practical definitions of causality and impact size.

Readings: Text Book, chapter 8

“Regression: A review for managers”, provided by instructor on Canvas
"Mindset Metrics: Consumer Attitudes and the Bottom Line”, Shuba Srinivasan,
Marketing Intelligence Review, 7 (1), 2015, 29-33.

Case: Key Performance Indicators for a snack (provided by instructor)

Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

You are hired as a consultant by a company marketing the leading national brand in a
snack category. While national brand competitors pose little threat to your client, it has
seen some of its lunch eaten by the rise of store brands. Your client gives you a weekly
dataset over the last 2 years (in the course folder as ‘keyperformanceindicators.xls’),
including 99 variables that may potentially explain sales performance of the national
brand (nb). It is your task to propose a shorter list of up to 10 variables that may be
included in a marketing dashboard for the company’s top management. Dashboard
variables will be monitored weekly, and should be leading indicators of national brand
sales, so that management can take action based on the dashboard information.
a) Which set of metrics would you track in the dashboard?
b) Which criteria and procedure did you use to arrive at your answer?
c) How will the dashboard help the client diagnose what is going wrong/well and
predict the performance effects of proposed marketing initiatives?

Each student answers the questions a-b in a 1-page assignment (go with your intuition
– spend at most half an hour on this assignment!). The selected students also prepare a
3-7 slide presentation to convince us of their proposed metrics. The presentations are
due by email (kpauwels@northeastern.edu) before noon of the class day.

Session 6: Marketing Effectiveness and Attribution

Now we know how to select leading KPIs, we turn to estimating the effectiveness of
marketing actions to drive them.

Watch short videos on AB tests: https://www.youtube.com/watch?v=zFMgpxG-chM

And True experiments: about 3 minutes until you are asked to register

Read: CASE: Artea (A): Designing Targeting Strategies

Staring at her customer dashboard, Alex Campbel, the cofounder and CEO of online
retailer Artea, was surprised to find that most website visitors never make a purchase –
despite wonderful engagement metrics and satisfaction with website and product. She is
contemplating offering coupons, and her team ran an AB test / experiment, offering half
of the site visitors a 20% off coupon. With the results in hand, Alex now has to decide on
whether to offer coupons and, if so, to whom.

Key case questions:

a) How does an AB test / experiment assess marketing effectiveness as compared to
e.g. a regression analysis based on past data on coupons and sales?
b) Are the coupons effective?
c) Which customers should be targeted and what is the predicted outcome?
d) What are the risks of using your proposed targeting strategy?

Session 7: Customer-level Data

Continuing with the Artea case, we now investigate whether obtaining customer-level
geographic data changes the calculation. Artea is approached by third party Trackify with
the offer to buy data of its predictions of customer-level demographics (such as gender
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

and ethnicity). How could such data be used and what are the ethical concerns from
buying and using the data?

Read: Algorthmic bias, HBS coursepack

CASE: Artea (B) Including Customer-Level Demographic Data

Key case questions:
a) Should Artea buy the demographic data from Trackify? For what purpose?

b) Are there any ethical concerns from buying and using customer-
level demographic data?

c) What do you learn from the demographics data? Do your

answers to Artea (A) change? How?

Due: Project Description (2-3 pages, email to me before class): having interviewed at
least 1 organization employee, what do you think is the underlying problem or
opportunity for this organization? Which of the course concepts and tools are relevant to
it? What exactly can you do in this semester to help the organization address it? How will
you get data, from interviews, from company sources, from the web?

Session 8: Customer Lifetime value and cohort analysis

Customers yield a stream of revenue, affected by their choices and the firm’s actions.
Cohort analysis allows us to investigate this without needing customer-level data. We
apply this to the typical freemium model found in startups today and analyze the
conversion funnel.

Readings: Text book, Chapter 9

MASB Video Dan McCarthy
"True Synergy for Real Effects’, Prasad Naik and Kay Peters, Marketing Intelligence
Review, 7 (1), 2015, pages 35-41.

Case: eSig Conversion Funnel Analysis

eSig web browser-based product allows users to electronically sign a PDF or Word
document for free with a freemium business model. Founded in 2013, the company
officially launched its product in February 2015 and started to scale up its marketing.
That year in November, eSig’s Director of Growth Wayne Dalton was analyzing the user
acquisition data to plan for Q1 2016. The key questions feeding into the plan are:
(a) How many users do you predict for eSig on March 31, 2016 as a baseline (i.e. if
eSig would stop all paid marketing starting December 2015)?
(b) How can you explain conversion rates in the main eSig paid marketing channels?
(c) How important is the ‘virality’ of a free eSig user?
(d) How much of the $1M budget should go to each of the paid marketing channels?
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Each student answers the questions a-d for themselves as preparation of class
discussion. The selected students also prepare a 3-7 slide presentation to convince the
other students and the instructor of their proposed metrics. The presentations are due
by email (kpauwels@northeastern.edu) before noon of the class day.

Session 9: Marketing Performance measurement in Emerging markets

Looking beyond the USA, we explore why marketing performance metrics would work
differently across the world and to what extent the measurement and reward systems
should be standardized. We are also learning how to read papers in academic journals,
with a focus on abstract and introduction, visuals and conclusions.

Readings: Textbook, chapter 10

https://analyticdashboards.wordpress.com/2014/03/20/how-should-you-adjust-your-marketing-across-cultures/
"Winning hearts, minds and sales: How marketing communication enters the purchase
process in emerging and mature markets." K. Pauwels, S. Erguncu and G. Yildirim
International Journal of Research in Marketing 30.1 (2013): 57-68.
“App popularity: where in the world are consumers most sensitive to price and ratings?”

Session 10: Dashboard design, assessing risks and performing ‘what-if’ analyses

Risk is the R-word in marketing-finance interaction: while it comes naturally to finance,

it is often foreign to marketers, who are supposed to display enthusiasm and confidence
in their plans. Ironically, the short tenures of CMOs indicate that managing marketing is,
in itself, risky business! Now we’ve got the right metrics measured in the right way and
established their dynamic relationships, it is time to design a dashboard that visualizes
key insights and allows uncertainty and what-if analyses.

Readings: Text Book, Ch 11, 12

“How much unnecessary risk is in your marketing plan?”, Marketing NPV, 1, 1, p. 4-8.

Optional Readings: “The ROI of Blogging”, Forrester Research Report

"True Synergy for Real Effects’, Prasad Naik and Kay Peters (Links to an external site.),
Marketing Intelligence Review, 7 (1), 2015, pages 35-41.

Case: Endless Crab Feast (provided by instructor)

In the ‘Endless Crab Feast’ case, a seafood restaurant is planning a major ‘All you can
Eat’ Crab promotion. As seasoned managers, we ask the tough questions:
a) What could possibly go wrong?
b) How would each answer to (a) impact the profit from this proposed initiative?
c) How can we manage risk? Which risk factors can we eliminate/investigate before
we run the campaign, and how can we manage the others if they materialize?
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Each student answers the questions a-c for themselves as preparation of class
discussion. The selected students also prepare a 3-7 slide presentation to convince the
other students and the instructor of their risk management proposal. The presentations
are due by email (kpauwels@northeastern.edu) before noon of the class day.

Part III: Using Metrics to improve Marketing Performance: Action for Results

Session 11: Allocating marketing for profits

Are you willing to change decisions based on the insights from your marketing
performance measurement? We discuss the different scenarios of your decision making
power (only budget allocation or also budget setting) and decision making style
(optimizing or satisficing).

Readings: Text book, chapter 13

“A New Theorem for Optimizing the Advertising Budget”, M. Wright, Journal of
Advertising Research, 49 (2), 164-169.
‘Brands are continuing to overinvest in content nobody sees’ AdAge, 2020

Case: Atrium: Marketing budget allocation: perception and facts (provided by instructor)
Atrium is a medium-sized manufacturer and marketer of a fast moving consumer good.
While Atrium’s market had not experienced an economic crisis, global issues had
deteriorated its financial situation, requiring the company to scrutinize its marketing
budgets. As the newly appointed Chief Marketing Office (CMO), Peter had been charged
with proposing the new advertising and promotion budgets for the coming November
2018-October 2019 fiscal year. In the past, managers had decided on marketing spending
by experience of what worked before. The new CEO made it clear this could no longer be
the case. She expects Peter to cut the overall marketing budget by one fifth. Peter knew
he had to come up with excellent fact-based arguments to justify any marketing budget
and allocation, especially one that did not satisfy the CEO’s expected cost savings.
“Price is basically dictated by market conditions, and average contribution margin
fluctuates marginally based on production efficiencies. Both should remain at similar
levels in the next fiscal year. My two decision variables are the advertising (above the
line) and the sales promotion (below-the line) budgets.” Peter turns to me with a simple
question: “what should I do? ”.

a) From the data in excel sheet ‘marketingperformancebudgetallocation.xls’, calculate

the monthly net contribution (in thousands of euros) and the return on investment.
What do you observe from plotting the data? What are the key drivers of sales &
profit?
b) Do you believe Atrium has been overspending or underspending on advertising and
on sales promotion? If so, by how much?
c) Which advertising and sales promotion budgets would you propose for the next year?
Prof Koen Pauwels Fall 2018 MKTG 6230 - Marketing Performance Measurement

Each student answers the questions a-c for themselves as preparation of class
discussion in separate teams. Make sure to answer question (a) without running any
model. Do run a model to answer (b) and use the Wright (2009) reading to answer (c).

Assignment 4: Digital natives Amazon and Google have been spending considerable
funds on advertising lately, and both external and internal debate have grown over
whether they are overspending. Using 2019 (and thus pre-pandemic) data, please evaluate
whether you can make statements to that effect, using these facts and assumptions:
1. Both companies have a yearly revenue growth of 20%
2. Both companies have an advertising-sales elasticity of 0.10
3. Amazon has a unit contribution margin of 40%, Google of 50%
4. In 2019, Amazon had revenues of $280.5 Billion , and Google of $162 Billion
5. In 2019, Amazon spent $11 Billion on advertising, and Google $18.46 Billion
The questions to answer:
a) What is the optimal 2019 spending for each company?
b) Are Amazon and Google overspending, underspending or about right?
c) How much more effective would the advertising have to be (versus the
benchmark assumption in (2) above) for the spending to be optimal?
Please write up in a page, and email me before class.

Session 12: Please use this session to make progress on your project. Class won’t meet.

Session 13-14: Nurture the culture and practice of accountability in your organization
Last but not least, we discuss how to (re) start marketing performance journey in your
organization. How to you get top management support and employee engagement? How
important is the existing organizational culture to marketing performance system success
and how will it be affected by your proposed system?

Reading: Text Book, Chapter 14 and conclusion

"From Metrics to Action: Interview with Nicholas Chesterton from Unilever”, Marketing
Intelligence Review, 7 (1), 2015, pages 48-53.

Assignment: Your presentation on your final project