REPORT

OPTIMIZATION
AUTOMATION

24.D6 Villa, Saigon Pearl Villas, 92 Nguyen Huu Canh Str., Ward 22, Binh Thanh Distr., HCM City,
Vietnam
www.pmax.com.vn | support@pmax.vn | (+84) 936-498-995 - (+84) 973-474-528
OPTIMIZATION

01 Optimization Process
02 Testing Framework
03 Modeling/Algorithm
01 OPTIMIZATION PROCESS
1. WHAT IS OPTIMIZATION?

Optimization is the ongoing, data driven process of continually discovering

and quantifying the most effective experience, both online and offline, for
your consumers and customers.

2. WHY OPTIMIZATION?
01 OPTIMIZATION PROCESS
3. OPTIMIZATION PROCESS

PREPARE
● Finalize testing plan
● Define KPIs
● Define algorithms
REVISE
04 ● Reallocate budget
according to actual insight
● Update algorithms
● Update testing plan

01 03
RUN
ANALYSE
● Execute plan
● Take action by algorithms ● Breakdown performance
● Keep 80-20 ratio
02 by different dimensions
● Consolidate insight
02 TESTING FRAMEWORK

Lay a ground work

Understand what attempt to optimize Key driven for KPIs

Establish a baseline

Calculate where we are starting from

Form testable hypothesis

Identify biggest barriers to your goals to design potential test

Design your testing roadmap

Priority task list Testing strategy ( A/B testing, multivariate testing..)

Reporting and Assessment

Success test: apply and re-test others metrics Failed Test: re-examine data, design a new test
TESTING IN DIGITAL MARKETING

Testing allows us to (1) make careful changes before the mass launching (2)
construct hypotheses & learn better why (3) improve performance all the time ...

1. Conditions
2. Methods
3. Elements
Testing conditions

● Design: Control, Variable, Variation

● Sample size: http://www.evanmiller.org/ab-testing/sample-size.html#!2;90;5;10;0
● Statistical significance:
● Experiment time:
Testing method

A/B Testing

Winner!
A/B Testing is a method of comparing 1 or several variations (of an element) against each other and
against the control to determine the winner.
● A true test is different from a time-series analysis
● Testing and Analysis go hand in hand to achieve the best outcomes

TESTING
Which layout works better?

ANALYSIS
What content do people look at more?
Testing method

Multivariate Testing

Winner!
 Testing method

Comparison

• Easy to do experiment • Can't see resonant impact

• Small visit sample between variables
A/B test • Less time consuming process • Take time when we have to
• Easy to analyze test a lot of variables
• Suitable for all websites

• Comprehensive test • Complicated

Multivariate Test • See resonant impact when • Big visit sample
combine multi variables • Time consuming process
• Save time • For high traffic websites
Testing Process GOALS

RESULTS HYPOTHESIS

EXPERIMENT VARIATIONS
02 TESTING FRAMEWORK

WHAT TO TEST?
EVERYTHING as long as you can ensure the conditions

OPTIMIZATION MINDSET

80% 20%
Keep in mind…

• Optimization not as an end goal but as an ongoing process.

• Because the way we do business is always evolving and customers’ needs change over time, you will
never reach the point where you’ve run “enough” tests.
02 TESTING FRAMEWORK
Know the math ☺
What does someone running an A/B or MVT test need to know about math? How detailed should they be?

Statistical Confidence = confidence in a repeated result

The confidence level, or statistical significance indicate how likely it is that a test experience's success
was not due to chance. If you ran this test again, it is likely you would see similar results
Confidence interval = a range within the true value that can be found at a given confidence level

What impacts confidence interval?

Sample size - as a sample grows the interval will shrink or narrow
Standard deviation or consistency - similar performance over time reduces standard deviation
03 MODELING & ALGORITHM

1. WHAT IS ALGORITHM?
Algorithm is a set of rules to take actions to optimize for final KPIs.
03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)

Budget allocation model

• Define baseline (benchmark)
• Define and predict correlation btw bidding & KPIs for each channels
➢ Finding a rule to define optimal bid for each campaign
03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)

Budget allocation model

• Define KPIs

• Define and predict correlation btw

spending & KPIs for each channels
➢ Calculate optimal spending for each
channel
03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)
Example: How optimization flow work through campaign implementation process

Campaign preparation
Campaign Set-up:
Product Structure, Naming
selection convention, Algorithm to manage
Targeting, Factors campaign

List of product Campaign execution

1/ Rule based for
selecting product
and creative Auto Create
template Ads

Campaign
Aired

Ad performance
3/ Campaign synthesis:
Deactivate Non-go
Factors
Bad ad 2/ Rule for Good ad Adjust weight allocate
Performance ON/OFF Ads Performance budget in each combo
03 MODELLING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)

Do you think you can solve it?

Facebook Paid search Total

Metric Spend CPO Spend CPO Order
Baseline 1,000 10 1,500 13 215
Case 1 1,000 10 2,000 14 243
Case 2 1,500 12 1,500 13 240

You are spending 2,500 usd every day as above. If you have 500 usd to spend more, which
case you will prefer?
03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)

Do you think you can solve it?

Facebook Paid search Total
Metric Spend CPO Spend CPO Order
Baseline 1,000 10 1,500 13 215
Case 1 1,000 10 2,000 14 243
Case 2 1,500 12 1,500 13 240

Facebook Paid search Total

Metric Spend CPO Incre-CPO Spend CPO Incre-CPO Order
Baseline 1,000 10 1,500 13 215
Case 1 1,000 10 2,000 14 18 243
Case 2 1,500 12 20 1,500 13 240
03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)

Do you think you can solve it again?

Max CPM
Max imp CPC CTR CR Max imp eCPC CPO
Facebook 3,000,000 3,000 1.0% 1.0% 3,000,000 3,000 300,000
ANTS 2,500,000 2,000 0.4% 0.8% 2,500,000 2,500 312,500
GDN 3,750,000 1,500 0.3% 0.4% 3,750,000 1,000 250,000

1. You have 50M vnd, suggest budget allocation to maximize #order?

03 MODELING & ALGORITHM
Building algorithm & model (Budget allocation & bid adjustment)
• Can you solve it again?

CPC
Max cost CPC CTR CR CPO
Facebook 3,000 1.0% 1.0% 300,000
ANTS 2,000 0.4% 0.8% 250,000
GDN 1,500 0.3% 0.4% 375,000

Your answer is…?

03 MODELING & ALGORITHM

?
? Metrics ? Data

Actions Benchmarking Algorithm

Automation
Algorithm

Building algorithm Take action base on algorithm

03 MODELING & ALGORITHM

Final KPIs

Lookback
Benchmark
window

Actions
03 MODELING & ALGORITHM

1. You are agency who is in charge of Project A on Facebook.

Model CPL = 30.000 vnd
What is the rule you set for this project?
OPTIMIZATION - SUMMARY

Optimization is an ongoing process

It doesn't happen overnight… but can deliver big win
Optimization needs 3 elements:
• KPIs
• People & process
• Tools
Optimization includes 2 things:
• Testing (reason)
• Algorithm & Modeling (action)
REPORT

Description & scope

❑ Report is to update performance of the

project to the management or client (what
we did & results, plan for next steps)
❑ Time-based (Monthly/Weekly/Daily) or
Campaign-based
❑ Professional report can:
○ Ensure the transparency
○ Build trust to management or client
 Agenda of a report

Mandatory

● Overview performance
● Deepdive & Insights
● Key learnings & Next steps

Optional

● Open discussion
 Overview performance

● Metrics: Scale & Efficiency

● Compare with: Last week/month & Target
● High-level breakdown
● Comment:
○ We are on track or not (vs target) in
term of scale & efficiency, we are
improving or not (vs last week/month) in
term of scale & efficiency.
○ What we did last week, link with the
result
Deepdive & Insights
● Breakdown & comment
○ By product/service
○ By channel
○ By audience
○ By content/banner
○ By keyword
○ By date/time
● Best practices
● Demo

* Note: Comment needs to take into account both metric types: scale and
efficiency. Some are good for scale but not good for efficiency and vice versa
 Key learning & Next step

● Key learnings - Summary from

deepdive/insights
● Next actions
○ Next action with PICs and timeline
○ Budget allocation/ Updated media
plan
 Best practices & Common mistakes
Best practices:
● Lead title on each slide
● Synthesize instead of Summarize
● Synthesis must be backed up by data
● Write in bullet points, short, concise
● Be visualized, best is to use chart, not excel
table

Common mistakes:
● Write in essay-style. Many sentences in a
bullet points
● Numbers and chart show 1 thing, synthesis
mentions sth else completely
Synthesize vs. Summarize

“Syn” = Together
Learning + Thinking
Video
Synthesize vs. Summarize
Synthesize vs. Summarize
Lookalike Engagement had the lowest CiR, while Engagement performed
the worst with no order
Key highlight:
- Lookalike Engagement & Lookalike Purchase had the lowest
CiR
- Direct, Engagement and VIP did not bring any order. But the
install volume was stil low for Direct and VIP
Next actions:
- Stop Engagement
- Keep Lookalike Purchase, Direct and VIP
- Lower volume for Lookalike VIP
- Push for Lookalike engagement
- Refresh creatives (including KV and content style) to get higher volume
Synthesize vs. Summarize
Most people who are interested in Samsung ABC are male aged 18-34, interested in Media &
Entertainment, Technology, and Lifestyles & Hobbies.
STRUCTURE

Overview

❑ Shows overall results comparing with plan and last report.

SLIDE
/
STRUCTURE
TEXT SLIDE
Basic elements ❑ Breakdown performance by time, channels, product, audience…
Deep dive ❑ Explain data, show comments and insights.
• We are on track or not (with target) in term of scale and efficiency.
• We are improving or not in term of scale and efficiency.
Time breakdown
• What we did last time, link with the results.
TITLETITLE TITLE

SLIDE
/
STRUCTURE

Deep dive
Audience breakdown by gender

SLIDE
/
STRUCTURE

Deep dive
Channel breakdown

SLIDE
/
STRUCTURE

Deep dive Other breakdown base on solution.

Audience breakdown by age

SLIDE
/
STRUCTURE

Next steps
Budget & KPIs
❑ Show budget & KPI for upcoming plan.
❑ Can be breakdown by Time, Channels, Audiences… (based on client's request)

SLIDE
/
STRUCTURE

Next steps
Action plan ❑ What we have to prepare for next campaigns
❑ What we will do on next campaigns

SLIDE
/
REPORT STRUCTURE & EXAMPLE

Excel:
https://docs.google.com/spreadsheets/d/1I1mx_77Y4dnG7ew3AsyXSjWAHv
pduM5j2_eHlB8f-0E/edit#gid=1203389703

Studio:
https://datastudio.google.com/u/0/org//reporting/0B7KfwNuov4l7UEI4bHRoZ
2ZFRE0/page/nQN

45
AUTOMATION IN
REPORTING &
OPTIMIZATION
PROBLEMS

1. How to set-up the campaigns with

thousands of adgroup/ ads across
many channels quickly & smartly?

2. How to extract, consolidate data

and generate report
automatically?

3. How to optimize ads all the time

while I have only 8 hours to work?
1. MASS-UPLOAD - Facebook

What is Facebook Mass Upload?

HOW TO SET UP OR EDIT 100 ADS WITHIN 30 MINUTES ?

What can Facebook Mass Upload do?

20% - 30%
optimize smoothy

Increase 15% - 20%

work efficiency

50% - 60%
Save Time
1. MASS-UPLOAD - Facebook

What is process of Facebook Mass Upload by SpreadSheet?

Export Campaign Edit in Excel file Import Campaign Customize & check
1. MASS-UPLOAD - Facebook

Set up 40 ads with different Set up 50 ads with different

tracking link just into tracking link just into
15 MINUTES by Facebook 20 MINUTES on Platform
SpreadSheet
1. MASS-UPLOAD - Google

What do you need when we use Facebook Mass Upload?

● Know well how to use Ads Manager

● Template for mass upload
● Materials creative: content, banner, web Link
● Understand about objectives of Facebook campaign
● Ad formats of Facebook campaign

What do you need to know about Facebook Mass Upload?

● Campaigns have different objective can not use the same template when Export & Import.
● It depends on the actual situation to decide we will edit on the system or use import & export
1. MASS-UPLOAD - Google

What is Mass Upload in Adwords?

Using bulk uploads is an efficient way to make changes in your AdWords account by spreadsheets or quickly create a
new campaign by Excel file.
What can Google Mass Upload do?

20% - 30% 50% - 60% 20% - 30%

Workload Save Time Accuracy
1. MASS-UPLOAD - Google

What is process of Mass Upload by Spreadsheet?

Download Spreedsheet Edit Data offline Upload to AdWords

CLICK HERE
1. MASS-UPLOAD - Google

What is process of Mass Upload by Excel?

Import to Google Export to Google

Create Excel Template Edit Data Offline
Adwords Editor Adwords

CLICK HERE
1. MASS-UPLOAD - Google

Edit and Air New Campaign Set up Campaign 3 times

just into just into
30 MINUTES 90 MINUTES
By Google Adwords Editor By Google Mass Upload
2. DATA ANALYSIS & REPORTING

STEP DETAIL

1 Extract ads data from each

platform
(Adwords, Facebook Ads,
Adnetwork, …)

?
2 Extract data from tracking system
(Google Analytics, Webtrekk,
Internal system..)
3 Consolidate data

4 Analyze data

5 Report & Visualize data

6 Next action plan

2. DATA ANALYSIS & REPORTING

STEP DETAIL SOLUTION

1 Extract ads data from each
platform Use spreadsheet + add-on (google
(Adwords, Facebook Ads, analytics, supermetrics, blockspring,
Adnetwork, …) google studio…)

=>
2 Extract data from tracking system
(Google Analytics, Webtrekk, Use 3rd party platform: Acquisio,
Internal system..) adstage, marinsoftware, optmyzr …
3 Consolidate data

4 Analyze data Build internal system by connecting

5 Report & Visualize data API with ads platform

6 Next action plan

2. DATA ANALYSIS & REPORTING
Step 1: Export data
2. DATA ANALYSIS & REPORTING
Step 2: Create template & link data
2. DATA ANALYSIS & REPORTING
Step 3: Visualize data
2. DATA ANALYSIS & REPORTING

3rd Party platform report

 Adwords
3. OPTIMIZATION

Other Options for

Level Detail Action
Action
Campaign Pause
Enable
Change budget Set new budget
Increase budget (%)
Decrease budget (%)
Send email

Adgroup Pause
Enable

Change Adgroup bid Set new budget

Increase budget (%)
Decrease budget (%)
Send email

Ad Pasue
Enable
3. OPTIMIZATION

Adwords - Scripts
 Facebook ads
3. OPTIMIZATION

Other Options
Level Detail Action
for Action

Campaign Turn On

Adset Turn Off

Ad Adjust Budget Increase daily budget

Decrease daily budget

Increase lifetime budget

Decrease lifetime budget

Adjust manual bid Increase bid by

Decrease bid by
3. OPTIMIZATION

3rd party platform

REMEMBER

1. Automation
- Decrease the manual workload
- Increase the efficiency

2. Automation doesn’t mean “No

need people to involve”
• Q&A
 THANK YOU
FOR WATCHING

24.D6 Villa, Saigon Pearl Villas, 92 Nguyen Huu Canh Str., Ward 22, Binh Thanh Distr., HCM City, Vietnam

www.pmax.com.vn | isupport@pmax.vn | (+84) 936-498-995 - (+84) 973-474-528