JMP Discovery Summit 2014

SAS Headquarters, Cary, NC

Customized JMP Analytics:

Mondelz International Consumer
Test Evaluation Package

Jeff Stagg, Manager MEU Statistics

David Rose, SAS Professional Services
Tuesday, Sept. 16, 4:00 4:45 pm
 Workflow for presentation
This presentation will demonstrate live specific features of Mondelz Internationals
Multiproduct Consumer Test Evaluation (MCTE) package on a real case study
The application is too extensive to demonstrate fully, we hope that showing specific
features with some interpretation will make it more meaningful & impactful for you.

Order of play
MCTE Package Overview
Benefits
Functionality
Brief recap of 2012 Summit presentation
JMP Summit 2014 Regional consumer test
Case Study Description
Case Study Analysis
Variant selection using DoE platform
Consumer Drivers of Liking
Customized Report Generator

Q&A
2
 Benefits of MCTE Package
Double Efficiency
2-5 rather than 5-10 man days for analysis, interpretation and report generation.
Graphics can be customized within agreed report formats and exported into business
presentation.
Simpler, less error prone
Point & Click on JMP replaces use of several statistical software packages and self-
customisation of macros for case study specifics.
Extended user capability
Several active users per RD&Q site rather than reliance on a few company experts/
external agencies.
Local knowledge of project/process applied better to the interpretation.
Use of best practice analysis
Customized scripts use Mondelz Internationals best statistical practices for global
deployment experts focusing more on this.
Harmonization of Systems
Replacement of other more expensive analysis options.
Training efficiencies.

3
 The MCTE Package
Most of the analyses described in this presentation were performed in JMP using a bespoke
analysis toolkit (MCTE) that was written in a collaboration between Mondelz International
and SAS UK.

This application has featured twice in previous Summits (2010 & 2012). Since then its
functionality has been continually improved to take full advantage of JMP v11 upgrade and
broaden its capabilities to cover Mondelz Internationals best analysis practices.
MCTE application is now deployed in the form of a JMP Add-In to Mondelz International
RD&Q sites around the world, together with its own customized help file.

4
 The MCTE Package - Functionality
Main menu
Package functionality is comprehensive;
scripts are customised to Mondelz Internationals
requirements for data manipulation, best practice
analysis & reporting of consumer test studies

Sub menus:
Basic data manipulation Raw Data Analysis Summarised Data Analysis

Customized report generator

5
 The MCTE Package Graphics management
Graphics are edited via control panel; changes to colours, text, scale,
reported statistics & labels within graphics and size & layout of multiple
graphics are all possible

Multiple Comparisons by
Subgroup control panel

Sub menus:

6
 JMP Summit 2012 Presentation Overview
Business Objective
Launch a line extension chocolate
tablet product
Technical approach
Use new/different process
technology to give points of
difference
Design of Experiments
Consumer Test Design
Central location test (CLT)
153 consumers
12 products (DoE)
Each consumer evaluates all
the products
Iterative clustering 50 runs
Outcome
Current product [237] dominates cluster
1 but is rated worst and significant lower
than all other products in cluster 2
Prototype 815 (Fluffy_F_3x5_DC) was
launched to complement current product
in market; having all four points of
differentiation.

Firmness as main overall

point of difference

All Test persons - Overall Liking

Consensus clustering solution
Cluster 1 (59%) Cluster 2 (41%)

7
 JMP Summit 2014 Case Study Description

Business Objective
Identify a product design that satisfies both regional and within country consumer
acceptance; product selection weighted by country volume & margin.

Technical approach
Use new/different process technology to give points of difference from current
product
Select 6 products that have required functionality and have point of difference
from current product and from other.
Execute a consumer test to acquire consumer acceptability of products and use
JMP MCTE package to analyze
o Identify consumer drivers of liking from JAR ratings
o Recommend new product design for harmonization strategy across region.

8
JMP Summit 2014 Case Study Description

Consumer Test Design

Central location test (CLT)
3 countries
100 consumers per country; pre-recruited
6 products; same set tested in each country
Consumers evaluate all 6 products in one session
Consumers rate products for
o Overall & attribute liking questions (scale 1-9)
[1=Dislike Extremely, 9=Like Extremely]

o Just About Right (JAR) attribute intensity ratings

[1= Much too Little; 3= Just Right; 5= Much too Much]

Managed by external market research agency

9
JMP Summit 2014 Live Demonstration
What we will show today
Three new analysis procedures will be covered.
1. Candidate Selection a novel way to select a product set for consumer test; one
that has optimal coverage of target flavor space and includes required reference
products.
A technical, repeatable solution to an otherwise judgmental process.
2. Consumer Attribute Drivers of Liking a novel way to summarize traditional
penalty analysis results. Two new metrics of JAR ratings are introduced
a) Penalty Impact: Drop in liking when a product is
not scored Just About Right .
b) Penalty Balance: An estimate of the opportunity
to adjust a product attribute to improve its liking.
Facilitates simple product change discussions/ decisions with the project
stakeholders.
3. Customized Report Generator this new MCTE capability enables user
customization of statistical report contents & layout. Its introduction replaced
another reporting system on a key Mondelz International RD&Q site.
MCTE application now used on all main RD&Q sites (global deployment).

10
JMP Summit 2014 Product Selection using DoE Platform

Product Selection Objective

Select 6 products from the 19 possible that lie within the target profile space for
consumer testing these should have optimal coverage of this space and include
current product.

Technical approach
Conduct descriptive sensory analysis (QDA) on all products using an in-house
expert sensory panel (trained consumers).

Perform principal component analysis (PCA) to generate sensory factors from 42

sensory attributes.

Use main (3) sensory factors as covariates in DoE Custom Design to identify six
products including current that will best estimate sensory factor main effects

11
 JMP Summit 2014 Product Selection using DoE Platform
Product selection results
Free choice selection doesnt include current product.
Current product is forced into selection by entering References as a 4th covariate
This principle can be used to force a set of required products & get an optimal
selection to cover target space from the rest Forced entries.

Naming convention: P?_Flavor:Firmer:Coarser

5 levels = Low (L): Low-Med (l): Med (M): Med-High (h): High (H) 12
 JMP Summit 2014 Case Study Description
The Data
The consumer test data is read into JMP from an Excel File and stored for analysis by JMP
& MCTE package
MCTE

13
JMP Summit 2014 - Case Study Analysis Raw Data Analysis R2 Multiple Comparisons by Subgroup

RGT Results - Overall liking

Regional
P5 & P4 variants perform better than Current
product for all counties; and in Regional assessment,
softer is rated significantly higher.
Results shown are un-weighted for country volumes
& margins

Naming convention: P?_Flavor:Firmer:Coarser

Note : Products joined by bars are rated statistical parity
for range test applied [here Fishers LSD, = 0.05]

Country = AAA Country = BBB Country = CCC

14
JMP Summit 2014 - Case Study Analysis

RGT Results - Overall liking

When 2014 current weightings are applied to centered country liking scores,
P5 & P4 variants still perform better in regional assessment than Current
product; P5 most liked
=> P5 variant to validation test then launch as regionally more liked and
business efficiencies to harmonize.

When 2017 projected weightings are

applied P4 variant is now most liked
=> Investigate softer plus coarser variant design options in further testing

Note: These results are not generated by MCTE tool as weightings are applied to country centered
averages, derived from different consumer bases
Naming convention: P?_Flavor:Firmer:Coarser 15
JMP Summit 2014 - Case Study Analysis Raw Data Analysis RA JAR Scale tabulations

RGT Results Drivers of Liking (DoL)

Penalty Analysis of all samples for all countries combined generates regional
rank order of importance of JAR attributes

Most
important

Consumer
Attribute Drivers
of Liking

Least
Penalty Impact numbers are weighted for
important
2014 current country volume & margin

JAR frequency is the % of consumer scoring attribute Just About Right

Penalty Impact is how much on average overall liking is penalized for not being Just About Right

16
JMP Summit 2014 - Case Study Analysis Raw Data Analysis RA JAR Scale tabulations

RGT Results Drivers of Liking

Penalty Analysis across all countries combined generates regional insights for product
improvements
DoL
Most
important

Least Penalty Balance numbers are weighted for

important 2014 current country volume & margin

Penalty Balance is a measure that indicates the potential opportunity to adjust an attribute in
order to improve performance [less opportunity as PB get closer to zero]
All variant adjustments to improve performance are to correct over-delivery on attributes.
Softer variant has little insight on how to improve it, whereas all other variant require texture
adjustment [make softer]; higher flavor variants over- deliver on taste & sourness.

Naming convention: P?_Flavor:Firmer:Coarser 17

JMP Summit 2014 - Case Study Analysis Customized Report Generator T1 Interactive Data Labeller

RGT Results Report Generator setup

Consumer attribute ratings are

assigned roles according to how
they can be analysed.
Optionally full questions from
consumer test questionnaire
can be imported and assigned
to JMP variables (column
headers) . Reporting can then
either use shorter or long form.

18
JMP Summit 2014 - Case Study Analysis Customized Report Generator T2 Interactive Data Builder

RGT Results Report Generator output default format

Here, default statistics are reported for a set of liking attributes then for a set of
JAR scales
Naming convention: P?_Flavor:Firmer:Coarser 19
JMP Summit 2014 - Case Study Analysis Customized Report Generator T2 Interactive Data Builder

Report Generator output flexible format

Products & attributes can be

selected as required.
The order of products & attributes
entered is the order in which they
are reported.

Grouping variables can be

specified to report data by group
level

Default statistics are set for each

variable role; these can be
changed for customized reporting

Switch between labels and full

questions

Reports can be in horizontal or

vertical layout

20
JMP Summit 2014 - Case Study Analysis Customized Report Generator T2 Interactive Data Builder

Report Generator output a flexible format

Here, report has specified order of products & attributes; showing selected
statistics for country AAA only
Naming convention: P?_Flavor:Firmer:Coarser 21
 JMP Discovery Summit 2014
SAS Headquarters, Cary, NC

Customized JMP Analytics:

Mondelz International Consumer Test
Evaluation Package

Q&A

Jeff Stagg, Manager MEU Statistics

David Rose, SAS Professional Services

22
Consumer Test Evaluation Using a Combination of JMP Functionality Plus
Customized JSL Scripts Ticks All Efficiency and Effectiveness Boxes for Mondelz
International RD&Q

Jeff Stagg, MEU Statistics Manager, Mondelz International

David Rose, PhD, Analytics Consultant, SAS Marlow UK

Topic: JSL Application Development Level: 3

Mondelz International empowers its staff by developing user-friendly tools for customized data handling,
calculation and reporting. JMP software houses a multiproduct consumer test evaluation (MCTE) tool used by
the global Consumer Science function. Previously used for single-consumer test evaluation, the tools recent
enhancements in functionality enable easier regional (multi-country) assessment and flexible format report
generation. Approaches to product selection using standard JMP functionality and regional analysis and
reporting using Mondelz Internationals MCTE tool will be demonstrated. More specifically, customized JMP
scripts evaluate Consumer JAR scales that summarize attribute importance and product change opportunity.
JAR scales measure levels of a product's attribute relative to a respondent's theoretical ideal level. These scales
have an anchored midpoint of "just-about-right and endpoints anchored to represent intensity levels of the
attributes that are higher and lower than ideal. A user-friendly report generator is another recent
development. While innovative in design to meet diverse user requirements, its success was more dependent
on its capability to produce structured Microsoft Word reports. The MCTE tool has significantly reduced
training and analysis time and ensured use of good statistical practices. Its success results from a close
collaboration between Mondelz International UK and SAS Professional Services in Marlow.

23