Professional Documents
Culture Documents
Stamatiadis Informa Jan 2013
Stamatiadis Informa Jan 2013
Stamatiadis Informa Jan 2013
Contents
A bit of philosophy About performance management How to choose your KPIs How to use your KPIs Building the right scorecard
We are what we repeatedly do. Excellence then, is not an act, but a habit
Aristotle 384-322 BC
In order to keep things simple we have to schematize like in geometry When we do that we loose some of the detailed information (a real circle is never a perfect circle) And, like in geometry, we sometimes end up with a result that is irrational But in the end, simple representations work pretty well, provided that we have agreed on their meaning
KPIs Informa 2013 5
Contents
A bit of philosophy About performance management How to choose your KPIs How to use your KPIs Building the right scorecard
Study physician
Biostatistics
CRA
Drug safety
Contracts
CTL
Supplies
Medical writing
n = { z2 * 2 * [ N / (N 1) ] } / { ME2 + [ z2 * 2 / (N 1) ] }
I bet he is spying on me
Strategy defines the long term goals of an organization and the steps to achieve it. It is the map of the journey Before starting on a journey it is of course important to decide which parameters to monitor Just like in a clinical trial, we formulate a hypothesis, we define the data-points and we write a SAP The general framework for KPIs is given by the strategic objectives that detail the goals of the strategy The individual KPIs (metrics) must be used in accordance to the strategic plan
10
Contents
A bit of philosophy About performance management How to choose your KPIs How to use your KPIs Building the right scorecard
Courtesy Greg Nelson: Implementing Metrics Management for Improving Clinical Trials Performance
11
Occam's razor
among
competing hypotheses, the one that makes the fewest assumptions should be selected
12
Value Creation
rarely suffer due to having too few measures (Kaplan and Norton, creators of the BSC, 1996)
14
Define your strategy Break down the strategy into clear objectives grouped in four categories: People, Processes, Customers and Financials Identify the stakeholders of these objectives Decide together with the stakeholders what is the most representative variable(s) for each of the strategic objectives and rationalize the calculation rules Determine the right source of the data Define the reporting mechanism(s) (scorecard, dashboard, real time reports) Collect the data and validate the results (people must believe in the data) Continuously evaluate the metrics (get feedback, study the utility of your measurement results)
15
Do not index bonuses on KPIs Do not measure retrospectively. Only prospective evaluations have meaning and value
Do not select a KPI unless you have defined the expected target result and variation margins, how you will process them and what actions you will take depending on the values obtained (like an SAP)
16
Ownership
KPIs are owned by those who do the work. They must choose them, define them, report on them. PM only coordinates the process KPIs must be clearly defined and documented before the work starts Describe scope, units, timing, source data, person in charge Define when and how these can be revised Set targets according to the strategic plan. Long term should drive short term
KPIs Informa 2013 17
Clear definition
Site staff performance Programming productivity Enrollment quotas EDC performance Patients per CRA Resources
KPIs Informa 2013 18
One example
Protocol
Is the protocol signed-off in a standard way or can the signature be deferred until the last minute? How is study start measured? FPFV? A better measure would be 80% of the sites ready to recruit and FPFV How important is this measure? What are the thresholds and the actions in case of under-performance?
KPIs Informa 2013 19
Contents
A bit of philosophy About performance management How to choose your KPIs How to use your KPIs Building the right scorecard
20
21
Dashboard or Scorecard?
A dashboard displays actual values for a number of selected indicators A scorecard is an organized plan that follows the strategy, displays strategic objectives, measures and targets A Balanced Scorecard breaks down the strategy in weighted key categories and links strategic objectives to a strategy map and ultimately to a destination statement A BSC can be cascaded to the desired depth in the organization and aligned to individual objectives A BSC is a great tool for communicating to the organization
22
Use of a common tool keeps everyone aligned with common goals and vision Precise metrics allow all to better define their objectives The weighting of each measure helps you balance your priorities Long-term, robust measures allow you to track your progress towards your strategic goals Continuous monitoring of results allows you to evaluate whether your strategic initiatives are having the desired impact The BSC serves as a guide for your actions in your daily work
KPIs Informa 2013 23
Employee Engagement Level Quality of Performance and Development Activity Voluntary Turnover of Employees Rated AA/AAA
L2
Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
P2 P3
150 140
PEOPLE Become Best Place to Work
L1 Ensure Employee Engagement
Get rapid approval for the best labels Ensure payers and providers reimbursement
C1.1 C2.1
C3.1
Measure L1.1 L1.2 L2.1 C3 Engagement Bring superior treatments to more patients Employee Level
L2
Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
C3.2 Quality of Performance and Development Activity FINANCIAL Grow Shareholder Value Voluntary Turnover of Employees Rated AA/AAA F1 Grow sales and revenue F1.1 Measure F2 Increase return on sales (ROS) F2.1 Number of new preclinical and clinical projects: DP0s and DP1s (N) F3 Contribute free cash flow F3.1 Project timelines vs plan to PoC/Launch (months)
Measure Target Max Weight P1.1 Number of new preclinical and clinical projects: DP0s and DP1s (N) Develop a balanced pipeline of innovative P1 Employee Engagement Level products 80% 84% 88% 5% P1.2 Project timelines vs plan to PoC/Launch (months) Quality of Performance and Development Activity 72% 76% 80% 5% P2 Supply our products efficiently to the markets P2.1 Actual vs potential sales based on stock-outs (%) Voluntary Turnover of Employees Rated AA/AAA 6% 3% 0% 10% P3 Comply with all current standards & regulations P3.1 Major CAPAs/issues resolved on time (%) Measure Min Target Max Weight CUSTOMERS Address Customer Needs Measure Number of new preclinical and clinical projects: DP0s and DP1s (N) 6 8 10 10 % C1 Get rapid approval for the best labels C1.1 Approvals on time within target label (%) Project timelines vs plan to PoC/Launch (months) 4 2 0 5% C2 Ensure payers and providers reimbursement C2.1 Reimbursements on time at target price (%) Actual vs potential sales based on stock-outs (%) 97.5% 99% 99.5% 5% C3.1 Unit market share of Rebif, 1Q in arrears (%) C3 Bring superior treatments to more patients 80% Major CAPAs/issues resolved on time (%) 90% 100% 5% C3.2 Market share of Erbitux in 1st line mCRC (%) Measure Min Target Max Weight FINANCIAL Grow Shareholder Value Measure Approvals on time within target label (%) 60% 80% 100% 7.5% F1 Grow sales and revenue F1.1 Annual growth in total revenues (%) Reimbursements on time at target price (%) 60% 80% 100% 7.5% F2 Increase return on sales (ROS) F2.1 Return on sales (ROS) (%) Min Target Max Weight Unit market share of Rebif, 1Q in arrears (%) 20.1% 22.4% 23.7% 5% F3 Contribute free cash flow F3.1 Free cash flow vs total revenues (%) 80% 84% 88% 5% Market share of Erbitux in 1st line mCRC (%) 15% 17% 19% 5% 72% 76% 80% 5% Measure Min Target Max Weight 6% 3% 0% 10% Annual growth in total revenues (%) 2.5% 4.5% 6.5% 10% Min Target Max Weight Return on sales (ROS) (%) 11.5% 12.5% 13.5% 10% 6 8 10 10 % Free cash flow vs total revenues (%) 24.8% 26.8% 28.8% 10% 4 2 0 5%
60% 60%
20.1% 15%
80% 80%
22.4% 17%
100% 100%
23.7% 19%
7.5% 7.5%
5% 5%
Min
2.5% 11.5%
Target
4.5% 12.5%
Max
6.5% 13.5%
Weight
10% 10%
24.8%
26.8%
28.8%
10%
P2 P3
Actual vs potential sales based on stock-outs (%) Major CAPAs/issues resolved on time (%)
Measure
97.5% 80%
Min
C1 C2 C3
Get rapid approval for the best labels Ensure payers and providers reimbursement Bring superior treatments to more patients
C1.1 C2.1
C3.1 C3.2
Approvals on time within target label (%) Reimbursements on time at target price (%)
Unit market share of Rebif, 1Q in arrears (%) Market share of Erbitux in 1st line mCRC (%)
60% 60%
20.1% 15%
Measure
F1.1 F2.1 Annual growth in total revenues (%) Min Target Max Weight Return on sales (ROS) (%) 80% 84% 88% 5% Free cash flow vs total revenues (%) 72% 76% 80% 5%
Min
F1
2.5%
F2 Increase return on sales (ROS) Employee Engagement Level F3 Contribute free cash flow Quality of Performance and Development Activity
11.5%
F3.1
24.8%
L2
10% Weight 10 % 5% 5% 5%
Weight
Strategic Plan
99% 90% 99.5% 100%
Max
5% 5%
Target
Weight
80% 80%
100% 100%
7.5% 7.5%
5% 5%
22.4% 17%
23.7% 19%
Target
4.5%
Max
Weight
10% 10%
6.5%
12.5%
13.5%
26.8%
28.8%
10%
P1.1
Number of new preclinical and clinical projects: DP0s and DP1s (N)
P1.2Min Project timelines vs plan to PoC/Launch (months) Target Max Weight P2.180%Actual vs potential 88% sales based5% on stock-outs (%) 84% P3.172%Major CAPAs/issues resolved on time (%) 76% 80% 5% 6% Measure 3% 0% 10%
Supply our products efficiently to the markets Employee P2 Engagement Level P3 Comply with all current Activity standards & regulations Quality of Performance and Development
CUSTOMERS Address Customer Needs Voluntary Turnover of Employees Rated AA/AAA
L2
130
PEOPLE Become Best Place to Work
L1 Ensure Employee Engagement
C1.1Min Approvals on time within label (%) Target Max target Weight C2.1 6 Reimbursements on time at target price (%) 8 10 10 %
C3.1 4 Unit market Rebif, 1Q in arrears (%) 2 share of 0 5% C3.2 Market99% share of Erbitux 97.5% 99.5%in 1st line 5%mCRC (%)
60% 60%
20.1% 15%
80% 80%
22.4% 17%
100% 100%
23.7% 19%
7.5% 7.5%
5% 5%
Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
Ensure payers andprojects: providers reimbursement Number of C2 new preclinical and clinical DP0s and DP1s (N) Project timelines vs plan to PoC/Launch (months) C3 Bring superior treatments to more patients Actual vs potential sales based on stock-outs (%) FINANCIAL Grow Shareholder Major CAPAs/issues resolved on time (%) Value
Measure F1 Grow sales and revenue
P2 P3
80%Measure 90%
100%
5%
Min
2.5% 11.5%
Target
4.5% 12.5%
Max
6.5% 13.5%
Weight
10% 10%
F1.1Min Annual growth in total revenues (%) Target Max Weight F2.160%Return80% on sales (ROS) (%) 100%
Get rapid approval for the best labels Ensure payers and providers reimbursement Bring superior treatments to more patients
C1.1 C2.1
C3.1
return on sales (ROS) Approvals F2 on time Increase within target label (%)
7.5%
L2
FINANCIAL Grow Shareholder Value Employee Engagement Level F1 Grow sales and revenue Quality of Performance and Development Activity F2 Increase return on sales (ROS) Voluntary Turnover of Employees Rated AA/AAA F3 Contribute free cash flow Measure Number of new preclinical and clinical projects: DP0s and DP1s (N) Project timelines vs plan to PoC/Launch (months) Actual vs potential sales based on stock-outs (%) Major CAPAs/issues resolved on time (%)
Measure Min Target Max Weight Approvals on time within target label (%) 80% 84% 88% 5% Reimbursements on time at target price (%) 72% 76% 80% 5% Unit market share of Rebif, 1Q in arrears (%) 6% 3% 0% 10% Market share of Erbitux in 1st line mCRC (%) Min Target Max Weight Measure 6 8 10 10 % Annual growth in total revenues (%) 4 2 0 5% Return on sales (ROS) (%) 97.5% 99% 99.5% 5% Free cash flow vs total revenues (%) 80% 90% 100% 5% Min Target Max Weight
C3.2 Market share of Erbitux in 1st line mCRC (%) Min Target Max Weight Measure 84% 88% 5% Annual growth in total revenues (%) 76% 80% 5% Return on sales (ROS) (%) 3% 0% vs total10% Free cash flow revenues (%)
Min
2.5% 11.5%
Target
4.5% 12.5%
Max
6.5% 13.5%
Weight
10% 10%
24.8%
26.8%
28.8%
10%
Weight 10 % 5% 5% 5%
Weight
Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
Actual Performance
24.8% 26.8% 28.8% 10%
C1 Get rapid approval for the best labels Employee Engagement Level C2 Ensure payers and providers reimbursement Quality of Performance and Development Activity
C1.1 C2.1
C3.1
60% 60%
20.1% 15%
80% 80%
22.4% 17%
100% 100%
23.7% 19%
7.5% 7.5%
5% 5%
L2
Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
P2 P3
C3 Turnover Bring superior treatments to AA/AAA more patients Voluntary of Employees Rated C3.2 Measure FINANCIAL Grow Shareholder Value Number of new preclinical and clinical projects: DP0s and DP1s (N) F1 Grow sales and revenue F1.1 Project timelines vs plan to PoC/Launch (months) F2 Increase return on sales (ROS) F2.1 Actual vs potential sales based on stock-outs (%) F3 Contribute free cash flow F3.1 Major CAPAs/issues resolved on time (%)
Measure
Min
2.5% 11.5%
Target
4.5% 12.5%
Max
6.5% 13.5%
Weight
10% 10%
24.8%
26.8%
28.8%
10%
C1 C2 C3
Get rapid approval for the best labels Ensure payers and providers reimbursement Bring superior treatments to more patients
C1.1 C2.1
C3.1 C3.2
Approvals on time within target label (%) Reimbursements on time at target price (%)
Unit market share of Rebif, 1Q in arrears (%) Market share of Erbitux in 1st line mCRC (%)
60% 60%
20.1% 15%
80% 80%
22.4% 17%
100% 100%
23.7% 19%
7.5% 7.5%
5% 5%
Measure
Annual growth in total revenues (%) Return on sales (ROS) (%)
Min
2.5% 11.5%
Target
4.5% 12.5%
Max
6.5% 13.5%
Weight
10% 10%
F3
F3.1
24.8%
26.8%
28.8%
10%
Y1
y2
y3
y4
y5
y6
y7
24
Contents
A bit of philosophy About performance management How to choose your KPIs How to use your KPIs Building the right scorecard
25
Measures
Min-Target-Max
Min Target
Weight
Max Weight 5% 5% 10% Weight 10 % 5% 5% 5% Weight 7.5% 7.5% 5% 5% Weight 10% 10% 10%
PRODUCTS Deliver Superior Products P1 Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
P2 P3
CUSTOMERS Address Customer Needs C1 C2 Get rapid approval for the best labels Ensure payers and providers reimbursement Bring superior treatments to more patients
C3
Objective Numbers
FINANCIAL Grow Shareholder Value F1 F2 F3 Grow sales and revenue Increase return on sales (ROS) Contribute free cash flow
26
PRODUCTS Deliver Superior Products P1 Develop a balanced pipeline of innovative products Supply our products efficiently to the markets Comply with all current standards & regulations
which serve as the foundation of the BSC: Number of new preclinical and clinical projects: DP0s and DP1s (N) People, Processes, Customers Project timelines plan to PoC/Launch (months) andvs Financial goal leading to the vision of measures contribute
P2 P3
P2.1 Actual vs potential sales based onlong-term stock-outs (%) Divisional Statement of our
Objective
P3.1 Measure
CUSTOMERS Address Customer Needs C1 C2 Get rapid approval for the best labels Ensure payers and providers reimbursement Bring superior treatments to more patients
C1.1 Approvals onIdentifies time within the target label (%) Objective corresponding C2.1 C3.1 C3.2
Number
C3
FINANCIAL Grow Shareholder Value F1 F2 Grow sales and revenue Increase return on sales (ROS) Contribute free cash flow F1.1 F2.1 F3.1
Divisional objective for each measure For GBU/Function BSCs, Unit market share of Rebif, 1Q in arrears (%) identifies point of alignment Market share of Erbitux in 1st line mCRC (%) with Divisional BSC
Reimbursements on time at target price (%) Measure Annual growth in total revenues (%) Return on sales (ROS) (%) Free cash flow vs total revenues (%)
2 7
F3
Min
Target
Max
Weight 5% 5% 10%
Min
Target
Max
Weight 10 % 5% 5% 5%
P2 P3
CUSTOMERS Address Customer Needs C1 Get rapid approval for the best labels
Min
Target
Max
Weight 7.5%
Measure A precisely defined metric developed to support the achievement of the Divisional Ensure payers and providers C2 C2.1 Reimbursements on time at target price (%) 7.5% reimbursement objective.
Bring superior treatments to range more in a specified unit of measure (%, #, ) for the metric established at the Min-Target-Max Goal C3 patients Market share of Erbitux in 1st line mCRC (%) 5% beginning ofC3.2 the year. Performance is calculated on the min-target-max range. (See FINANCIAL Grow Shareholder Measure Min Target Max Weight next Value slide for target standards) C3.1 Unit market share of Rebif, 1Q in arrears (%) 5%
Weight
F2 F3
F1
An assigned percentage 5% reflecting the relative strategic priority of the measure. Increase return on sales (ROS) F2.1 on sales (ROS) (%) 10% The total weight ofReturn all measures on the BSC add up to 100%
Contribute free cash flow F3.1 Free cash flow vs total revenues (%) 10%
F1.1
10%
2 8
Min performance
Minimum acceptable performance below which corrective actions should be considered
29
100.0%
Performance
50.0%
0.0%
Below Min target Between red flag Min and Target: zone needs Performance is corrective at acceptable level action
Min
Target
Actual
Max
0%
50%
100%
95% 106% 120% 80% 113% 113% 122% 108% 133% 74% 108% 95% 101%
150%
OVERALL WEIGHTED AVERAGE PERFORMANCE PEOPLE Become Best Place to Work L1 Attract best people; have right skills at right time L2 Ensure employee learning & development L3 Ensure employee engagement PRODUCTS Deliver Superior Products P1 Develop a pipeline of innovative new products P2 Add value w ith effective lifecycle management P3 Supply our products efficiently to the markets P4 Comply w ith all current standards & regulations CUSTOMERS Address Customer Needs C1 Get rapid approval for the best labels 150% C2 Ensure payers and providers reimbursement C3 Bring superior treatments to more patients FINANCIAL Grow Shareholder Value F1 Grow sales and revenue F2 Increase return on sales (ROS) 150% F3 Contribute free cash flow
38% 0% 0%
100%
150% OVERALL WEIGHTED AVERAGE PERFORMANCE L1.1 Average time to hire (internal & external) L2.1 Grade 15+ positions filled from within Merck L3.1 Employee engagement survey P1.1 Number of new preclinical projects: DP0s P1.2 Project timelines vs plan to PoC/Launch P1.3 Annual growth in mean portfolio value P2.1 Annual growth in sales of mature products
133%
Overall BSC performance is calculated by adding up the performance weight of every measure on the BSC.* The BSC template includes an automated chart to provide significant insight in tracking progress from quarter to quarter. Overall BSC performance is directly linked to a BSC payout
74%
P3.1 Actual vs potential sales based on stock-outs P4.1 Major CAPAs/issues resolved on time C1.1 Approvals vs target label and date
150%
95%
C2.1 Reimbursements vs target price and date C3.1 Doctor & KOL survey of patient benefits C3.2 Market share of Rebif, 1Q in arrears C3.3 Market share of Erbitux, 1Q in arrears C3.4 Total revenues vs M&S and local R&D costs F1.1 Annual growth in total revenues F1.2 Organic sales growth vs market, 1Q in arrears F2.1 Return on sales (ROS) F3.1 Free cash flow vs total revenues
Q3-09
Q4-09
*During a given reporting period, one or more measures may not have any results, and so overall performance must be normalized to include only the measures with reported results. KPIs Informa 2013 31
32
More philosophy
34
35
Questions?