THE PHILLIP ROI MODEL

1. Definition

The Phillips ROI Model is a methodology and process for L&D and HR teams to tie the costs
of training programs with their actual results. It can be used the words “methodology”,
“process”, and “model” interchangeably to describe

In the early 2000s, Jack Phillips built on the Kirkpatrick model to develop a new level, level
5 - Return on Investment (ROI). In other words, Phillips ROI is an upgraded model from
Kirkpatrick. It classifies data from different types employee training programs to measure:

- The reaction of participants

- The actual learning of participants
- The behavior change from these learning
- The final result

But organizations that spend millions – or potentially billions – of dollars on training

programs want more than results. To sanction large budgets, they need to see the monetary
actual value of these programs – their return on investment (ROI).

The Phillips ROI Model makes the monetary benefit clear for organizations by adding a fifth
layer to calculate the ROI of each program. This additional level compares learning business
impact outcomes to total training costs.

The Phillips Methodology isolates the effects of the program from other influencing factors at
all levels of the training program’s evaluation. This filtered data empowers L&D teams to
derive accurate net monetary benefits from the various training programs.

2. The Five Levels of the Phillips ROI Framework

Level 1: Reaction
In common with the Kirkpatrick model, the Phillips ROI Models begin by evaluating the
participants’ reaction to the training they received. The most common approach is to use
short questionnaires or surveys to collect data about what people thought about their training.
This helps an organization assess whether the conditions necessary for learning were in place.

Level 2: Learning
In this step, participants complete a multiple-choice questions (MCQs) survey or quizzes both
before and after the training. Training managers interpret the responses to determine how
much knowledge they’ve acquired. Once again, the Kirkpatrick Model evaluates learning the
same way.
Level 3: Application and Implementation
Level 3 of the Kirkpatrick Model looks at workplace behavior and assesses whether
participants are using what they learned on-the-job. One of the main criticisms of the
Kirkpatrick Model has always been that it doesn’t gather enough data to help improve
training; it simply tells you whether the training was put into practice or not. Jack Phillips felt
that this level could be improved, so he expanded it to cover both application and
implementation. The Phillips ROI Model makes it far easier to see why training does or
doesn’t translate into workplace changes. If there is a problem, the Phillips ROI Model helps
you determine whether the issue lies with the application of the learning or with its
implementation? This subtle but crucial difference can prove extremely helpful for
organizations.

For example, let’s imagine that employees in a factory receive training to operate a new type
of equipment. The Phillips Model level 2 evaluation confirms that the training was
successful, but the level 3 evaluation shows that the employees are not using their new skills
and knowledge. A Kirkpatrick level 3 evaluation would simply tell you that the training
wasn’t being implemented. But a Phillips ROI Model level 3 evaluation would show you
more about where the problem lay. For instance, the employees could be able to operate the
new machines but are faced with a problem that prevents them from implementing it, such as
the machines being faulty or inoperable.

Level 4 – Impact
At level 4 of the Kirkpatrick Model, only business outcomes and impact are measured. The
Phillips ROI model expands the focus of Level 4 from unilateral outcomes to
multidimensional effects. The new model allows businesses to analyze whether factors
(external or internal) impact the training content and other factors contributing to the
participants' final performance.

Level 5: Return on Investment

Unlike Kirkpatrick, which only measures training outcomes based on stakeholder
expectations (ROE). Level 5 was designed by Jack Phillips to use cost-return on investment
to determine how the value of training programs affects employee productivity. This level
helps businesses measure the cost of investing in training, communicate the benefits of the
program to other parts of the company, and provide evidence to executives about the value of
training programs. create. answers to questions such as “are they profitable?”, “if so, how
much?”

3. Advantages of using Phillips ROI Model for Training Evaluation

● Traces the complete chain of impact
According to the creator of the model Jack Phillips, the business impact observed in level 4
and the ROI calculated in level 5 are not independent. Data across all preceding levels affects
the final ROI produced by the model.
A chain of impacts develops as participants gain skills and knowledge (level 2), apply them
on the job (level 3), and generate business impact (level 4). So, measuring data across all
levels helps evaluate the training program with high precision.

If level 5 produces positive ROI, data from past levels will show the specific impact of the
training program without any other factors. Without this data, the success of the training
program would only be guesswork.

Also, detailed data from the chain of impact allows debugging of the training program if level
5 produces a negative ROI figure. Training managers can pinpoint the exact reason for
failure.

● Measures intangibles
The Phillips ROI model accepts that you cannot measure certain outcomes in monetary value
and the final ROI won’t represent such outcomes.

Outcomes such as customer satisfaction, employee satisfaction, and team bonding are hard to
measure in numbers. Therefore, the Phillips Model measures such “soft” outcomes in
addition to “hard” numerical outcomes.

The Kirkpatrick Model measures business impact in only numbers and does not consider
intangible benefits during evaluation.

4. Disadvantage of Phillips ROI model

While the Phillips Model is a significantly better way to evaluate training programs than the
Kirkpatrick Model, it is not without its fair share of criticisms.

● Late measurement of ROI

The ROI of a training program evaluated with the Phillips Model is not known until the
training program is over. It would be impossible to make changes if we find out that the
training program was a failed project after the project is over.

Jack and Patti Phillips address the problem of delayed ROI in their 2010 book “The
Consultant’s Guide to Results-Driven Business Proposals: How to Write Proposals That
Forecast Impact and ROI”.

This book presents strategies to help estimate ROI, determine goals, plan execution, and set a
budget for a training program. These strategies help to predict training outcomes before the
organization decides to make a large financial commitment.

● Useful for only 5-10% of training programs

The Phillips Model provides more than enough data until level 4 to know how successful or
unsuccessful a training program is. The ROI calculation in level 5 is good to know, but it’s
not necessary in most cases.

According to Jack Phillips:

○ All programs need level 1 evaluation

○ About 90% of programs need level 2 evaluation
○ About 30% of programs need level 3 evaluation
○ 10-20% of programs need level 4 evaluation
○ Only 5-10% of programs need level 5 evaluation

Various factors such as visibility of the training program, its goals, and its costs will decide if
a level 5 study is necessary. Jack also remarks that these 5-10% of programs are usually the
most expensive training programs with the largest audiences and an extraordinary amount of
time and resources.

5. Phillips ROI Framework best-suited for

The Phillips ROI Framework is commonly used to measure the effectiveness and financial
impact of various types of training, such as leadership development, sales training, customer
service training, and technical training because of several reasons:
- Comprehensive approach: The Phillips ROI Framework provides a comprehensive
approach to evaluating training programs, from assessing participant satisfaction to
calculating the financial return on investment. This approach allows organizations to
evaluate the impact of training programs on multiple levels, including reactions,
learning, behavior, results, and ROI.
- Financial focus: The Phillips ROI Framework places a strong emphasis on the
financial impact of training programs, which is important for organizations that need
to justify the investment in these programs. By calculating the ROI of a training
program, organizations can determine whether the program is providing a positive
return on investment.
- Measurable outcomes: The Phillips ROI Framework focuses on measuring tangible
outcomes of training programs, such as increased productivity, sales revenue, and
customer satisfaction. These outcomes are important for organizations that want to
demonstrate the value of their training programs and make data-driven decisions
about future investments.
- Applicable to various types of training: The Phillips ROI Framework is applicable
to various types of training programs, including leadership development, sales
training, customer service training, and technical training. This flexibility allows
organizations to use the framework to evaluate different types of training programs
and determine their impact on employee performance and business outcomes.
- Sales Manager: Sales managers can use the Phillip ROI Model to measure the
effectiveness of sales training programs in terms of improving the performance of
 sales representatives. By evaluating the ROI of sales training programs, sales
managers can determine which programs are most effective and make data-driven
decisions on how to allocate resources for future training initiatives.
- Marketing Manager: Marketing managers can use the Phillip ROI Model to measure
the effectiveness of marketing training programs in terms of improving the skills and
knowledge of marketing professionals. By evaluating the ROI of marketing training
programs, marketing managers can determine which programs are most effective and
make data-driven decisions on how to optimize the performance of their team.
- Learning and Development Specialist: Learning and development specialists can
use the Phillip ROI Model to evaluate the effectiveness of training programs and to
identify areas for improvement.
- Business Analyst: Business analysts can use the Phillip ROI Model to analyze the
financial impact of training and development programs on the organization and to
make recommendations for improving these programs.
- Project Manager: Project managers can use the Phillip ROI Model to measure the
ROI of training programs for project team members and to ensure that these programs
are aligned with project objectives.

On the other hand, Phillips ROI Framework may not be the best approach to evaluate other
types of work, such as project management, product development, or operational processes.
In such cases, other evaluation frameworks may be more appropriate because these areas of
work may have different objectives and outcomes that are not related to training and
development programs. The Phillips ROI Framework is specifically designed to evaluate the
effectiveness and financial impact of training and development programs, and may not be
suitable for evaluating other types of work.

For example, project management and product development may have specific metrics and
KPIs (Key Performance Indicators) that are used to measure success, such as on-time
delivery, project scope, and quality of deliverables. Operational processes may have metrics
related to efficiency, cost savings, and quality control. These types of work may require
different evaluation frameworks that are tailored to their specific objectives and outcomes.