EVALUATION PROCESS

Evaluation Process

Conduct a Needs Analysis

Develop measurable learning objectives and analyze transfer

of training

Develop Outcome measures

 Evaluation Process

Choose an evaluation strategy

Plan and execute the evaluation

 Need Analysis

Need assessment helps identify what knowledge,

skills, behaviours or other learned capabilities are
needed
Helps identify where the training is expected to have
an impact.
Helps focus the evaluation by identifying the
purpose of the program, resources needed and the
outcomes that will provide evidence that the
program has been effective.
 Learning Objectives

Objective refers to purpose and expected outcome of

training activities.
Objectives can be for each session as well as overall
objectives for the program.
Training objectives based on TNA help employees
understand why they need training and what they
need to learn.
Also useful for identifying the types of training
outcomes that should be measured to evaluate a
training program’s effectiveness
 3 Components of training objectives

A statement of what the employee is expected to

do( performance or outcome)
A statement of the quality of level of performance that is
acceptable ( criterion)
A statement of the conditions under which the trainee is
expected to perform the desired outcomes( conditions)
Ex: “ After training , the employee will be able to express
concern( performance) to all angry customers by offering
a brief, sincere( fewer than 10 word) apology in a
professional manner( criteria) no matter how upset the
customer is ( conditions)
 Characteristics of good training objectives

Provide clear idea of what the trainee is expected to be

able to do at the end of the training.
Include standards of performance that can be measured
or evaluated.
State specific resources ( tools, equipments) that the
trainee needs to perform the action or behaviour
specified.
Describe condition under which performance is
expected to occur – physical work environment-night,
high temp; mental stress- angry customers, equipment
failure
 Outcome measures

Based on learning objectives and analysis of transfer

of training , outcome measures are designed to
assess the extent to which learning and transfer have
occurred.
 Evaluation strategy

Formative / Summative
Evaluation design
 Determining whether outcomes are appropriate

Training outcomes need to be

 Relevant,
Reliable,
 Discriminative and
Practical
 Criteria Relevance

Refers to the extent to which training outcomes are

related to learned capabilities emphasized in the
training program.
One way to ensure the relevancy of the outcomes is
to choose outcomes based on the learning objectives
of the program
 Criterion deficiency

Refers to the failure to measure training outcomes

that were emphasized in training objectives.
For excel training to calculate statistics on a data set.
Evaluation design that uses test of knowledge on just
key strokes is deficient because it does not measure
outcomes that were included in training objective –
mean, SD etc.
 Criterion Contamination

Refers to the extent that training outcomes measure

inappropriate capabilities or are affected by
extraneous conditions.
It can occur if trainees are asked to perform learned
capabilities using equipment, time constraints or
physical conditions that are not similar to those in
working conditions.
Is the measurement commensurate with the training?
14

Measured Trained

Contamin Releva Deficien

ation nce cy
 Criteria Reliability

Refers to the degree to which outcomes can be

measured consistently over time.
A reliable test includes items for which the meaning
or interpretation does not change over time.
A reliable test allows the trainer to have confidence
that any improvements in post-training scores from
pre-training scores are the result of a learning that
occurred in the training program not test
characteristics (items more understandable second
time) or test environment(comfortable classroom)
 Criteria Discrimination

Refers to the degree to which trainee’s performance

on the outcome actually reflects true differences in
performance.
The test must detect true differences in trainees
knowledge
People who score high on the test should have better
understanding than people who score low.
 Criteria Practicality

Refers to the ease with which the outcome measures

can be collected.
For ex in evaluating a sales person training program,
it may be impractical to ask customers to rate sales
persons behaviour