Criteria development

Why criteria development

• Various stakeholders define success of training

differently.
• Objectives of training and requirements on the job
may vary.
• Objectives of training and measures of success of
training may not be the same.
 Evaluation of criteria
• Good training outcomes need to be relevant, reliable,
discriminative and practical.
Relevance:
• Criteria relevance refers to the extent to which training
outcomes are related to the learned capabilities
emphasized in the training programme
Two ways in which training outcomes may lack relevance:
• Criterion contamination refers to the extent that
training outcomes measure inappropriate capabilities
or are affected by extraneous conditions.
• Criterion deficiency refers to the failure to measure
training outcomes that were emphasised in the training
objectives.
 Criterion deficiency, relevance, and contamination:

Outcomes
Outcomes Identified by
Outcomes Needs
Measured in Related to
Training Assessment and
Evaluation Included in
Objectives
Training
Objectives

Contamination Relevance Deficiency

• Reliability: Reliability refers to the degree to which
outcomes can be measured consistently over time.

• Discrimination: refers to the degree to which

trainees’ performances on the outcome actually
reflect true differences in performance.

• Practicality: refers to the ease with which the

outcomes measures can be collected
 Dimensions of Criteria

• Criteria can differ with time or based on use –

learning criteria and performance criteria.
Levels of criteria: Kirkpatrick’s model
Reaction: what the trainees thought of the particular
program. Does not measure learning.
• Positive reaction to a training program may make it
easier to encourage employees to attend future
programmes.
• Reaction can also help the organisation to decide
whetehr to continue with a trainer in future or not.
• Guidelines: Design a questionnaire based on
information obtained during the needs assessment
phase.
• Design the instrument – response to be tabulated and
quantified.
• Make the questionnaire anonymous.
• Provide space for opinions
• Pretest the questionnaire
Learning: Measuring the learning of principles, facts,
techniques, and attitudes that were specified as
training objectives.
• The measures must be objective and quantifiable
indicants of the learning that has taken place in the
training program. They are not measures of
performance on the job.
• Measuring the learning may involve a quiz or a test.
Reliable, clear scoring measurements need to be
established.
Behaviour: measurement of job performance
• Observation and interview over a period of time are
required to assess change, relevance of change, and
sustainability of change.
• 360-degree feedback is useful method.
• Assessments can be designed around relevant
performance scenarios and specific key performance
indicators or criteria.
• Results: Relate the results of the training program to
organisational objectives.
• Business data and financial data are used to evaluate
the training. To ensure the relation of trainee’s input to
business results, at the beginning itself, trainee
accountability and relevance should be communicated.

• Criticism of this model: Simple and one-dimensional

 Alternate model

• Alternate model given by Kraiger, Ford and Salas.

• Learning outcomes are divided into 3 central classes:
1. Cognitive outcomes: reflects concepts like
knowledge and cognitive strategies
2. Skill based outcomes such as automaticity
constructs
3. Affective outcomes such as attitudinal or
motivational constructs like self-efficacy and gioal
setting
 Learning

Learning Skill-based Affective outcomes

outcomes outcomes Altitudinal
Verbal knowledge Compilation Motivational
Knowledge Proceduralisation Motivational
organisation Composition disposition
Cognitive Automaticity Self efficacy
strategies Goal setting
 Outcome measures and summative evaluation;
Process outcomes and formative evaluation

• Outcome measures refers to criteria, like learning and

performance, that represents various levels of
achievement Thus summative evaluation describes
assessments using outcome measures that focus on
the effectiveness of completed interventions.

• Formative evaluation focuses on process criteria to

provide further information to help understand the
training system so that the originally intended
objectives are achieved.
 Time dimension

Immediate criteria Proximal criteria Distal criteria

(obtained in the (obtained in advanced (obtained after
training program) training or early in considerable time
transfer setting) in transfer setting)

Time
 Types of criteria

Criterion-referenced and Norm-referenced

measures:
• Criterion-referenced measures provide a standard of
achievement for the individual as compared with
specific behavioural objectives and therefore provide
an indicant of the degree of competence attained by
the trainee.
• Norm-referenced measures compare the capabilities
of an individual with those of other trainees.
• Objective and subjective measures:
• Measures that require the statement of opinions,
beliefs, or judgments are considered subjective. Eg.,
rating scales
• Objective measures. Eg., Absenteeism rate.