HR2 TRAINING & DEVELOPMENT 1

LESSON 6: Training Evaluation

Evaluation Designs
Threats to validity refer to factors that will lead an evaluator to question either:
(1)Internal Validity- the believability of the study results
(2)External Validity- the extent to which the evaluation results are generalizable to other groups of
trainees and situations.

Methods to Control for Threats to Validity

Pretests and Post-tests- One way to improve the internal validity of the study results is to first
establish a baseline or pre-training measure of the outcome. Another measure of the outcomes
can be taken after training. This is referred to as a post-training measure.
Use of Comparison Groups- Internal validity can be improved by using a control or comparison
group. A comparison group refers to a group of employees who participate in the evaluation
study but do not attend the training program. The Hawthorne effect refers to employees in an
evaluation study performing at a high level simply because of the attention they are receiving.
Use of a comparison group helps show that any effects observed are due specifically to the
training rather than the attention the trainees are receiving.
Random assignment refers to assigning employees to the training or comparison group on the
basis of chance alone. That is, employees are assigned to the training program without
consideration of individual differences (ability or motivation) or prior experiences.

Types of Evaluation Designs

Post-test-only -design refers to an evaluation design in which only post-training outcomes are
collected.
Pretest/post-test -refers to an evaluation design in which both pre-training and post- training
outcome measures are collected. There is no comparison group.
Pretest post-test with comparison group- refers to an evaluation design that includes trainees
and a comparison group. Pre-training and post-training outcome measures are collected from
both groups.
Time series -refers to an evaluation design in which training outcomes are collected at periodic
intervals both before and after training.
Solomon four-group-a design combines the pretest/post-test comparison group and the post-
test-only control group design. In the Solomon four-group design, a training group and a
comparison group are measured on the outcomes both before and after training. Another
training group and control group are measured only after training. This design controls for most
threats to internal and external validity.

A more rigorous evaluation design (pretest/post-test with comparison group) should be

considered if any of the following conditions is true:
1. The evaluation results can be used to change the program.
2. The training program is ongoing and has the potential to have an important influence
on employees or customers.
3. The training program involves multiple classes and a large number of trainees.
4. Cost justification for training is based on numerical indicators
5. Trainers or others in the company have the expertise to design and evaluate the data
collected from an evaluation study.
6. The cost of the training creates a need to show that it works.
7. There is sufficient time for conducting an evaluation. Here, information regarding training
effectiveness is not needed immediately.
 HR2 TRAINING & DEVELOPMENT 2

8. There is interest in measuring change (in knowledge, behavior, skill, etc.) from pre-
training levels or in comparing two or more different programs.

Determining Return on Investment

Return on investment (ROI) is an important training outcome. Cost-benefit analysis in this situation is the
process of determining the economic benefits of a training program using accounting methods that
look at training costs and benefits. Training cost information is important for several reasons:
1. To understand total expenditures for training, including direct and indirect costs
2. To compare the costs of alternative training programs
3. To evaluate the proportion of money spent on training development, administration, and
evaluation, as well as to compare monies spent on training for different groups of employees
4. To control costs

Seven categories of cost sources are costs related to program development or purchase, instructional
materials for trainers and trainees, equipment and hardware, facilities, travel and lodging, salary of
trainer and support staff, and the cost of lost productivity while trainees attend the program. This
method also identifies when the costs are incurred. Onetime costs include those related to needs
assessment and program development. Costs per offering relate to training site rental fees, trainer
salaries, and other costs that are realized every time the program is offered. Costs per trainee include
meals, materials, and lost productivity or expenses incurred to replace the trainees while they attend
training.
To identify the potential benefits of training, the company must review the original rea- sons that the
training was conducted. For example, training may have been conducted to reduce production costs
or overtime costs or to increase the amount of repeat business.

To calculate ROI, follow these steps:

1. Identify outcomes
2. Place a value on the outcomes.
3. Determine the change in performance after eliminating other potential influences on training
results.
4. Obtain an annual amount of benefits (operational results) from training by comparing results after
training to results before training (in dollars).
5. Determine the training costs (direct costs + indirect costs + development costs + over- head costs
+ compensation for trainees).
6. Calculate the total benefits by subtracting the training costs from benefits (operational results).
7. Calculate the ROI by dividing operational results by costs. The ROI gives an estimate of the dollar
return expected from each dollar invested in training.

Example:

𝑅𝐸𝑇𝑈𝑅𝑁 𝐵𝐸𝑁𝐸𝐹𝐼𝑇𝑆 − 𝐶𝑂𝑆𝑇𝑆 220,800 − 32,836

𝑅𝑂𝐼 = 𝑅𝑂𝐼 = 𝑅𝑂𝐼 = = 𝟓. 𝟕𝟐
𝐼𝑁𝑉𝐸𝑆𝑇𝑀𝐸𝑁𝑇 𝐶𝑂𝑆𝑇𝑆 32,836

Once the costs and benefits of the program are determined, ROI is calculated by dividing return or
benefits by costs. In this example, ROI was 5.72. That is, every dollar invested in the program returned
almost $6 in benefits.