Chapter 8

Evaluating and improving the effectiveness of Hearing Conservation Program

 HCP should be evaluated to:

o HCP is in compliance with the regulations
o HCP is successful in conserving hearing
 HCP subcomponent of Outcome Evaluation:
o Make changes for correcting deficiencies in a timely and cost-effective
manner
 Sample of Applicable Regulations:
o OSHA:
 Administer continuing effective HCP
 Perform an ongoing assessment of the HCP
o DoD:
 Administer annual evaluation of the HCP based on:
1. Standard threshold shift (STS) during annual audiograms
2. Percentage of employees enrolled in the HCP receiving annual
audiograms
o US Navy HCP Requirements:
 Required at least annual evaluation of the HCP effectiveness using:
1. Percentage of employees enrolled in HCP program receiving
annual audiograms
2. Percentage of employees with STS
3. Percentage of employees with OSHA-recordable hearing loss

 Frequency of Hearing Conservation:

o Help identify unnoticed deficiencies that can negatively affect outcome of
HCP:
1. Ideally evaluation of HCP effectiveness should occur on annual
basis.
2. Correction of deficiencies or revision of HCP must occur:
 When employer notices an Increase in noise he should notify
 Determine the cause of STS when recorded through careful
program evaluation
 Correct If due to poorly fitted HPD
 Individuals Responsible for assessing Effectiveness:
o Management
o Program Implementers
 o HCP Team
o Industrial Hygienist
o Audiologist
o Safety specialist
o Exposure monitor
o Noise control engineer
 Responsibilities of Management:
o Phase 1: Evaluation of Effectiveness (qualified workers, resources,
attending to comments, and evaluation on regular basis)
 Dedication of adequate resources for comprehensive program
evaluation
 Hiring and maintaining competent and qualified personnel for the
evaluation of the HCP program
 Performing regular evaluation of the effectiveness of the HCP
 Attending to comments and reactions to noise exposed workers and
evaluate the deficiencies
o Phase 2: Using the outcomes of the evaluation (accept and correct
deficiencies, financial and personnel resources commitment, and
determination to institute and disciplinary actions for non-compliance)
 Dedication of financial and personnel resources to correct any
deficiency
 Commitment to accept and correct any deficiencies revealed by the
periodic evaluation
 Determination to institute and carry out any necessary disciplinary
actions for noncompliance (not using HPD)
 Responsibilities of the Program Implementers or HCP Team:
o Phase 1: Evaluation of Effectiveness (time and resources to evaluate,
willingness to ask and interact, audiometric database analysis, and detect
problems sooner following best practices).
 Commitment of sufficient time and other resources to evaluate
HCP effectiveness
 Willingness to ask questions and interact with all members of the
team
 Performance of audiometric database analyses
 Commitment to detect problems or deficiencies sooner following
best practices
o Phase 2: Using the outcome of the evaluation: (Communication of
findings with employers and persistence of correcting deficiencies)
 Persistence and commitment to take steps to correct deviancies
  Effective communication of findings with management and to
affected employers

 Responsibilities of employees during the evaluation of the effectiveness:

1. Phase 1: Evaluation of the effectiveness
 Willingness to communicate any problems or concerns
 Paying attention to the noise levels and reporting any perceived
changes in levels
 Provision of feedback to management and HCP team
2. Phase 2: Using outcomes of the evaluation
 Cooperate in correcting deficiencies and maintain any noise control
measures

 Basic Approaches to program evaluation: (compliance with regulations,

evaluating hearing thresholds and every component of the HCP, quantify
improvements, and detect deficiencies in implementation)
1. To assess if the HCP is in compliance with the related regulations (OSHA,
DoD, MSHA, FRA)
2. To detect deficiencies in the implementation of the program’s component
3. To quantify improvements in the HCP
4. To evaluate audiometric thresholds of the workers
5. To evaluate the effectiveness of every component of the HCP

Zooming in:

1. Assessment of program compliance with related regulations:

o Read the regulations carefully
o Prepare a checklist of the relevant things to review
o Identify best practices that can help in efficient evaluation of the program
 Potential limitations of checklist Audits:

Evaluator takes info from program implementer. How accurate if this

info?

- It’s not as simple as following a checklist. It needs to be

accurate
o The rates of STS may be high in the presence of high
compliance reported on checklists
 - A careful review with implementers and cross check with data
of all records is very important to validate the evaluation
outcome
2. Seeking feedback from management and employees: (insights on deficiencies,
reasons, and solutions)
o Provide insights on HCP deficiencies
o Reasons on for such deficiencies
o Potential solutions for correcting the deficiencies that may not be apparent
using other assessment procedures
 Feedback attained using:
- Structured interviews
- Focus groups

Interviews of Management and Employees:

 Advantage:

Quick in detecting discrepancies

 Disadvantage:

It’s highly prone to bias due to fear from job loss or lack of promotion by the
management because their responses might not match that of management. This
consequently leads to underestimation of program deficiencies.

3. Focus Groups:
 A qualitative research tool that is frequently used in social research

 Goals of focus groups: (a historical viewpoint on HCP, gain understanding on

implementation, Identify worker acceptance, and spot training needs related to
HC)
o Obtain a historical viewpoint of hearing conservation program
o Gain understanding on actual implementation of worker for components
of the program
o Identify worker acceptance of the program
o Spot ongoing worker training needs related to hearing conservation

 Advantages: (Spontaneous responses, in-depth-interaction, good for those with

low literacy skills, and allows the worker to build up on his co worker response)
- Allows in-depth interaction between the facilitator and the
worker
 - Allow the worker to build up on other’s worker response
- Responses/comments are spontaneous
- Good for workers with low literacy levels (migrant workers)
 Limitations: (should have excellent ability in handling different personalities,
listen with empathy and sensitivity while thinking at the same time, analyses of
unstructured interviews are time-consuming, workers responses try to conform
the opinion of supervisors and managers )
- The group facilitator have to have excellent skills in handling
different personalities, listen attentively with empathy and
sensitivity and be able to think at the same time
- Workers responses may try to conform the opinions of
supervisors and managers
- Analyses of unstructured interviews is time consuming
4. Use of Task-based Statistics:
o Compliance with Audiometric Testing
 (The number of individuals enrolled in HCP who have a current
audiogram within the last 12 months/ number of ndividuals
enrolled in the HCP)X 100%
 Reasons for non-compliance may be explored and consequences
for non-compliers could be implemented (without the presence of
a valid reason)

o Compliance with Hearing Protection Device Use:

 (Number of individuals in an area where HPD use is required who
are observed (through random checks) using HPD/ Number of
individuals required to use HPDs) X 100
 This approach can be strengthened by determining the
number of workers who are using their HPDs correctly
with proper insertion depth and proper replacement of
worn out devices as needed

5. Use of Noise Exposure Statistics include: ( based on exposure during certain

tasks, percentage of employees removed from HCP, dosimeter recordings,
hazardous impact of noise exposure measured, number of areas where engineering
controls where implemented, and Effectiveness of noise control measures in dB)
o Percentage of workforce in potentially hazardous noise based on
dosimeter
 o Percentage of workforce in potentially hazardous noise based on
hazardous impact on noise exposure levels
o Percentage of employers removed from the HCP due to reduction of
noise dose
o Effectiveness of noise control measures in dB in terms of the
difference in pre-control noise levels and post-control noise levels.
o Percentage of workforce exposed to hazardous noise in certain tasks
even though the daily noise level may be low
o Number of areas of machines where engineering noise controls were
implemented

6. Evaluation of Audiometric Data:

o Evaluation based on the variability in Serial Monitoring Audiometry:
o Compile the audiograms including demographic data
o Compile first baseline audiogram per person per year
o Choose a restricted database for analysis
 A sample size of at least 30 employees is needed
 For seniors employees, choose those who at least have 10
monitoring audiograms and assess the variability in the 4
latest monitoring audiograms
 For newly hired employers, choose those who have at least 3
monitoring audiograms and assess the variability among these
audiograms
 To assess HCP effectiveness/evaluate the entire the group,
choose those who have a baseline audiogram during the year
of HCP and all the annual/monitoring audiogram until the
current year
 To evaluate the effect of noise exposure, then match the control
group of non-exposed employees to noise exposed restricted
group
o Plot the average auditory thresholds for each year for each test frequency
and examine visually if there is any aberrations
o Calculate the following two indicators of variability for the control and
the restricted groups:
 Percent Worse Sequential :
o The percentage of workers who show a 15 dB
worsening in thresholds in two sequential
audiograms in any ear at any frequency
 Percent better or Worse Sequential:
 o The percentage of workers who show a 15 dB
improvement or worsening in thresholds at any test
frequency in any ear between two sequential
audiogram
 If analysis is being conducted based on comparisons of tests 4 and 5 and any
later tests then one should use both %W s and %BWs statistics. Use the
following criteria to rate the HCP program:
o %W s < 17 is acceptable, 17 – 27 is marginal, and > 27 is unacceptable.
o %BWs < 26 is acceptable, 26 – 40 is marginal, and > 40 is unacceptable.

 If analysis is being conducted includes the first 4 sequential tests, then use
%Ws statistics to rate the HCP performance. This is because the threshold
improvement can appear initially due to the learning effect. The criteria for
evaluation if the following:
o %W s < 20 is acceptable, 20 – 30 is marginal, and > 30 is unacceptable

 If the rating fall into two different ranges you can describe the HCP performance
using two different ranges: e.g. marginal to acceptable.
 If rating is unacceptable, search for the sources of high variability (e.g. improper use
of HPDs, audiometric testing conducted using poorly controlled procedures).

 Limitations of the use of variability in serial monitoring audiometry for assessing

HCP:
o Although the rating may be marginal or acceptable the some individuals
may suffer significant threshold shifts (Sig TS) from the baseline
o Those with poor auditory sensitivity may show greater threshold
variability than those with better hearing. This may not be necessarily
connected to poor performance of an HCP

7. Evaluation of Individual Effectiveness:

 Significant changes in auditory thresholds in the absence of other contributing
factors suggest the possibility of ineffective HCP
 If no workers had STS, then the HCP is effective

Evaluation of Overall Program Effectiveness

When we can’t evaluate employer’s audiometric thresholds, NIOSH suggests two

ways to evaluate the program effectiveness
 Step 1: Evaluate the integrity of the audiometric data (ANSI): Using sequential
thresholds (%W s, %WBs)

 High variability can occur because of : (Poor recording, insufficient

control, poor calibration, and poor test environment)
o Insufficient control of audiometric procedures
o Poor control of test environment
o Poor calibration procedures
o Poor recording of audiometric information
 Low variability:
o Well controlled program

Step 2: Consideration of the rate threshold shift due to occupational noise or solvent
exposure (should be of the same age )

Disadvantages: All workers must have an audiometric testing including those who are
non-exposed

 In the absence of availability to audiometric thresholds of those who are non-

exposed, an STS of 3 % or less for those who are exposed indicates an effective HCP.
The 3% criterion was calculated by NIOSH by using data for those who are not
exposed.
 Other recommended criterion: 3-6%. Higher than that indicates that HCP is
ineffective.
 If there was no control age matched group, workers who have other contributing
factors should not be included in the 3 % criterion. These are counseled and
followed up separately.
 ISO 1999 can be used to predict other contributing factors effects such as age,
gender, noise exposure on hearing loss using the equal-energy principle. Several
other factors are taken into account like medical history, hearing loss, drugs and can
be entered into the Noise Scan database which is based on ISO 1999.
 It also assumes that a person with lower confounding factors is less vulnerable to
noise than a person with more confounding factors (smoking, firearms). The
probability of NIHL is based on the number of these risk factors such as the
following:
o 80 %: ontologically normal population with no reported history of
excessive noise exposure/1 risk factor
o 50 %: For those with two or three risk factors
 o 25 %: For those who have more than 3 risk factors

 Limitations: The use of STS rates/ High STS rates can result from deficiencies in
HCP. In the presence of low STS rates the following factor should be ruled out as
a cause:
 High employee turnover
 Previous baseline revisions in a high percentage of the population
 Changes in the audiometric practices or instruments from the
baseline till the current audiograms

 Confounding factors:
o Exposure to solvents/ototoxins can increase the risk for NIHL
 Prevention Index: Useful for governing agencies to identify
industries showing poor performance in terms of reducing injuries
and identify who might need more assistance
 The incidence rate (i.e. high vs low) for worker
compensation claims
 The number of worker compensation claims
 The two rankings are averaged to generate PI
 Use of Surveillance data
 The true magnitude of NIHL may be underestimated by the
surveillance system
Suggestions to improve the usefulness of the data for
surveillance (Middendorf, 2004):
o Check the data for accuracy.
o Include all measured noise exposures even if the
permissible exposure limit and action level
exposures were measured on the same worker.
o Include additional noise exposure parameters, such
as sample durations and frequency of exposure.
o Include information about engineering controls.
o Include information about the use of hearing
protection.
o Include standardized job codes and processes
 Use of database software
o Generate several types of statistics that can check
compliance and effectiveness of the HCP, for
example:
o Average thresholds of all workers.
o List of workers showing OSHA STS.
o List of workers showing OSHA recordable hearing
loss.
o List of workers with missing or invalid audiometric
data