Professional Documents
Culture Documents
IAEA Safety Culture Perception Questionnaire For License Holders - V12
IAEA Safety Culture Perception Questionnaire For License Holders - V12
Working document
May 2017
1
Contents
1. About the IAEA Safety Culture Perception Questionnaire ................................................................... 3
1.1 Background ......................................................................................................................................... 3
1.2 The IAEA Safety Culture Framework ................................................................................................... 4
2. How to Implement the SCPQ-LH ............................................................................................................... 6
2.1 Communication with Participants....................................................................................................... 6
2.2 Questionnaire Procedures .................................................................................................................. 8
2.3 Response Rates ................................................................................................................................. 10
2.4 Example of Survey Progress Checklist............................................................................................... 11
Appendix A – Sample Recruitment Email ................................................................................................... 12
Appendix B – Sample Recruitment Flyer .................................................................................................... 13
Appendix C – Sample Participation Email ................................................................................................... 14
Appendix D – Sample Participation Handout.............................................................................................. 15
Appendix E – IAEA Safety Culture Perception Questionnaire – Version 13................................................ 16
Appendix F – Behavioural Items – Optional Supplement to IAEA SCPQ-LH* ............................................. 24
Appendix G – IAEA SCPQ-LH Advisory Group ............................................................................................. 26
2
1. About the IAEA Safety Culture Perception Questionnaire
1.1 Background
1
For details on IAEA Safety Standard Guidance on assessment of safety culture, see sections 6.35-6.39 in
INTERNATIONAL ATOMIC ENERGY AGENCY, Application of the Management System for Facilities and
Activities, IAEA Safety Standards Series, Safety Guide No. GS-G-3.5, IAEA, Vienna (2009). For a full description
of the IAEA safety culture independent and self-assessment methodologies, see
INTERNATIONAL ATOMIC ENERGY AGENCY, OSART Independent Safety Culture Assessment (ISCA)
Guidelines, IAEA Services Series No. 32, IAEA, Vienna (2016) and INTERNATIONAL ATOMIC ENERGY
AGENCY, Performing Safety Culture Self Assessments, IAEA Safety Report Series No. 83, IAEA, Vienna (2016).
3
The aim of the IAEA is to continuously improve the SCPQ-LH through analysing survey data.
Since the development of the IAEA SCPQ-LH is a work in progress, the current psychometric
properties (reliability and validity) have not been fully tested. Organizations receiving support
from the IAEA in implementing the questionnaire are also invited to take part in the further
scientific development of the questionnaire. Feedback on the questionnaire is generally sought in
order to facilitate the continuous improvement process.
The current version of the IAEA SCPQ-LH can be found in Appendix E of this working
document. A set of optional behavioural items that can be added to the SCPQ-LH is included in
Appendix F.
The items in the SCPQ-LH were designed to assess the five IAEA safety culture characteristics
found in the IAEA’s safety culture framework2, which is summarized below:
2
For the full IAEA safety culture framework, see Section 2.36 in “INTERNATIONAL ATOMIC ENERGY
AGENCY, Application of the Management System for Facilities and Activities, IAEA Safety Standards Series,
Safety Guide No. GS-G-3.1, IAEA, Vienna (2006)”.
4
Safety is learning driven:
The organization learns from operational experience and uses assessments (internal and external)
to improve safety procedures and behaviours. A questioning attitude is present across all
organizational levels and people are willing to report deviations and errors. Safety performance
is tracked and effective actions are taken to improve.
5
2. How to Implement the SCPQ-LH
Before Distribution
In order to ensure the timely and voluntary participation of staff, it is useful to communicate
some general information regarding the questionnaire prior to distribution. In the weeks prior to
distribution (e.g.: two weeks before the distribution date), staff is to be informed of the purpose
of the questionnaire, who will be conducting it, the confidentiality agreement, and when and
where they may be able to participate. It is important to stress both the anonymous and
voluntary nature of the questionnaire, as coercion is not conducive to honest responses. A sample
participant recruitment flyer can be found in Appendices A & B. This recruitment flyer can be
used both in paper and email format.
Typically, nuclear organizations have established protocols when questionnaires such as the
SCPQ-LH are distributed to employees. These protocols may require input and permission for
distribution from various stakeholders such as the Senior Management Team, Internal
Communications Function, Information Technology Department and Line Managers. It is
therefore important for the survey administrators to assure that these stakeholders are sufficiently
informed and have provided necessary support, input and/or approvals of the process prior to
survey administration (see also Section 2.3 Response Rates).
The communication efforts required to facilitate staff participation varies based on how the
questionnaire will be provided to staff members. The below information assumes that the
questionnaire is distributed through an online survey software. At most modern facilities, this is
an effective way of collecting and also analysing responses, since the data will be readily
inserted into a software. Most nuclear organizations have stringent firewall protocols that are
designed to protect their internal information technology assets and information. It is therefore
vital to test beforehand the compatibility of the online survey software with the information
technology firewall and protection measures. Nuclear organizations are typically also reluctant to
divulge their staff members’ email addresses to external parties. It may therefore be advisable to
seek the cooperation of the Nuclear Power Plant’s internal communication or ICT department to
administer the survey questionnaire internally to all staff.
If the questionnaire is distributed in person, a mechanism ensuring that participants can return
the questionnaires after completion while maintaining participant anonymity is needed. A
common method is to invite groups of employees to complete the questionnaire in a dedicated
location during their working hours.
6
When Emailing the Questionnaire Link
When the employees being surveyed have email addresses and computer access through work on
a regular basis, the participation information can be sent directly, including a link to the
questionnaire itself. This email needs to restate that the questionnaire is anonymous and that
participation in the assessment is voluntary for the employee. In addition to providing a link to
the questionnaire, this email includes the contact information of the person(s) responsible for the
questionnaire, and the window during which the questionnaire will be active. The email provides
information about receiving the report containing the results of the questionnaire. A sample
email is provided as Appendix C.
Once the questionnaire is completed by all interested staff members the data needs to be
analysed. The IAEA can organize support in analysing results upon request. The resulting report
will constitute one part of the information needed to perform a full safety culture assessment, as
the cultural themes coming out of the questionnaire need to be analysed together with the results
from the other methods. The overarching results are then presented to the organization as part of
the full assessment.
Participant Anonymity
As the SCPQ-LH has been designed as a means of assessing culture of the organization, it is not
in any way beneficial for the organization or researchers to be able to identify the responses of
any individual participant. It is specifically important to maintain participant anonymity in order
to promise participants that there will be no negative consequences that may arise from honest
responses.
In cases of facilities with few potential participants, or those in which certain positions are only
held by a small number of employees, certain questions will not be included in the questionnaire.
For example, it is possible that certain facilities employ a very small number of administrative
staff where it would be possible for management to identify individual participants and thus
remove anonymity by looking at the responses of participants who have identified
‘administrative’ as their position. In such cases, the questionnaire is not to include certain
demographic breakdowns (such as gender and years of employment, and position within the
organization). Regardless, this information would not provide any deeper insight on safety
culture perceptions of an organization as a whole.
7
2.2 Questionnaire Procedures
If the working language of the facility is anything other than English, the SCPQ-LH needs to be
translated, and the translation also needs to be back-translated, to ensure that meaning has
remained the same3. Upon request, the IAEA may provide support in producing an accurate
translation. The IAEA may also be consulted to ask for the availability of the SCPQ-LH in other
languages than English.
Selecting Participants
The IAEA General Safety Requirements on Leadership and Management for Safety, GSR Part 2,
require that “Senior management shall ensure that self-assessment of leadership for safety and of
safety culture includes assessment at all organizational levels and for all functions in the
organization” (emphasis added). Therefore, the aim is that participants constitute a
representative sample of the facility as a whole. The goal would be to have each department
represented by an equivalent percentage of participants. However, this is impossible to ensure
with a voluntary sample. For this reason, it is best to either solicit participation from all staff
members, or to use a stratified sample. It is important to ensure that all departments and work
groups are represented in the final sample.
Larger facilities may choose to use a stratified sample in order to maintain a manageable number
of responses. This method of sampling is partially random, in that the individuals selected from
each department are selected randomly. A useful way to use this method of sampling is to first
establish a representative number of potential participants from each department (ex: 50% of
workers in each department). Next, this number of participants is selected at random from all
employees in that department. To do this, one may assign numbers to all employees in a
department, and then use a random number list or generator to select participants until the quota
for the department is met.
The first thing that needs to be determined is what type of access staff have to computers at
work. This may vary between departments and positions. For staff with regular computer access
at work, all that is needed is a flyer to be sent out via e-mail (example included in Appendix A)
3
Back-translation refers to a process where a document previously translated into another language is translated
back to the original language. This is done by a translator who has not previously been involved in the original
translation. The two versions of the document are compared, conveying any potential errors or misconceptions,
potentially owing to cultural differences that are hard to spot without going through this process. The back
translation process is therefore very important to ensure that the accuracy and scientific quality of the questionnaire
is kept.
8
prior to the questionnaire distribution in order for potential participants to decide if they are
interested in participating, as well as put forth any concerns they may have.
For staff without regular computer access, computer stations need to be set up at which
participants will be able to respond to the questionnaire. These stations need to be set up in a way
that will ensure that participants are given the privacy required to feel comfortable and guarantee
that their responses remain anonymous. Once it is known when these stations will be available
and where they will be located, a flyer is distributed to all potential participants (example
included in Appendix B). The flyer provides an introduction to the questionnaire and relevant
contact information to a person within the organization as well as to the assessor/researcher (if
these roles are not covered by the same person).
When distributing the questionnaire among employees, it is important to ensure that all
participants will be provided the time, and method to access the questionnaire, as well as
assuring that all responses will remain anonymous. It is best to send the questionnaire directly to
participants who have regular computer access during work hours, as this is the easiest way for
them to complete the questionnaire at work, within the time allotted without disruptions to their
day.
For staff without regular computer access, the location and availability of computer stations is to
be made clear once again during the period in which the survey is active. Additionally, computer
stations need to be private to allow participants to take the survey without feeling as though
someone is judging their responses.
The survey is to be made available for a period of time that allows people to participate at a time
that best suits them, such as a two week window. The period needs to have a clear end date, in
order to encourage people to complete it in a timely manner.
Reminder
Half way through the survey period, it is encouraged to send out a reminder either verbally or
through email, reminding staff who have not participated, how much time is left in the survey
period, along with all the required information for accessing the survey.
Reporting Results
Once the survey period is complete, the data collected needs to be analysed. The results of the
need to be interpreted with the assistance of a qualified safety culture assessor. The raw data is
normally used to compile a summary report to be provided to the safety culture assessors
according to the assessment methodology. Once the safety culture assessment team have
prepared a report of their full findings, this report and the actions stemming from it needs to be
9
communicated to the entire organization4. A summary of the survey results may also be made
available to all staff who participated in the survey.
A sufficiently high response rate is crucial to delivering reliable data as a result of the survey.
However, response rates vary largely across organizations depending often upon cultural factors
(e.g. is responding to questionnaires an established practise in the organization’s culture?). In
many cases it is suggested to solicit participation from all staff members, in order to increase the
likelihood of having an adequate representation. Also, this reduces the perception of inequality
between staff members holding similar positions, in the case where only some staff members are
invited to participate.
Achieving a high response rate requires an active process. In addition to following the above
suggestions regarding communication with survey participants this may include allocating
dedicated work time to complete the survey, ensuring that senior management express support of
the survey completion, and/or ensuring that respondents are aware of benefits of sharing their
opinions. The latter relates to the clear communication of how results are acted upon, as response
rates are likely to be influenced by whether or not the organization is perceived to be using the
information collected through surveys to effectively improve.
4
See INTERNATIONAL ATOMIC ENERGY AGENCY, Performing Safety Culture Self Assessments, IAEA
Safety Report Series No. 83, IAEA, Vienna (2016) for further detail on reporting assessment results.
10
2.4 Example of Survey Progress Checklist
Activities Timing Responsible party
11
Appendix A – Sample Recruitment Email
We would like to invite you to participate in a safety culture questionnaire that [name of the
organization] is conducting. This survey is a part of a larger safety culture assessment aimed at
improving safety performance within the utility and organization as a whole.
This questionnaire will be available online starting [start date] until [end date]. Time used for
participation will be considered part of regular work time. The link will be provided in a future email on
the first day the survey is active.
If you have any questions, please contact either [contact person in support organization (when
applicable)] or [Contact point].
12
Appendix B – Sample Recruitment Flyer
We would like to invite you to participate in a safety culture questionnaire that [name of the
organization] is conducting. This survey is a part of a larger safety culture assessment aimed at
improving safety performance within the organization as a whole.
This survey will be available online starting [start date] until [end date]. Computers will be available at
[location of computers] between the hours of [start time] and [end time] to participate in the survey.
Time used for participation will be considered part of regular work time.
If you have any questions regarding the survey, please contact [contact person] or [contact person in
support organization (when applicable)].
13
Appendix C – Sample Participation Email
As per our previous email, we are now conducting an online safety culture survey.
This study involves completing an on-line survey, which takes approximately 30 minutes to complete. All
information obtained in this study will be kept strictly anonymous and confidential. The results of this
study will be presented as a summary and no individual participants will be identified.
We will be using the results of this survey to assess our safety culture. The results will be made available
[method of distribution to employees].
If you have any questions regarding the survey, please contact [contact person] or [contact person in
support organization (when applicable)].
14
Appendix D – Sample Participation Handout
As per our previous letter, we are now conducting an online safety culture survey .
We will be using the results of this survey to assess our safety culture. The results will be made available
[method of distribution to employees].
This study involves completing the on-line survey, which takes approximately 30 minutes to complete.
All information obtained in this study will be kept strictly anonymous and confidential. The results of
this study will be presented as a summary and no individual participants will be identified. Your
responses will not be monitored, and your time will be counted towards regular work hours.
If you have any questions regarding the survey, please contact [contact person] or [contact person in
support organization (when applicable)].
15
Appendix E – IAEA Safety Culture Perception Questionnaire – Version
13
Please read the following before you fill out the questionnaire
INSTRUCTIONS
2. Answer all the questions as completely and honestly as possible. If you are not comfortable
answering a question, leave it blank and move onto the next question.
Please Note:
Your individual answers will be kept confidential to the research team, your participation is
voluntary and you can withdraw from responding to the questionnaire at any point up until you
submit it (see information sheet for specific instructions).
If you have any questions please ask the person who is coordinating your session. You can also
contact [support person in supporting organization when applicable] at [phone number], or at [e-
mail address].
16
DEMOGRAPHICS: The information you provide below will be used to see how various groups of employees
differ in their opinions about safety. The information will be used for descriptive purpose only. The
information will only be reported in terms of large groups, ensuring that your responses remain completely
anonymous.
…
…
…
…
…
…
Contractor/External
17
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
I feel free to ask questions about any issue 1 2 3 4 5 6 7
18
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
I feel free to report mistakes 1 2 3 4 5 6 7
19
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
Our corrective actions are effective 1 2 3 4 5 6 7
We adhere to procedures 1 2 3 4 5 6 7
20
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
Internal assessments helps us improve our safety
1 2 3 4 5 6 7
performance
21
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
Workgroups cooperate effectively with one another 1 2 3 4 5 6 7
22
INSTRUCTIONS: The following statements are related to how nuclear safety is managed; please rate the
extent to which you agree with each statement.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
We have access to the training we need 1 2 3 4 5 6 7
23
Appendix F – Behavioural Items – Optional Supplement to IAEA SCPQ-
LH*
*During the development of the IAEA SCPQ-LH, an optional set of behavioural items were
developed for the evaluation of the questionnaire. License Holders using the SCPQ-LH have reported
these as useful, and they are therefore included as optional items that can be added to the
questionnaire in Appendix E.
INSTRUCTIONS: The following statements ask your involvement in safety. Please circle the number that
most closely represents your work experience.
Somewhat
Somewhat
Disagree
Disagree
Disagree
Strongly
Strongly
Neutral
Know
Don’t
Agree
Agree
Agree
I promote the safety program within the organization 1 2 3 4 5 6 7
24
INSTRUCTIONS: Review each statement and indicate the extent to which you perform the following
activities.
A B C D F
25
Appendix G – IAEA SCPQ-LH Advisory Group
26