Professional Documents
Culture Documents
Employee Survey Methodology
Employee Survey Methodology
If the answer to the first question is yes and the answer to the second question is no, you need to
conduct an employee satisfaction survey, even if you have only a few employees and gather the
information through informal discussions.
The urgency of the matter is another thing entirely. The need to survey is greater when one or
more of the following factors are present.
At first glance, this may look like a daunting process, but it isn't. It actually takes only a few
weeks from start to finish. Committees, however, can slow the process considerably, and in some
cases result in poor decisions.
It is common for one farsighted person in an organization to realize the need for an employee
satisfaction survey. The task then becomes convincing critical decision makers of the need for
such a survey. If the survey is unbudgeted, the task may seem formidable - but it is not.
In the absence of information about how employees view the workplace, decisions affecting
productivity, morale, turnover, pay, and benefits costs are made in an information vacuum.
Money and resources directed toward items that don't need improvement (or, worse yet, result in
decline) in these items are essentially wasted. If a company were to focus resources on
improving a particular benefit when the real need employees have is more training and better
communication, the end result would be a more costly benefits program than necessary and
continued employee frustration.
One client of ours reduced turnover in the organization from more than 50% to less than 30%
after acting on the survey results we provided. Their estimated annual cost savings are $2 million
per year. Their investment? Approximately $10,000. The survey paid for itself 200 times over.
Estimates of how much is saved by reducing turnover vary depending upon the amount of
training needed for someone to reach full productivity. Organizations with more highly-skilled
workforces will realize the greatest cost savings and increases in productivity. Harvard Business
Review reports that a 5% decrease in retention results in a 10% decline in costs and a
productivity boost of between 25% and 65%.
The above represent the "hard" benefits. There also are "soft" benefits, which are somewhat
more difficult to quantify. Employees appreciate being asked for their opinions, and appreciate it
all the more when survey results are acted upon. When changes are made that make employees
feel better about coming to work, several benefits, aside from decreased turnover, can result.
These include:
Advances in technology have made it possible for management to act on survey results in a very
timely manner. It is common for our clients to have survey results, (including an analysis of
strengths and weaknesses, and recommendations), in hand within four weeks after contacting us.
This is a much simpler process than it used to be just a few years ago. The old model was to have
a consultant visit your physical location, interview employees for several days or weeks, and
create a questionnaire. This approach had many drawbacks; among them:
Our approach both simplifies and improves this old process. We developed a base questionnaire
containing core questions necessary for substantially all employers; questions addressing issues
such as supervisor fairness, corporate communication, and physical work environment. Starting
with this core document, we customize the questionnaire to fit it to the particular needs of the
client, adding and deleting items as appropriate. Periodically, we review recent projects to see
whether any of the custom items merit inclusion into the base questionnaire and whether any of
the standard questions can be dropped (based on low importance, lack of differentiation between
scores, etc.). This enables all clients to benefit from the creativity that has gone into other
questionnaires, and makes possible the establishment of normative results against which to
benchmark individual organizational results.
To determine what customization is required, we ask questions of the key client contact (and of
others, as the contact deems necessary) during telephone conversations. These questions include:
We then craft a first draft of the questionnaire for the client's review. We exchange drafts via
email until the client is happy with the document.
Some clients bring question wordings of their own to the survey process. We review these and
suggest any wording changes we feel are appropriate.
The entire process usually takes a few days. It can take just a day or two for an organization with
200 or fewer employees, especially if the CEO is directly involved. It can take longer if the
organization is either very large, has a culture of risk aversion, is highly political, or if a
committee is involved in the process.
Larger organizations can expedite the process if they have a comprehensive list of departments
and employee counts readily available. Much of the effort in questionnaire design when a few
thousand or more people are employed by an organization goes into the layout of the department
question, making sure there are no ambiguous department descriptions and that all departments
are included.
The process, though simplified from the olden days, may sound daunting. However, it actually
works quite well, and our clients always are pleased with the resulting questionnaire, the speed
with which it is assembled, and with its ability to uncover the key issues in need of improvement
There are two primary methodologies for conduction employee satisfaction surveys -- Internet
and paper-and-pencil. The choice of methodology should be based on what is best for the
respondents -- best in terms of convenience, ease of use, trust in the method. There is little
difference in price between either method until the sample size numbers in the thousands, in
which case Internet is more cost effective.
If all employees, or substantially all employees have access to the Internet from work, Internet is
our recommended methodology. It generally results in a higher response rate and is faster than
paper-and-pencil. It also give a bit more control in terms of how the questions are presented and
gives the ability to require that certain questions must be answered. We have a secure server, so
there is no issue with regard to confidentiality of sending results when an Internet survey is done
with us.
If only a small percentage of employees have Internet access from work, we recommend using
the traditional paper-and-pencil methodology. Even if cost savings could be realized by forcing
employees to a central location to complete the survey, it is better to use paper-and-pencil in this
circumstance to avoid user errors caused by a lack of familiarity with the Internet medium on the
part of some users.
In some cases, it is appropriate to mix methodologies. This is the preferred approach when there
are large contingents of employees with Internet access and large contingents without. The
mixed method generally will yield a higher response rate than will the paper-and-pencil method.
One of the advantages of conducting a survey via the Internet is that you can require that certain
questions be answered. There is an inclination upon the part of some clients to, when they realize
this is possible, want to make every question required. However, we don't recommend this.
There are a number of reasons employees do not answer particular questions. These include:
If employees don't feel qualified to answer, their answers may well be uniformed guesses. Such
guesses only serve to muddy the data, making insightful analysis more difficult.
If employees are afraid to answer, but are required to do so, they may intentionally select an
answer which does not reflect their real feelings. This will result in findings that do not represent
the true feelings of a subset of the subject population.
If employees are apathetic about the issue being raised, they may answer randomly if an answer
is required. This also serves to muddy the data.
The fourth issue to be addressed is that of employees who forget to answer a particular question.
This rarely occurs with regard to the ratings scale questions in an employee satisfaction survey.
The lion's share of scale questions are answered by 97% to 100% of the employee base. The
main exception to this is the overall satisfaction question, which 5% to 10% of employees
typically forget to answer if not so required. It is probable that most leave it blank until they have
completed the remainder of the questionnaire, meaning to review their responses before reaching
a decision on the question, and they simply forget to go back and complete it. Consequently, we
always require the overall satisfaction question in an online employee satisfaction questionnaire.
Another question employees often do not answer is the one in which they are asked to indicate
their department. This is not a case of forgetfulness - it is a conscious effort upon the part of
some employees to avoid being identified. We, upon consulting with the client, sometimes
require that this question be answered. When we do, we recommend that the client remove as
many of the other demographic questions as possible to mitigate the fears employees have that
they will be identified.
The remaining demographic questions often are not completed by many (10% or so) employees.
We usually do not require that these questions be answered. (In fact, we cannot recall making
them required in any survey.) It is more important to know the department in which an employee
resides than it is to know their age, sex, income, etc.
We have not yet discussed one other possible impact of requiring that questions be answered -
that of respondent frustration. When employees are repeatedly sent back to the questionnaire to
complete answers, some will get upset. Some may respond randomly, and some may just give
up. To minimize respondent frustration, whenever a question is required, we state this in bold red
type and remind employees of which questions are required in text immediately preceding the
"submit" button.
In the first part of this stage, both we and the client proof the questionnaire (whether paper or
Internet). This is not just a check for misspelled words - we are looking for proper skip patterns,
question numbering, grammar, and format.
Once proofing has been completed for a paper-and-pencil survey, the questionnaire goes straight
to the printer. Internet Surveys, however, still need to be programmed and tested.
After programming, we conduct a series of tests to ensure that the questionnaire is working
properly and that it is populating the correct file. We like to allow one day for this.
Once the questionnaire has been completely tested, the next step is - MORE TESTING.
One of the biggest potential GOTCHAS for an Internet survey is the email invitation. In most
cases, the employer (our client) sends the email invitation to employees. It is critical that the link
properly work. Just because the link seems to have the proper URL, it is no guarantee that it will
link to the correct URL. There could be a very minor typo, or there could be underlying code that
does not match the wording of the URL. THE ONLY WAY TO MAKE CERTAIN THAT THE
PROPER URL IS BEING REFERENCED IS TO ACTUALLY SEND THE EMAIL TO
SOMEONE AND CLICK ON THE LINK. We like to review each invitation email a client plans
on sending before it is sent.
If changes are made to an Internet questionnaire (other than typographical errors and minor
cosmetic adjustments), the survey will have to be retested.
Your employee satisfaction survey should be treated as seriously as would any of your products
or services. It must be of high quality, and it must be marketed correctly. This tip discusses the
marketing aspect of the process.
As in every marketing campaign, several things are necessary. Among these are:
Before you even begin, you will need to have appointed someone to manage the communication
process. This often is the project manager for the entire program, but it can be a different
individual.
There really are two target audiences for your Employee Satisfaction Survey Marketing
Campaign.
• Employees,
• Managers of employees.
The reason for employees being a target is self-evident. Managers also are a necessary target --
their support is essential in the communication and facilitation of the employee satisfaction
survey program. A manager can make or break response rate in his/her area.
Creating Awareness
Two to three weeks before the survey invitation goes out, you should begin creating awareness
of it.
Step 1. Communicate to Senior Managers the importance of participation, the confidential nature
of the survey, and that it is a priority for the top manager(s) of the company.
All managers should schedule meetings with the employees immediately reporting to them to
discuss the survey process.
In all pre-survey communications, including meetings with employees, the following should be
addressed.
• The importance of responding. "We can't fix things if you don't tell us what is broken."
• The confidentiality and anonymity of the process and the data.
• How the results will be disseminated to employees.
• How the information will be used.
The actual invitation, which can be in email, letter, or memorandum form, should reiterate all of
the items in these last two sections, as well as outlining the start and end dates of the survey. The
invitation should come from a person high in management, the CEO or person in charge of the
local operation, if possible. Details of the invitation are included
• Market the employee satisfaction survey to the employees and managers before the
survey begins.
• Make it clear that the survey can be completed on company time, and make sure all
supervisors support this. (We have seen complaints in employee comments when this is
promised but one or more supervisors do not support it.)
• Make sure employees understand their responses will be confidential. Support your
statements to the best extent possible, e.g., "Your questionnaires will be received directly
by the research firm doing the tabulations. Nobody in our organization will see your
ratings and comments."
• Let employees know how many people have responded at various points in time.
• Establish an overall response rate goal and give all employees "something extra" if the
goal is reached.
• Send reminder messages.
• When the results come in, let the employees know what they were. This will help the next
survey's response rate.
• Act on the results. This also will help the next survey's response rate.
In the value of normative data in employee survey analysis, we discussed how norms add
significant value to the interpretation of results. The question often comes up - "against what
types of organizations should I compare my organization?" For most of our clients, the entire
normative database is the most appropriate base to use for comparison. The reasons for this are
as follows.
• In many industries, employees jump between industries. Thus, you compete for
employees with industries outside your own. As such, you need to know how your
organization compares with the real competition for your "people" resources.
• In most cases, the differences between industries are slight in terms of how employees
feel about where they work. For example, Table 1 contains the scores for one attribute,
"my supervisor treats me fairly," for three industries and for the database as a whole.
• Using too small a small subset of a normative database makes it more difficult to
determine
whether Many People Find the Small Differences Between Industries Surprising
observed Table 1
differences "My supervisor treats me fairly"
to the norm
are Net
statistically Industry Percentage
significant. Agreeing
This causes Manufacturing 76%
uncertainty Financial Services 78%
with regard
High Technology 82%
to how
different All Industries in Database 80%
your
organizations results really are, and can lead to erroneous decision making if the user
ignores any lack of statistically significant differences.
As is the case in much of life, there are exceptions. People in the healthcare industry tend to stay
in the industry and the results do tend to differ slightly from those of employers as a whole. Also,
we know that employees of very small organizations tend to be happier than employees of very
large organizations.
When a client does not want to compare against the entire normative database, we try to strike a
sense of balance between using organizations substantially identical to the one for which we are
conducting the employee satisfaction survey and benefiting from the large sample sizes
associated with using the entire database. Sometimes this means eliminating organizations above
or below a certain size or organizations in particular industries from the database against which
we compare the results.
In some instances, it is appropriate to only compare against organizations in the same industry as
the client. Fortunately, we have a large enough sample to do this for many industries, including,
but not limited to, healthcare, high technology, and manufacturing.
Some of our clients have very high aspirations and levels of employee satisfaction. For such
clients, it can be appropriate to "set the bar high" by comparing against only the top 10% or 20%
of all organizations (based on overall satisfaction with an organization as a place to work).
The most important thing to remember is that norms add an extra layer of analytical capability
and value to the employee satisfaction survey process.
It is astounding how much "data" can be created by a survey. Each scale question has multiple
possible answers, each of which must be reported along with each item's average score; there are
results for various subgroups, and there is statistical analysis. The data must first be packaged
into information (via proper presentation of tables, graphs, and statistical analysis). Then, it must
be properly interpreted. Different analysts may take different approaches. Here is one approach.
We look at the statistical relationship between each attribute measured and overall
satisfaction. Items with high leverage (correlation to overall satisfaction) will have more
impact on satisfaction than will items with low leverage. By plotting the leverage
(correlation) scores and the performance scores (the percentages or average scores for
each attribute) in two-dimensional space, it becomes apparent which items need
immediate attention.
When each question's correlation and average rating are plotted on a chart, the result is a
quadrant map. Those questions that plot in the northwest corner of the map are those
areas that deserve immediate attention and company resources.
4. Comment Analysis. Simply reading comments can give one a flavor for the types of
issues on employees' minds. However, proper interpretation becomes difficult, if not
impossible, for two reasons.
o The reader may have pre-conceived notions about what the comments will say,
and may tend to give more weight to comments that mirror his/her point of view.
o The shear number of comments can be overwhelming, as scores, hundreds, or
thousands of pages of comments may exist.
The solution is to "code" comments. This is done by reading all or a large sample of
comments, and creating categories into which comments seem to fall. Each comment
must then be read and "coded;" that is, placed into one or more categories. At the end of
the day, this yields a table of results showing how prevalent are comments of various
types.
Usually, the items identified in the leverage analysis also will show up in the comments.
However, it is not uncommon to find one or two other items rising to the top in the
comment analysis.
2. Sub-group Comparisons. In some cases, most or all subgroups will feel the same about
a matter. In others, there will be large differences in how sub-groups (e.g., departments,
people of different tenures) feel. We have seen instances where a large sub-group is
almost completely responsible for a low score for an entire company. Without an analysis
of sub-groups, corrective action may not be as effective, since targeted action will be
necessary in certain cases.
3. Summarize Findings. In this stage, the analyst wraps up what has been found. The end
result is recommendations of key items and areas to target for improvement on an overall
organization level and at the sub-group level.
4. Recommendations. At this stage, the analyst recommends actions to take to address the
items targeted for improvement. Further discussions with the client, before and after this
phase, often are necessary.
1. Everyone must know where the organization as a whole and their individual areas stand if
you are going to fully leverage the creativity and efforts of the employee base.
2. Employees need to know that the time they spent in completing the survey was
worthwhile.
Each organization has its own information-sharing culture in place. While in some cases,
particularly if the survey showed communication to be a problem, the process will need some
adjustment, we recognize that each organization will have an approach to information
dissemination that it typically leverages. As such, modifications to our recommended approach
may be in order to account for an organization's information-sharing culture.
1. Be honest. An organization must be willing to share both its strengths and its areas in
need of improvement. Employees will see through attempts to hide or "spin" information.
2. Be timely. The sooner you release results, the sooner the organization can begin to move
toward positive change.
3. Share appropriate information at each level. Senior management will need
encapsulated results and access to detailed results for the organization as a whole and
differences between divisions/departments. Division managers will need to know how
their division compares to the organization as a whole and how departments in the
division compare to each other. Department managers will need to know how their results
compare to the organization as a whole and to the division to which they belong.
4. Don't embarrass people in front of their peers. Teamwork and morale can be harmed
if, for example, a division manager shows all department scores to the department
managers in a group setting. Rather than pointing out low-scoring departments to all
department managers, let all department managers know how they fared compared to
other departments via one-on-one meetings.
5. Discuss what happens next. After the results have bee presented, let the audience know
what steps will be taken to improve those items in need of improvement.
6. Respect confidentiality. Don't present information that would make people feel that their
responses are not confidential. For example, it would not be appropriate for anyone in the
organization to have access to comments for a small department, since some people may
be able to accurately guess who made what comment. Your research supplier should
assist in this by not providing information that could breach, or could be perceived to
breach, confidentiality.
Process Considerations
1. Have a plan in place to disseminate information before the survey has been completed.
2. The CEO/president should be briefed by the internal project manager and/or the research
supplier.
3. The CEO/president should share the results with division managers. Overall results
should be shared in a group setting. Individual results should be shared in one-on-one
meetings.
4. Division managers should share results with department managers in a similar manner to
the approach the CEO/president took with them.
5. Key findings and implications should be highlighted in each presentation. Detailed results
also should be presented. However, take care to avoid drowning people in information.
This can be done by relying more heavily on graphics than on detailed tables to
communicate.
6. Give employees an overview of overall results through the best means possible. For some
organizations, this will be in a group setting. For others, it will be via email, Intranet, or
newsletter. Consider using multiple methods.
7. Department managers should share departmental results with employees in a group
meeting. It may be helpful to have an HR manager assist in the presentation. If HR
managers will be part of this process, planning ahead will help the meetings to proceed
smoothly and take place in a timely manner.
8. In all communications, make sure the communication is "two way." Questions should be
encouraged
Your company has committed the resources to survey its employees, the employees were
enthusiastic about participating and now you have the survey results in your hand. What do you
do with the survey data? What are the next steps in improving the satisfaction of your
employees? How should the results be distributed? How do you identify the most important areas
to focus on? Once the areas are identified, what's the most effective method of realizing
improvements in those areas?
While the answers to many of these questions are largely the product of a company's
organizational culture, it's crucial that the employees be actively involved in the post survey
phase if an organization is to attain significant improvements in employee satisfaction with
minimal resources.
From the beginning, senior executives must determine who gets what type of report. Should
department managers receive reports from other departments? Are divisional reports needed and
who should receive them? Should supervisors receive specific reports and line managers receive
reports? Who should review employee comments?
Once the survey results are distributed, an action plan must be created to decide which areas are
most important to the employees and how best to improve those areas.
One of the biggest mistakes organizations can make in the survey process is to fail to
communicate to everyone the findings of the survey and the plans created to improve weak areas.
While we do not suggest distribution of the complete survey report to everyone, we do
recommend an open and honest discourse of both organizational strengths and areas for
improvement at the department level if possible. Along with the survey findings, dialogue of
what action the company plans to take to address those organizational weaknesses also must
occur.
Admittedly, the method of company communication and the content of those messages are again,
influenced by the company's culture. Failing to reveal the results of the survey and to take action
to correct shortcomings can prove detrimental to company satisfaction. In fact, companies are
better off not surveying at all if they don't plan to take action. Inaction by company decision
makers results in an almost immediate decline of employee morale and a general feeling of
apathy. Furthermore, failing to act can severely impact future attempts to gauge the satisfaction
of your employees in the future. It's been our experience, however, that those organizations that
solicit input from its employees both in terms of deciding which areas to focus on and creating
novel approaches to deal with those areas are the ones that achieve the greatest amount of
improvement in the shortest amount of time with minimal resources.
Once the survey results have been distributed and critical areas identified, organizations must
decide upon the most effective means to improve those items. Some executives charge the
individual department managers with creating and implementing improvement processes. Other
more culturally dynamic organizations involve employee in the process by making them part of
task forces and focus groups to better understand and act upon the survey findings. Effective use
of employee groups creates new levels of employee trust and organizational commitment. In
addition, focus groups facilitate the implementation of meaningful changes, which have a real
impact on employee satisfaction. A recent client in the pharmaceutical industry which surveys its
employees semi-annually has realized valuable gains in employee satisfaction using focus
groups, decreasing employee turnover by nearly 50% in a 2-year period. Whichever method is
used, there are three keys to success:
We often hear the question, "how often should I conduct an employee satisfaction survey?" Our
recommendation is once a year for most organizations.
Depending upon methodology and the size of the organization being measured, it usually takes
three to four weeks from the implementation of a survey until the issuance of the final report and
departmental reports. (It can be done more quickly, especially if the Internet is used and the
organizational culture encourages rapid response.) If a good post-survey communication plan is
in place, results should be disseminated throughout the organization in approximately two weeks.
The time it takes to design action plans will vary from organization to organization, but two to
four weeks following the dissemination of results throughout the organization is a good target.
This means that an organization can begin to make progress toward improving the items found in
need of improvement within two to three months following the commission of the survey. This
leaves at least nine months for improvement to take hold.
Nine months is a sufficient time for improvement to be seen. If the efforts are effective, a survey
one year after the prior survey will show that efforts to improve have been successful. If the
efforts are not effective, conducting another one year after the previous one will give the
organization a chance to alter course before waiting two years to do so.
There are instances when an organization might choose to repeat at time intervals other than one
year. If an organization's culture is such that results can be disseminated and acted upon more
quickly than the schedule outlined above, a six-month schedule can be considered. However, this
is too quick for most organizations, as it does not give sufficient time for improvement before the
next measurement. It would be good for us monetarily if all clients went to a six-month
timeframe, but we do not recommend this in the lion's share of circumstances because we feel
most clients will not receive sufficient value from such an aggressive schedule.
• It is such a long length of time that an organization could actually improve in targeted
areas, and then relapse. The subsequent survey then might show no change, when in fact
two changes had taken place -- one positive and one negative. In such a circumstance, the
erroneous conclusion might be that the efforts to change had no impact.
• It can be difficult for an organization to maintain focus for two years. Other competing
priorities will inevitably arise, and it will be easy for some people to shelve, at least
mentally, employee satisfaction improvement programs when the next measurement is in
the distant future.
• People move around. In two years you might expect to have many new people and many
people in new positions. The people responsible for implementing change may no longer
be in their old positions. The new people may be unaware of how they should be
contributing to the organization's planned changes which resulted from the previous
employee satisfaction survey.
We recognize that, even though employee satisfaction surveys can pay for themselves, it can be
difficult for some organizations to set aside the funds needed to complete a program annually.
Our recommendation in such circumstances is to consider a "pulsed" approach, where in the first
year and every other report the organization goes with our (or the equivalent of our) Gold
program (with full written analysis), and in the off years the organization goes with a Silver
program (where there is no written analysis, but there are complete charts and graphs at all
appropriate levels). This saves money, while still enabling an organization to reap the benefits
that come with an annual employee satisfaction measurement program.