How many people need to be surveyed?

A primer on selecting an appropriate sample size.

Cheryl A. Boglarsky, Ph.D.

1
Quite often we get the question, “What is a
significant sample size for my
organization?” In the past we would
recommend at least 40% of the
organization’s employees be surveyed to
get valid results. However this rule of
thumb is no longer being accepted by
clients – they want to know why this is
appropriate. As you can imagine, this is
not a simple question to answer.
As always, there is a need for clarification
of terms. First we must make a distinction
between response rate and sample size.
Response Rate
Response rate can be defined as the
number of respondents who complete a
questionnaire compared to the number of
questionnaires assigned (this is usually
expressed as a percentage). For example:
Acme Company buys enough OCIs to
survey 500 of its members. After two
weeks (and many reminders) 368 OCIs are
mailed back to HSI. Based on Acme
timelines, our scoring department decides
to close the collection phase of the project
and begins entering the data. At this time,
the response rate is 74% – that is, divide
368 by 500 and then multiply by 100.
The next question is usually, "What
response rates can we expect from a
successful culture study?" Of course, the
question has no general answer. For some
studies, in some industries, a 2% response
rate is about as good as you can expect,
and may be highly informative. In other
studies, with other topics, a 90% response
rate can be achieved.
All in all, a well accepted, satisfactory
response rate for organizational surveys is
70%. Therefore, Acme Company’s
response rate can be viewed as acceptable.
2
See Appendix A for strategies for
increasing response rate.
Well, that answers a few questions.
However…how did the original number of
500 come about?
Sampling and Sample Size
Basically, sampling can be defined as the
process of selecting units (e.g., people)
from a population of interest so that by
studying the sample we may fairly
generalize our results back to the
population from which they were chosen.
Determining how large a sample is needed
requires several steps:
1) Define your population;
2) Determine how sure you want to be in
your “guesstimate;”
3) Determine your sampling method; and
4) Draw your sample.
Define your population
The group you wish to generalize to is
called the population in your study. This is
the group from which you will choose a
sample. In the Acme Company, the
population is all of the employees –
including the leaders (n=16), managers
(n=250) and non-managers (n=1234). This
would mean that the population of Acme
Company is n=1500.
Determine how sure you want to be in your
“guesstimate
The key is to sample just enough people to
assure confidence in the results, but no
more. For most survey work in the social
sciences, a frequently used standard of
confidence is expressed this way: "a
margin of error of less than 5 percentage
points at a 95-percent confidence level."
This means that if 100 different random
samples of that certain size are drawn from
the same population, in 95 of those
samples (95 percent of them), the results
will not vary more than five percentage
points, higher or lower, from the results
which would have been obtained if all
respondents in the study population had
been surveyed.
What is meant by "vary no more than five
percentage points?" Since we can never be
sure that a single sample's results (e.g., 58
percent in favor) will match the results we
would get if all employees were surveyed,
we have to be cautious and say something
like, "we are 95-percent sure that the true
percentage of people who feel this way is
between 53 percent and 63 percent," or
five percentage points on either side of the
sample's 58 percent.
When this is applied to OCI results
(Humanistic-Encouraging average raw
score = 35), we can say something like,
“we are 95% sure that the Humanistic-
Encouraging raw score truly expresses the
how the entire population of Acme
Company feels is between 33.25 and
36.75,” or 35 minus 5% and 35 plus 5%.
Determine your sampling method
There are two general ways to choose the
group of people, from your population, to
be in your study – namely simple random
sampling and stratified random sampling.
A probability sampling method is any
method of sampling that uses some form of
random selection. In order to have a
random selection method, you must make
sure that each member of your population
3
has an equal chance of being chosen to
take the survey. There are many ways to
do this – from picking a name out of a hat,
or choosing the short straw. However we
tend to use computers to generate random
numbers and assign them to individuals as
the basis for random selection. Because
this simple random sampling is a fair way
to select a sample, it is reasonable to
generalize the results from the sample back
to the population.
But what happens if you have several
groups (e.g., departments or locations) that
need to be surveyed? This would entail
stratified random sampling, which
involves dividing your population into
subgroups and then taking a simple
random sample in each subgroup.
There are several reasons why you might
prefer stratified sampling over simple
random sampling:
1) it assures that you will be able to
represent not only the overall
population, but also key subgroups
of the population, especially small
groups.
2) If you want to be able to talk about
subgroups, this may be the only
way to effectively assure you'll be
able to.
3) There will generally be more
statistical precision than simple
random sampling (especially if the
groups are similar on important
variables).
Draw your sample
Let’s review:
1. Population defined – you have
defined the group to which you
would like your results to
generalize;
 2. Standard set – you determined an
acceptable confidence level and
confidence interval; and
3. Sampling method chosen – you
chose between a simple random
sample and a stratified random
sample.
Now, how large a sample do you need
from your population to achieve the
standards that you set?
Well, once you establish the conditions for
your investigation (e.g., population,
confidence level and interval, and
sampling method) – the rest is quite
straightforward.
Step 1. Collect the names and contact
information of ALL members of your
population.
Step 2. If you chose simple random
sampling all you need to do is place the
names in a spreadsheet program (such as
Microsoft Excel). If you chose stratified
random sampling you will need several
separate spreadsheets – one for each group.
Place the names in the appropriate group.
Step 3. To determine the number of people
needed to secure a specified confidence
interval, a relatively complicated formula
is used. But instead of giving a formula
and then having you derive an answer, we
included Table 1 for your convenience.
Table 1 lists two confidence levels (95%
and 99%) and three confidence intervals
(1, 3 and 5 percents) and associated sample
sizes for several populations. Use this table
to determine the number of people needed
to survey.
Step 4. Now that you know how many
people you need, how do you decide which
people to survey? Remember, every person
4
should have the same chance of being
chosen. The best way to do this is to assign
each person a random number. Your
spreadsheet program has this capability.1
Step 5. Decide how you are going to use
the random numbers. That is, once each
person has been assigned a number (by the
spreadsheet program) – what decision rule
are you going to use to choose people?
Historically I have used the random
numbers to order the people in the
population and start with the lowest
random number. From this list I choose the
appropriate number of people starting with
the lowest number.
For example, let’s say that we are
interested in Acme Company’s managers,
of which there are 250. We choose a 95%
confidence level and a 5% confidence
interval. Looking at Table 1 we see that the
appropriate sample size is 124. So now we
go to the spreadsheet that has all of the
managers’ names. In a separate column we
use the program to generate a random
number for each manager. Once that is
done we sort the spreadsheet by random
number (lowest to highest). From this list
we choose the 124 names with the lowest
random number. These names will be in
our sample.
Now … run your survey
Now that you have determined an
appropriate sample size, you are confident
that as long as you get the number of
people you can appropriately generalize.
However there is one requirement: The
people you choose through random
sampling must be the only people who
1
For example, in Microsoft Excel it is under the
Tools – Data Analysis – Random Number
Generation command.
answer your survey. It’s not good enough
to just get a certain number of people – you
must survey the specific people you
randomly chose.
Let’s use the previous example – surveying
1500 employees of the Acme Company.
Using stratified random sampling and
choosing a 95% confidence level and 5%
confidence interval we determine that the
following sample is appropriate:
Population Sample2 Acceptable
Response
Rate3
Leaders 16 15 11
Managers 250 124 87
Non- 1234 205 144
Managers
Total 1500 344 242
After listing the employees and assigning a
random number, we choose the appropriate
number of people from each group.
Sounds easy, right? It is, and it’s
statistically sound. However there are
some questions that may arise while
performing the assessment.
1. Since there are only 16 Leaders,
why just survey 15 of them? Why
don’t I just survey all 16? You
must sample only the amount
specified by the equation because
you don’t want to give too much
weight to one group. If you
oversample one group (sample over
the statistically determined
number), your results will not
represent the entire group.
2. I know some of the people who
were chosen (and not chosen).
Some of the people chosen to be
surveyed don’t want to answer and
some not chosen want to answer.
Can I switch respondents? No, you
must survey only those who were
chosen by the random number
assignment. If you do it the way
suggested you will not have a fair
representation of the entire
population – you will only get the
answers of people who like filling
out surveys.
3. OK—I gave the surveys to the
people chosen by random
sampling. However all of these
people didn’t return the surveys!
Can I now give the surveys to some
people who want to fill them out?
No, only the people who were
chosen should be sampled. This is
where we address response rate.
Remember, a 70% response rate is
acceptable in organizational
surveys – so if you get 70% for
each group it is considered
acceptable.
Granted, there will be more questions and
concerns – but if you follow the guidelines
you will have a representative sample and
be very sure of your results.

2
Derived from the formula in Table 1.
3
Based on 70% response rate.

5
Table 1. Confidence levels, confidence intervals and associated sample sizes .

Confidence level 95% 99%

Confidence ±1% ±3% ±5% ±1% ±3% ±5%
interval
Population Size
25 25 24 23 25 25 24
50 50 47 42 50 48 45
100 98 87 71 99 92 81
250 240 183 124 244 202 158
500 462 289 165 478 351 230
750 669 358 185 701 459 271
1000 860 406 197 914 541 298
1250 1039 442 206 1118 607 317
1500 1206 470 211 1314 661 331
1750 1362 491 216 1502 705 342
2000 1509 509 219 1683 742 350
3500 2230 572 230 2632 883 379

The formula used to derive the sample size values:

n = M / ((((E / ZCL)2 * (M - 1)) / (pq))) + 1)

Where:

M = size of the population to be sampled

E = maximum margin of error we choose (expressed as a percentage).

ZCL = Confidence level expressed as a Z score

ZCL = 1.96 for a 95% confidence level

ZCL = 2.576 for a 99% confidence level

p = probability of choosing a specific answer – expressed as a percentage.

q = probability of not choosing a specific answer– expressed as a percentage.

NOTE: Since the OCI uses a response scale with five choices (e.g., 1: Not at all through
5: To a very great extent), the value of p will always be 20 (20% or .20 probability of
choosing one of the five response options) and the value of q will always be 80 (80%
or .80 probability of choosing one of the other four response options).

6
Appendix A: Tips to increase o Provide both web-based and
response rate e-mail options where possible
o Be aggressive about tracking
 Ensure that the survey topics and responses and sending reminders
questions are of interest, concern and
value to the participants.
 Produce a professional looking
 Develop a value proposition for questionnaire.
participants  Provide incentives - sweepstakes,
 Communicate ahead of time. etc.
Alert everyone that the survey is  Follow-up communications after
coming. Have managers questionnaire is distributed (memo,
communicate with their people, phone, e-mail, voice-mail,
explain the value and importance of newsletters, etc.) to stimulate
the survey and encourage their participation.
participation.  Communicate feedback on survey
 Make participation voluntary, findings -- meetings, articles, videos,
anonymous and confidential. etc.
 Use a cover letter signed by the
highest ranking executive possible
which explains:
 Purpose of survey and the
value to the participant
 How the survey
information will be used
 How to return the
completed questionnaire
 Assure confidentiality or anonymity

For paper and pencil questionnaires -

Make it easy to return:

 Postage paid, pre-addressed

return envelope

 Fax back questionnaire

 800# telephone touch-tone
responses
 e-mail questionnaire with auto-
programmed return

For e-Surveys:

o Keep it simple and user-

friendly