Radiography (2005) 11, 131e136

REVIEW ARTICLE

The purpose, design and administration of

a questionnaire for data collection
Gill Marshall *

Division of Medical Imaging Sciences, St. Martin’s College, Lancaster LA1 3JD, UK

Accepted 20 September 2004

Available online 1 December 2004

KEYWORDS Abstract This paper considers the use of questionnaires as a data collection tool.
Reliability; A range of issues will be discussed, including whether previously validated
Validity; questionnaires already exist for utilisation, the advantages and disadvantages of
Piloting; employing questionnaires, optimum design features including the wording and
Approval; language used and the ordering of questions, distribution methods, ways of
Question types maximizing response rates and managing non-responses, reliability and validity, the
benefits of piloting the questionnaire and approval before the questionnaire is
used. Questionnaire-based research is increasing evident within the NHS and
elsewhere and from this presentation it is clear that a range of issues must be taken
into account so that collected data can be gathered effectively in a format that can
be appropriately used.
ª 2004 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.

Introduction own as the sole research instrument or in associ-

This paper aims to explain the purpose, design and
administration of questionnaires. A questionnaire
is a method of data collection which is completed
by the respondent in written format.1 It is a pop-
ular research instrument, but designing a question-
naire to yield high quality usable data and
achieving a high return rate is not as easy as it
may, at first, appear.2 Indeed this needs careful
planning and design3 and an effective method of
distribution. Questionnaires can be used on their
* Tel.: C44 1524 384639; fax: C44 1524 384591.
E-mail address: g.marshall@ucsm.ac.uk
ation with other research tools2 e.g. objective
measurements of image quality may be taken from
radiographs after the injection of a drug, as well as
asking patients about the side effects of the drug
via a questionnaire. A questionnaire is a cost
effective way to collect data, from large numbers
of the population.4 However, before opting for
a questionnaire the research needs to be planned.
The researcher must consult with the literature
and experts in the field to decide what they need
to know e.g. what are the research questions? Only
then can it be decided whether a questionnaire is
the most appropriate tool, which will provide the
information required.5 If in the planning and

1078-8174/$ - see front matter ª 2004 The College of Radiographers. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.radi.2004.09.002
132
reviewing the literature phase it is found that little
is known about a particular area, a qualitative
approach is best used first e.g. a focus group
(where respondents are assembled as a group to
answer questions on the research area of inter-
est).1 Using the information gained it may then be
appropriate to develop a questionnaire.2
Use of validated questionnaires
An important factor in searching the literature is to
establish whether there are already validated
questionnaires that explore the research topic.2 If
so, they should be used, as a questionnaire de-
veloped by a novice researcher is unlikely to have
the reliability and validity of one that has been
rigorously tested during the design phase. The use
of a validated questionnaire will save time and
resources, allow comparison with data derived
from other studies and it may be easier to get the
results published.2 Books and articles6e11 contain
a wide range of validated scales pertaining to
measuring health status, satisfaction with services
and quality of life in health care settings. Another
advantage of using validated scales is that
some, e.g. Speilberger’s State and Trait Anxiety
Inventory,12 give rise to interval or ratio data, the
strongest data achieved by the use of a calibrated,
validated scale, facilitating a robust parametric
analysis.
Advantages of questionnaires
With careful planning, questionnaires can yield
high quality usable data, achieve good response
rates3 and provide anonymity, the latter encour-
aging more honest and frank answers, than for
example interviews. This can help to reduce bias.
A questionnaire is a useful data collection tool
where the following conditions are met:
 the target audience, even if geographically
spread, can be clearly defined and identified,
 the majority of respondents know what is
asked of them,
 the focus of the analysis is numerical i.e. the
questionnaire yields quantitative data.4
Disadvantages of questionnaires
Questionnaires, as other data collection tools,
have certain disadvantages: they are unsuitable
G. Marshall
for subjects with poor literacy, visual impairment
and non-English speakers unless translated; the
researcher generally has no idea if the question-
naire was filled in by the respondent it was meant
for; if there are confusions caused by the ques-
tionnaire the researcher cannot clarify these;
there is little flexibility for respondents to present
their own perspective on issues unless there are
several open questions. Questionnaires can give
rise to very poor response rates,5,13 although care-
ful planning, design and administration can in-
crease this.3,4 Without careful planning regarding
the administration of the questionnaire, there can
be recruitment bias. For example if the views of
patients attending for MRI were sought, a notice
could be put up in the department asking inter-
ested patients to ask for a questionnaire at the
desk. The respondents, however, would only be
highly motivated attenders and it would have been
more representative to issue a questionnaire to
every user or every tenth user on a particular day.14
Question types
Questions should be short and focussed, consisting
generally of 12 words or less. However, when
addressing sensitive or personal issues, longer less
abrupt and threatening sentences are preferable.2
The more structured the questions are, the easier
they are for the researcher to interpret,5 as the
data produced will be quantitative i.e. information
that is quantified and thus numerical.1 To yield
qualitative data i.e. non-numerical observations
and narrative data1 a high proportion of open
questions must be asked. A range of question types
exists and these are summarised below:
 Open questions are those where the expected
response is in words. These questions should
only be asked if the researcher is seeking
narrative, qualitative information15 e.g. if your
decision not to attend for breast screening is
not listed please give your reasons. Some sort
of content analysis is required for the re-
sponses.5 This involves analysing the responses
in a systematic and objective fashion, usually
to convert the information into data that can
be quantitatively measured.1
 Closed questions are where a respondent is
offered a choice of alternative replies e.g. did
you travel to the hospital by car, taxi, bus,
other. They are of various types.5,13
 Quantity where the response is a number e.g.
how many times have you been scanned during
this pregnancy?
Questionnaires e design, use, administration
 List where the respondent can select more
than one response, as the responses are not
mutually exclusive e.g. what medical services
have you accessed over the last six months?
Here the respondents can select as many of the
responses as are appropriate from a list.
 Category questions where there is a list of
mutually exclusive categories e.g. regarding
the respondent’s child bearing history e.g. no
live births, one to three live births, more than 3
live births
 Ranking/scales. There are many of these avail-
able e.g. the Likert scale where the respond-
ents choose a ranked option from a list,
indicating their degree of agreement or dis-
agreement with a statement.1 Such scales can
produce nominal, ordinal or interval/ratio
data.5 Interval or ratio data are the strongest
data achieved by the use of a calibrated scale
e.g. number of pregnancies; ordinal data have
a clear order but not from a calibrated scale
e.g. strongly agree, agree etc. from a Likert
scale; and nominal data, the least robust data
which categorise but do not rank data e.g.
a list of radiographers, radiologists and nurses
working in a particular work area.
 Grids are where more than one dimension is
measured e.g. where you are asking the
respondent two things in one question e.g.
what types of scans they have had and how
many of each type.5 Some researchers prefer
to divide such questions into two separate
questions.4
Things that are best avoided in questionnaires
are:
 hypothetical questions4,5,13 e.g. during your
next pregnancy would you like to be scanned in
a private clinic? The value of the data produced
is questionable, as the woman may not have
another pregnancy or maybe unable to attend
a private clinic even if she wished.
 imprecise questions4,5,13 e.g. do you exercise
frequently? What is meant by exercise and
what is frequently? These terms hold different
meanings for different respondents.
 ambiguous questions4,5,13 e.g. was your last
baby, at birth, above an average weight, an
average weight, below an average weight? Will
the respondent know what is the average
weight of a baby at birth? Will this weight be
the same for all ethnic groups? Is the answer
thus meaningful?
 assuming questions4,5,13 e.g. during your preg-
nancy how many times were you scanned? How
133
will the woman answer if she has had several
pregnancies? The researcher may have as-
sumed the woman would answer for her last
pregnancy, but this may not be so.
 double questions4,5,13 e.g. have you attended
for both breast and cervical screening? This
requires two separate responses to be valuable
and should thus be presented as two questions.
 leading or loaded questions.4,5,13 These tend to
use emotive language and lead the respondent
to agree with the statement presented e.g.
Isn’t it your right, to receive a free photograph
of your baby, as part of your scan?
 presuming questions4,5,13 which often result
from the researcher having strong views on an
issue e.g. do you believe employers allow
sufficient time off for antenatal appointments?
This presumes that all employers do allow time
off and that they should. Not all employers
would agree. The responses to such questions
are biased by the question as posed.
 memory questions4,5 particularly if referring to
more than six months previously e.g. during
your last pregnancy how many times did you
see the midwife?
Wording and language
Once the questions have been written it is essen-
tial that the questionnaire is ‘‘user friendly’’ to the
respondents. Things that need to be considered
are: is the word in everyday use, is there a simpler
word/s that could be used, are any of the terms
used jargon or specialist language?4 All of these
factors can affect your response rate.4
Questionnaires should ideally not challenge
respondents knowledge, but if this is essential this
must be done in a non-threatening manner and
must include a ‘‘don’t know’’ response.4 Question-
naires should only ask things directly relevant to
the research area. Read through the questionnaire
critically on completion and ask yourself how you
would feel if you received it, and show it to
colleagues asking for their opinion.5
Order of questions
Sequencing of questions is important to increase
response rate and it is better to start with easy,
non-threatening, non-sensitive questions. Filter
questions are appropriate where it is not necessary
for all respondents to answer all the questions,13
however, ensure that instructions regarding filter
questions are explicit and easy to understand,
134
otherwise respondents can become confused. They
may also choose to answer ‘‘no’’ in order to finish
the questionnaire more quickly.16 If there are
sensitive or embarrassing questions to pose, they
are best placed towards the end of a questionnaire.
Once the respondent has invested some time in
completing the questionnaire they are less likely to
stop, than if such a question was placed early on.16
Presentation
The typeface should be considered to ensure it is an
appropriate size and style for the respondents e.g.
larger for the elderly.16 The questionnaire should
be spaced appropriately with enough room for full
responses to open questions, whilst providing an
uncrowded appearance.5 Instructions on complet-
ing the questions, including open questions17 need
to be explicit, perhaps illustrated with an example
and the instructions should be in another typeface
or font to make them stand out clearly.5
Respondents generally find A4 sheets easier to
handle than folded A3 sheets.16 The questionnaire
has to include clear return instructions including to
whom and where it has to be returned and the
permitted time scale for the response. It is better to
give an exact date for return, rather than say return
within two weeks as it is easy to forget when
something was received.5,16 If the questionnaire is
being distributed to different groups of respondents
e.g. radiographers and radiologists it may help the
researcher to colour code their responses. Other-
wise patterned or coloured paper is best avoided.4
If the questionnaire analysis will be done by
computer a space for coding the responses to the
questionnaire should be left on the right hand side
of the page.5,17 The response boxes on the ques-
tionnaire should be kept in a line towards the right
of the questionnaire.5 This benefits both the re-
spondent and researcher.
The pages of the questionnaire should be num-
bered and PTO put at the bottom of each sheet to
ensure completion of all the questionnaire. The
respondent should be thanked at the end.
Distribution
The distribution and return of the questionnaire
along with cost implications, needs to be consid-
ered. If the questionnaire is posted out, whilst this
can reach a large geographical spread, it tends to
have poor response rates.4 It can be presented
individually to respondents which tends to result in
G. Marshall
higher response rates4 or group administered but
with the latter approach there is the risk of
respondents discussing their responses and thus
contaminating them.13 It could be posted on the
internet, which it is claimed has the potential to
overcome many of the problems,18 but will never-
theless not be universally available. If there are
different distribution methods to different groups,
it must be considered whether this will bias the
results. For example, consider this situation: ques-
tionnaires are personally handed out by the re-
searcher to a group of women who have chosen to
attend breast screening and the researcher is then
on hand to answer any questions. In comparison,
questionnaires to those women who have chosen
not to attend have been sent through the post. No
further clarification is available to these women.
This could bias the results.
Non-responses
The researcher will need to decide what to do
regarding non-responses. If the questionnaires are
truly anonymous, the researcher will not know
which have not been received back, and will need
to remind the whole sample.13 If they are not truly
anonymous, the respondents must be aware of this
and the researcher will have to devise some way of
ascertaining from the returned questionnaires e.g.
via a code, those that are outstanding. Accurate
notes of when questionnaires where sent out will
have to be kept, along with records on whether
they have been received back or not. This will
allow non-responders to be reminded to return
their questionnaires or be sent a second question-
naire, if that is what it has been decided to do.
A problem with non-responders is that they
often differ from respondents which could affect
the validity of the conclusions. A non-response
rate of higher than 10% may bias the results5 so it is
worth encouraging non-responders to participate
but after two reminders have been sent, it is
generally not worth further time and effort as
the responses tend to peter out.5
Ways to increase response rate
Ways to increase the response rate should be
considered. As well as the information above about
selecting appropriate questions which are well
worded and appropriately presented, the inclusion
of a logo from a respected sponsor13 or putting the
questionnaire, provided the necessary permission
has been sought, on official headed paper will help
Questionnaires e design, use, administration
in this regard. A covering letter should be enclosed
explaining the purpose of the study, the required
role of the respondent, what approval has been
given for the study, a guarantee of the respon-
dent’s anonymity, if this is the case, or a statement
regarding to whom the data will be accessible.
There should be a contact name, address and
telephone number for queries. How the results of
the study will be made available to participants
must be made clear. It must be stated that non-
participation in the study will in no way affect the
respondent’s care. A stamped addressed envelope
enclosed with postal questionnaires will promote
their return as will small incentives, if it is possible
to provide these. This approach is common with
market researchers who will e.g. enter respon-
dent’s details in a prize draw.4
Reliability and validity
Piloting4 must occur before the questionnaire is
administered to the research sample to ensure the
reliability and validity of the questionnaire.3,4
Reliability is the degree of consistency or de-
pendability with which the instrument measures
the attribute it is designed to measure, so that
differences in results come from differences in
respondents, not differences in how the question-
naire was understood.2 Internal consistency, is
a form of reliability, referring to the degree to
which subparts of the research instrument, in this
case the questionnaire, are all measuring the same
attribute or dimension.1 If a researcher places two
questions in a questionnaire, which in effect are
asking the same thing, and the respondent answers
differently the value of that questionnaire is
clearly dubious.
Validity is the degree to which the instrument
measures what it is intended to measure1,19 e.g. if
a patient is asked to self complete the units of
alcohol they drink, the researcher measures what
they say they have drunk, which may differ from
what they actually drink.2 An unreliable question-
naire cannot be valid; however, a reliable ques-
tionnaire can be invalid.20 The extent to which the
sample size represents the population is a factor in
assessing the validity of a study i.e. the extent
with which the results can be generalised to other
samples or situations.1,21
Piloting
The piloting stage is essential to check the re-
liability and validity of the questionnaire,17 to
135
remove any flaws e.g. wording or instructions that
have caused confusion, to check that the data are
usable via analysis and to allow the researcher to
then redraft the questionnaire. It can be useful for
the researcher to be present when the pilot sample
completes the questionnaire.17 Respondents to the
questionnaire can be asked how long it took them
to complete it, so that respondents in the main
study can be informed of this. If the questionnaire
is unduly long this may reduce response rate4 and
shortening the questionnaire may have to be
considered. The pilot sample can be asked were
all the instructions and questions clear and
whether any questions were inappropriate? Their
views on layout should be sought.4 They should be
asked, via an open question, whether there were
any areas they believed to be vital to the study area
about which they were not questioned? This relates
to the content validity of the questionnaire, i.e.
the representativeness of the questions to the
research area.15,20 It may seem tempting, if you
are pressed for time, to test the questionnaire on
friends or colleagues but this is not as good as
testing it on the study population, who may
represent, for example, a different age group and
educational level to the study population.4
At the piloting stage it is worth considering
‘‘acquiescence bias’’. This is where subjects char-
acteristically agree with statements irrespective
of their content.1 There are ways to get rid of this
bias e.g. by starting questions using Likert scales
off with strongly disagree and strongly agree
alternately. When checking the pilot sample re-
sponses it may, however, become apparent that
this strategy has caused confusion.
The data produced by the pilot should be
analysed or at least checked to ensure it produces
usable results.4
Ideally after redrafting, the questionnaire
should be re-piloted but this is time dependant.
Whilst it is tempting to cut corners at this stage
thorough preparation will save time when it comes
to analysis.5
Approval to use a questionnaire
In the United Kingdom, a research study on an NHS
patient or member of staff will need to be ap-
proved, formally, by the person registered with the
Department of Health as the research sponsor in
that organisation. It must comply with data pro-
tection law and be logged within the organisation’s
data protection files. It must be compliant with
research governance frameworks22 and be approved
by the appropriate research committee. The NHS
136
has a Central Office for Research Ethics Committees
(COREC) who has a standard electronic form and
guidance notes available, on line, for applications to
Research Ethics Committees in the UK.23
Also, if the questionnaire is being undertaken as
part of a university qualification the advice of the
university supervisor must be sought to ascertain
any additional permissions the university requires.2
Conclusion
Before embarking on designing a questionnaire,
background work needs to be undertaken to ensure
that it is the best tool. If so a literature search for
validated questionnaires should be undertaken. If
nothing suitable is available the questionnaire must
be carefully planned. How the questionnaire will be
delivered and which question types are most ap-
propriate, must be decided. The questionnaire can
then be produced considering such factors as
language, wording, sequencing and questionnaire
length. The layout of the questionnaire should take
account of the factors that increase response rates.
The questionnaire must then be piloted and revised,
before the main study begins. The necessary per-
missions need to be gained, depending on where the
questionnaire is to be used, before any respondents
are involved. Only at this stage, can this potentially
valuable data collection tool, be administered.
However tempting as it may be to try to
shortcut this process, remember that whilst any-
one can produce, copy and distribute a list of
questions it takes planning and skill to devise
a questionnaire which will yield a high response
rate of worthwhile data.
Acknowledgments
I would like to thank my colleague, David Manning,
for his constructive criticism during the writing of
this article.
References
1. Polit D, Hungler B. Nursing research: principles and
methods. 6th ed. Philadelphia: Lippincott; 1999. p. 201.
G. Marshall
2. Boynton P, Greenhalgh T. Selecting, designing and de-
veloping your questionnaire. BMJ 2004;328:1312e5.
3. Williams A. How to.write and analyse a questionnaire. J
Orthod 2003;3:245e52.
4. Jack B, Clarke A. The purpose and use of questionnaires in
research. Prof Nurse 1998;14(3):176e9.
5. Bell J. Doing your research project. 1st ed. Milton Keynes:
Open University Press; 1987. p. 58e69.
6. Bowling A. Measuring health: a review of quality of life
measurement scales. 1st ed. Buckingham: Open University
Press; 1991. p. 1e200.
7. Lohr KN. Assessing health status and quality-of-life instru-
ments: attributes and review criteria. Qual Life Res 2002;
11:193e205.
8. Van Hook MP, Berkman B, Dunkle R. Assessment tools for
general health care settings. Health Soc Work 1996;21:
230e4.
9. Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of
life measurement: bibliographic study of patient assessed
health outcome measures. BMJ 2002;324:1417.
10. Dijkers M. Measuring quality of life: methodological issues.
Am J Phys Med Rehabil 1999;78:286e300.
11. Howie JG, Heaney DJ, Maxwell M, Walker JJ. A comparison
of a patient enablement instrument (PEI) against two
established satisfaction scales as an outcome measure of
primary care consultations. Fam Pract 1998;15:165e71.
12. Speilberger CD, Lushene G, Vagg P, Jacobs G. Manual for
the State e trait anxiety inventory. California: Consulting
Psychologists Press; 1983.
13. Oppenheim A. Questionnaire design, interviewing and
attitude measurement. 2nd ed. London: Continuum In-
ternational Publishing; 1992. p. 49e82.
14. Greenhalgh T. How to read a paper. 2nd ed. Graphycems:
Navarra BMJ books; 2001. p. 45e52.
15. Parahoo K. Questionnaires: use, value and limitations.
Nurse Res 1993;1(2):4e15.
16. McColl E. Questionnaire design and construction. Nurse Res
1993;1c(2):16e25.
17. Youngman M. Designing and analysing questionnaires e
Rediguide 12. 1st ed. Oxford: TRC Rediguides; 1978. p.
1e40.
18. Newell R. Sampling and distribution issues. Nurse Res 1993;
1(2):33e43.
19. Giuffre M. Reading research critically: assessing the re-
liability and validity of the research instrumentation e part
1. J Post Anesth Nurs 1995;10:107e12.
20. Eby M. Validation: choosing a test to fit the design. Nurse
Res 1993;1(2):26e32.
21. Polgar S, Thomas S. Introduction to research in the health
sciences. 3rd ed. Melbourne: Churchill Livingstone; 1995. p.
243e306.
22. Department of Health Research Governance framework
for health and social care. London: Stationery Office; 2002.
!http://www.dh.gov.uk/assetRoot?o4?01/45/57/04014757.
pdfO. [accessed 21.7.2004].
23. NHS Central Office for research Ethics Committees.!http://
www.corecform.org.uk/AppForm/display/information.aspO;
2004. [accessed 23.08.04].