Questionnaire Design

Cathy A. Jenkins, MS Department of Biostatistics cathy.jenkins@vanderbilt.ed D!""#$ MC%& "!'()*

"

+ estionnaires in Clinical ,esearch

M

ch of the data in clinical research is -athered sin. estionnaires or intervie/s. 0he validity of the res lts depends on the . ality of these instr ments.

1ood . estionnaires are diffic lt to constr ct& bad . estionnaires are diffic lt to analy2e.

Diffic

lt to desi-n for several reasons3

4ach . estion m st provide a valid and reliable meas re. 0he . estions m st clearly comm nicate the research intention to the s rvey respondent. 0he . estions m st be assembled into a lo-ical, clear instr ment that flo/s nat rally and /ill keep the respondent s fficiently interested to contin e to cooperate.

+ ality aims in s rvey research

1oal is to collect information that is3 Valid3 meas res the . antity or concept that is s pposed to be meas red Reliable: meas res the . antity or concept in a consistent or reprod cible manner Unbiased: meas res the . antity or concept in a /ay that does not systematically nder! or overestimate the tr e val e Discriminating: can distin- ish ade. ately bet/een respondents for /hom the nderlyin- level of the . antity or concept is different

Steps to desi-n a . estionnaire3

#. ". 5.

6. 9. ).

7rite o t the primary and secondary aims of yo r st dy. 7rite o t concepts8information to be collected that relates to these aims. ,evie/ the c rrent literat re to identify already validated . estionnaires that meas re yo r specific area of interest. Compose a draft of yo r . estionnaire. ,evise the draft. Assemble the final . estionnaire.

Step #3 Define the aims of the st dy

7rite

o t the problem and primary and secondary aims sin- one sentence per aim. :orm late a plan for the statistical analysis of each aim. Make s re to define the tar-et pop lation in yo r aim;s<.

Step "3 Define the variables to be collected

7rite a detailed list of the information to be collected and the concepts to be meas red in the st dy. Are yo tryin- to identify3

Attit des %eeds Behavior Demo-raphics Some combination of these concepts

0ranslate these concepts into variables that can be measured. Define the role of each variable in the statistical analysis3

=redictor Confo nder > tcome

Step 53 ,evie/ the literat re

,evie/

c rrent literat re to identify related s rveys and data collection instr ments that have meas red concepts similar to those related to yo r st dy?s aims. Saves development time and allo/s for comparison /ith other st dies if sed appropriately. =roceed /ith ca tion if sin- only a s bset of an e@istin. estionnaire as this may chan-e the meanin- of the scores. Contact the a thors of the . estionnaire to determine if a smaller version of the instr ment e@ists that has also been validated.

Step 63 Compose a draft A#B3

Determine

the mode of s rvey administration3 face!to! face intervie/s, telephone intervie/s, self!completed . estionnaires, comp ter!assisted approaches. 7rite more . estions than /ill be incl ded in the final draft. :ormat the draft as if it /ere the final version /ith appropriate /hite space to -et an acc rate estimate as to its len-th C lon-er . estionnaires red ce the response rate. =lace the most important items in the first half of the . estionnaire to increase response on the important meas res even in partially completed s rveys. Make s re . estions flo/ nat rally from one to another.

'

Compose a draft A"B3

+

estion3 Do/ many c ps of coffee or tea do yo drink in a dayE =rinciple3 Ask for an ans/er in only one dimension. Sol tion3 Separate the . estion into t/o C

;#< Do/ many c ps of coffee do yo drink d rin- a typical dayE ;"< Do/ many c ps of tea do yo drink d rin- a typical dayE

#(

Compose a draft A5B3

+ estion3 7hat brand of comp ter do yo o/nE

;A< FBM =C ;B< Apple

=rinciple3 Avoid hidden ass mptions. Make s re to accommodate all possible ans/ers. Sol tion3

;#< Make each response a separate dichotomo s item Do yo o/n an FBM =CE ;Circle3 Ges or %o< Do yo o/n an Apple comp terE ;Circle3 Ges or %o< ;"< Add necessary response cate-ories and allo/ for m ltiple responses. 7hat brand of comp ter do yo o/nE ;Circle all that apply<
H H H H Do not o/n comp ter FBM =C Apple >ther

##

Compose a draft A6B3

+

estion3 Dave yo had pain in the last /eekE

A B >ften A B Iery often

A B %ever A B Seldom
=rinciple3

Make s re . estion and ans/er options

match. Sol tion3 ,e/ord either . estion or ans/er to match.

Do/ often have yo had pain in the last /eekE A B %ever A B Seldom A B >ften A B Iery >ften

#"

Compose a draft A9B3

+

estion3 7here did yo -ro/ pE

Co ntry :arm City

=rinciple3

Avoid . estions havin- non!m t ally e@cl sive ans/ers. Sol tion3 Desi-n the . estion /ith m t ally e@cl sive options.

7here did yo -ro/ pE

Do

se in the co ntry :arm in the co ntry City

#5

Compose a draft A)B3

+

estion3 Are yo a-ainst dr - ab seE ;Circle3 Ges or %o< =rinciple3 7rite . estions that /ill prod ce variability in the responses. Sol tion3 4liminate the . estion.

#6

Compose a draft A$B3

+ estion3 7hich one of the follo/in- do yo think increases a person?s chance of havin- a heart attack the mostE ;Check one.< A B Smokin-A B Bein- over/ei-ht A B Stress =rinciple3 4nco ra-e the respondent to consider each possible response to avoid the ncertainty of /hether a missin- item may represent either an ans/er that does not apply or an overlooked item. Sol tion3 7hich of the follo/in- increases the chance of havin- a heart attackE

Smokin-3 Bein- over/ei-ht3 Stress3

A B Ges A B %o A B Don?t kno/ A B Ges A B %o A B Don?t kno/ A B Ges A B %o A B Don?t kno/

#9

Compose a draft A*B3

+

estion3
;#< Do yo c rrently have a life ins rance policyE ;Circle3 Ges or %o< Ff no, -o to . estion 5. ;"< Do/ m ch is yo r ann al life ins rance premi mE

=rinciple3

Avoid branchin- as m ch as possible to avoid conf sin- respondents. Sol tion3 Ff possible, /rite as one . estion.

Do/ m ch did yo spend last year for life ins ranceE ;7rite ( if none<.

#)

Step 93 ,evise
Shorten

the set of . estions for the st dy. Ff a . estion does not address one of yo r aims, discard it. ,efine the . estions incl ded and their /ordin- by testin- them /ith a variety of respondents.

4ns re the flo/ is nat ral. Ierify that terms and concepts are familiar and easy to nderstand for yo r tar-et a dience. Jeep recall to a minim m and foc s on the recent past.

Step )3 Assemble the final . estionnaire A#B3

#$

Decide /hether yo /ill format the . estionnaire yo rself or se comp ter!based pro-rams for assistance3

S rveyMonkey.com Adobe Kive Cycle Desi-ner $.( 1C,C assistance 0he p rpose of the st dy Do/ the data /ill be sed Fnstr ctions on ho/ to fill o t the . estionnaire Go r policy on confidentiality

At the top, clearly state3

Fncl de identifyin- data on each pa-e of a m lti!pa-e, paper!based . estionnaire s ch as a respondent FD n mber in case the pa-es separate.

#*

Assemble the final . estionnaire A"B3

1ro

p . estions concernin- major s bject areas to-ether and introd ce them by headin- or short descriptive statements. >rder . estions in order to stim late recall. >rder and format . estions to ens re nbiased and balanced res lts.

#'

Assemble the final . estionnaire A5B3

Fncl

de /hite space to make ans/ers clear and to help increase response rate. Space response scales /idely eno -h so that it is easy to circle or check the correct ans/er /itho t the mark accidentally incl din- the ans/er above or belo/.

>pen!ended . estions3 the space for the response sho ld be bi- eno -h to allo/ respondents /ith lar-e hand/ritinto /rite comfortably in the space. Closed!ended . estions3 line p ans/ers vertically and precede them /ith bo@es or brackets to check, or by n mbers to circle, rather than open blanks.

Lse

lar-er font si2e ;e.-., #6< and hi-h contrast ;black on /hite<.

"(

4nhance response rate

7hen

/ritin- . estions and assemblin- the final . estionnaire, edit /ith a vie/ to/ards saliency3 apparent relevance, importance, and interest of the s rvey to the respondent Consider either pre!notifyin- those in yo r sample or sendin- reminders to those /ho received the s rvey ;if self!administered<. St dies have sho/n that makincontact /ith the sampled individ als increases the response rate. Ff possible, offer an incentive.

"#

%on!responders
Lnderstandin-

the characteristics of those /ho did not respond to the s rvey is important to . antify /hat, if any, bias e@ists in the res lts. 0o . antify the characteristics of the non!responders to postal s rveys, Moser and Jalton s --est trackin- the len-th of time it takes for s rveys to be ret rned. 0hose /ho take the lon-est to ret rn the s rvey are most like the non!responders. 0his res lt may be sit ation! dependent.

""

Concl sions
Go

need plenty of timeM

Desi-n yo r . estionnaire from research hypotheses that have been caref lly st died and tho -ht o t. Disc ss the research problem /ith collea- es and s bject matter e@perts is critical to developin- -ood . estions. ,evie/, revise and test the . estions on an iterative basis. 4@amine the . estionnaire as a /hole for flo/ and presentation.

"5

,eferences
Desi-nin-

Clinical ,esearch, Stephen B. D lley, et. al. ///.statpac.com8s rveys NDesi-n and se of . estionnaires3 a revie/ of best practice applicable to s rveys of health service staff and patientsO, Dealth 0echnolo-y Assessment, "((#. Iol.9, %o. 5#. Moser CA, Jalton 1. S rvey methods in social investi-ation. "nd ed. Aldershot3 1o/er& #'$#. + estionnaire Desi-n lect re, 0heresa Scott

"6

Ans/er key A#B3

+

estion3 Do/ many c ps of coffee or tea do yo drink in a dayE =rinciple3 Ask for an ans/er in only one dimension. Sol tion3 Separate the . estion into t/o C

;#< Do/ many c ps of coffee do yo drink d rin- a typical dayE ;"< Do/ many c ps of tea do yo drink d rin- a typical dayE

"9

Ans/er key A"B3

+ estion3 7hat brand of comp ter do yo o/nE

;A< FBM =C ;B< Apple

=rinciple3 Avoid hidden ass mptions. Make s re to accommodate all possible ans/ers. Sol tion3

;#< Make each response a separate dichotomo s item Do yo o/n an FBM =CE ;Circle3 Ges or %o< Do yo o/n an Apple comp terE ;Circle3 Ges or %o< ;"< Add necessary response cate-ories and allo/ for m ltiple responses. 7hat brand of comp ter do yo o/nE ;Circle all that apply<
H H H H Do not o/n comp ter FBM =C Apple >ther

")

Ans/er key A5B3

+

estion3 Dave yo had pain in the last /eekE

A B >ften A B Iery often

A B %ever A B Seldom
=rinciple3

Make s re . estion and ans/er options

match. Sol tion3 ,e/ord either . estion or ans/er to match.

Do/ often have yo had pain in the last /eekE A B %ever A B Seldom A B >ften A B Iery >ften

"$

Ans/er key A6B3

+

estion3 7here did yo -ro/ pE

Co ntry :arm City

=rinciple3

Avoid . estions havin- non!m t ally e@cl sive ans/ers. Sol tion3 Desi-n the . estion /ith m t ally e@cl sive options.

7here did yo -ro/ pE

Do

se in the co ntry :arm in the co ntry City

"*

Ans/er key A9B3

+

estion3 Are yo a-ainst dr - ab seE ;Circle3 Ges or %o< =rinciple3 7rite . estions that /ill prod ce variability in the responses. Sol tion3 4liminate the . estion.

"'

Ans/er key A)B3

+ estion3 7hich one of the follo/in- do yo think increases a person?s chance of havin- a heart attack the mostE ;Check one.< A B Smokin-A B Bein- over/ei-ht A B Stress =rinciple3 4nco ra-e the respondent to consider each possible response to avoid the ncertainty of /hether a missin- item may represent either an ans/er that does not apply or an overlooked item. Sol tion3 7hich of the follo/in- increases the chance of havin- a heart attackE

Smokin-3 Bein- over/ei-ht3 Stress3

A B Ges A B %o A B Don?t kno/ A B Ges A B %o A B Don?t kno/ A B Ges A B %o A B Don?t kno/

5(

Ans/er key A$B3

+

estion3
;#< Do yo c rrently have a life ins rance policyE ;Circle3 Ges or %o< Ff no, -o to . estion 5. ;"< Do/ m ch is yo r ann al life ins rance premi mE

=rinciple3

Avoid branchin- as m ch as possible to avoid conf sin- respondents. Sol tion3 Ff possible, /rite as one . estion.

Do/ m ch did yo spend last year for life ins ranceE ;7rite ( if none<.