Scribd Logo
Upload

Welcome to Scribd!

  • Developing A Questionnaire

    Uploaded by

    ahmed elnimeiri
    8 views51 pages

    Original Title

    Developing a Questionnaire--

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    Download as pdf
    8 views51 pages

    Developing A Questionnaire

    Uploaded by

    ahmed elnimeiri

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    Download as pdf
    of 51
    Ca ocact tea er rent Pieter eere rere Ree Se aed Ce ee en erie err ee eee es eee ee ed + how to analyse data Ce) eee eee Ce ee eee ery peer noes an nee anal Pesce Cae erage ee eee ee eee earl es ee perenne BILL aa c GILLHAM Seti tee me ‘eh Library oan in Puan Dn XGShlgaercotrtebek atte Bia, Contents Series Foreword Acknowledgements “The Pros andl Cons of Qvestinnaites Preparation Dealing the Questions (and Answers) Designing the Questionnaire Geting Them Out and Getting Them Back Displaying the Resuls or Analysis Content Analysis of Open Questions Statistical Analysis of Closed Questions Questionnaires as Part ofa Mulé-medhod Approach 10 Presenting Your Findings Recommended Further Reading Index 2% 7 6 6 1 eeee Series Foreword "The books in this eres ae tended for thowe doing small scale research in rebliesetcings. No previous knowledge of racarch methods is asuimed and the series is particulasly Suited wo practitioners studying fora higher degree or who ‘want to research some aspect oftheir practice, The thinking Underlying the series reflects a major shift in noc science research methods over the past fifteen years away fom a haturalsciencs sigle which emphasies deductive thenry- testing, prior determination of method (usually exper- rental) and ‘generalvable’ ress, towards «recognition that uch eqvirements are often unworkable and inp propriate the real worl, "This i nota dec, because the traditionally ‘sieniic™ methods ate often not adequate as 3 way of understating how people behave in coatext. This dors not mean that one should give up an empiscal,evidenceasel research tac tion but adapt to what is posible and, more importantly, hac sik co yield a truer picture i Gahan Acknowledgements Mrs Jane Cuthill produced the successive dats of the tmaniseipt with a speed and dll Thave come to take for ranted. My wife Judith plished dhe writing and coreeted Tapes of ste Postgraduate students at the Univenity of Strathclyde and the Glasgow School of Art made demands that ed me 10 clarify my thinking and not take too much for granted ‘The Pros and Cons of Questionnaires Ieiseasy tconsruct a questionnaire. With a word proceso its pouible to put togeeher in an evening something that Jools quite reapectale: Aer all, we Know what quetioa naires look lke: hardly @ week goes by without some such coming our way. Journalist often make them up (dhe term is exact oillin space in a magazine, with ts such as ‘Are You A Party Bore? Jnge Your Own Socal Clas They may be quite Amuning to fil in and noone takes them serio, feast all their originators But many soalled “research! question: aires are put together ina no more inpresve fashion ‘Yet developing a questionnaire dha will yield worthwhile lta i diicule And i has tobe sd here thatthe quality of the data emerging ftom even an adequately developed ‘questionnaire i not wonderfal. More on that later. The fssential point that good research cannot be but on poorly collected datas and even i the data are intended for tothing more than the oblivion ofa master's degree dissera ‘ion everything that allows fom them sans Ava ais for practical ation, poorly collected data il he wa lime and rnoney, and le the name of reveaeh "The great popularity of quetionnaie that they provide 4“quich fs’ for research methodeogy no singe metho has been 90 much abused, Thi is pity, Because questionnaires hae thie place as one method, of mest value when used Developing Questionaire tandem with other methods, This mutinethd approach reallile questions is important, because one approach is rarely adequate; and af the result of diffrent metheds converge (agree, oF Bt together) then we can have greater ‘confidence in he finding Why questionnaires? Questionnaires are just one of a range of ways of getting information from people (or answers to our research qu: tions, usually, but not always, by asking questions Ince lenally, one ofthe weakest of questionnaire is that they Seek to yet answers just by asking questions. "Table 1. presents this range schematically; this needs to be sto here you read on From Table I you can see that questonnaires are atthe “aeuctared end ofthe continuum, By this s meant that the rotarcher determines the questions that ate asked andthe range of aneer that can be given, OF course, dhe respon lent has acheter: you, agrrdangr, ticking one answer but of four or five, and soon. But the researcher ha lca decided on the posible anwwers all he or she wants 10 find ‘out is hich answers are selected, This makes it very tty lor the researcher and easy to analyse. Tees also be boring and Sistratng for the respondent although this depends to some extent on the amount of development work the researcher ‘put into che questionnaire efor it was distributed However, its obvious that fall the questions and ll the possible anmwers are determined in advance, the element of Alscovery is much reduced (unless there is 8 very unex pected pattern to the anrwers selected). You don’t know twhac lies Behind the responses selected or, more impor- tantly, answers the sespotdent sight have given had they been free to respond as they wished. Tt is partly for ths 2 i i er poe Developing Questionaire - reason that questionnaire data are necessity supertiil you only have the answer to goon, More or leas structure For most people doing smallscale esearch in their spare time, the most nstructered methods are impractical. But Sociologist and socal anthropologists who want to find out how different cultures (or sub-cltures) work have to se these methods. Tn doing s0 hey immerse dhemseles in the ‘ale’ culture hey look and they listen, and they sometimes ask questions. The formal stricture of an interew or {questionnaire would he both unacceptable ard intrusive, as well ar practically imposible, In the street culture. of Chicago or Liverpool as inthe river ibe ofthe Amazon, the artialty of mack methods is staringly obvious. But chat should make us eet on the constraints that such methods Impose anyway. Interviews andl questionnaires are sell-con- Seious thing. Ave the answers you are getting what people really think? When teachers fil questionnaire atthe end fan in-service training couse do they say the same things a8 ‘hey doin the informal atimosphere of the stallroom? Ammer chat are given axa resul of the mithod wed are nfs ht metho In short, there is tension here: betcen the oxginlity and discovery and validity of the verbal data, ad the conomy of time and effort and money in gathering the data. The only answer compeoinise; but that compromise has tobe kept in mind ‘Open and closed questions, What thete tems mean can be inferred fom what ha gone Defre bat, to make the distinction explicit, clued question The Pr at Cons Quatanaies isone where the pease annwers are predetermined: Mest quetionnaires and structured interviews) a eOmet tnielyofloed questions. Avery simple example it WA Of the filling newspapers do yon rad atleast ome © Wath? Ultowed by a lst of four or five). Sometimes thee fat “other category as well An oer question version of this Wick mewpapes do on red a ett ocr asec? where the respondent hae to think and serie inthe anne). Het tshere the anor is Eatual one and fey predictable close question x probably better But when the answers are in terms of opinions, boi or judgement, asia range of answer is much les Hkely to be represeatative. More than that, reearchers commenly find themselves resorting to techniques that force responses ito predetermined cat sories ‘Open questions are only occasionally ued in question naires because they are more difclt to analyse (andl more troublesome to answer). However, we should note here tht careful esructired interviewing prior to che question tae being constructed can menn tha nat of che probable ‘answers are idee. T argue below that apen questions can lead to a greater level af discovery but that their number and kitd has to be restricted to justify the ‘cost For and against questionnaires In research we have to balance the gains and lomes in anything we choose o do. Boxes 1 and 2 summarize the pros and cons of questionnaires, As you run your eye down {hes in Box Tit seems to present an unaguable case. Let us take each poinin turn, |, TORE HANDE ‘This isthe overwhelming Argument, You can send outa thonsand questionnaires 5 Developing a Queticcnnite Box For queicananes * Low cot in mead money ‘+ Easy to get information fom ot of pple very uty + Respondents can comple the quinn whe i + Analysis ofansvers to doe questions straighten + Las reste an imei respone, ‘ Repordnt anonymity, 1 Lack ofinterviwer bis, * Stananizaton of queions (but tre of sre ater vem). + Gan provide suggestive data fr testing am hypothesis jin the time takes todo two semistructured interviews, And pro rata the financial ests of mailing questionnates are minuscule compared with the probable traveling ‘ms, both in time and money, involved in interviewing ‘Telephone inersiewing (now more common} Eats ent the taaveling but i silvery time-consuming. ‘The main svn inthe use of questionnaires in time, not mney, and fr those doing reseaech ination toa fle jo ‘that can bea erica factor, Easy tga inforaton fo lat of op ey quickly. itn ficiently organized, responses to even large-scale ques: 'ionnair canbe pulled in thin a matter of weeks The ‘umber of interviews one could hope to complete in hat ‘ime would be litle more than a han (the people you Want to interview may not be readily aval or may be unwilling tobe interviewed when you do contact them) 3. Respondents am complete the guetinaie shen i ats hem Interviewing can be a complicated business fash Finding mutwally convesient times concemed. But you can just leave a questionnaire for someone to complete ‘when he or she has the time oe Prot and Con of Quetonaies ‘ i oe ‘in Fact have your analy sheets (ee . 30) ready in advance 1 that you cam ell fle) Answers asthe questionnaires are returned All that Straightforward because dhe answers are predetermine Only their frequen eemaine to be ditcovered. Yio can also have the hinds of display worked out in advance (tals line and bar graphs, pie charts, and s0 om see Cha 5 Respondents can answer i thee one time and at thet own pace They Want to think about their answers (or go a check on Something) then they can do so, Actually chi mined blesing, because the expectation of response i an imterview, can alo motivate dhe rexpondent to "work on 6 RU. cis is ot sighed and there are probably big individual difernes. Some people will undoubkedly feel freer in am anonymous Sipe of responding; but others may be cautious about ommiting themselves to paper, In smallacale research the ‘anonymity may be only nominal in character The general consenus i thatthe value of “anonymity” in encouraging respondents to dle fe uncertain, And there is one major practical disadvantage: that you ‘won't know who has tesponded and who has not, and therefore to whom yeu should send a flows prompt: ing letter ~ sce Chapter 5 2 There is much evidence t show that diferent interewers got diferent answers. Diller fences of perceived race, sex, social class, age and ‘educational level all affect the newer people give [although this ean be anticipated and planned fr) ‘This ‘standardization’ relates to the next point. 8. Tf everyone get the same ‘questions then i aa be claimed that another source of Developing » Question bias is liminated. However, whether these questions are andes in the same way amather mater © Gon pode set date Jr eting an pte, Wheter ei oF not resatchers commonly have ideas they are testing out or checking on when they construct a questionnaire. For example, a nurse may be iterated in sex dflerencesin response to advice on ae care following hospitalization, Do women follow this advice morecarfily than men? Ha significant dilference {sound then further in-depth research ean be justified os to why thisisso ‘The foregoing presents a pretty strong case for ques naires but we now need to consider their negative features (os 2) Box? Agsnn quetonnaite, ‘+ Problens of dat quale compleenes and accuracy) ‘Typically low repose rate ule sume “apie # The ace for reity aad reaiey spe questions * Miurnderstanings canto be coerce + Questionaie development fe pr: 1 Sek infra jut yang question, 4 Asumes repondents have anmwem avaiable in at ong Lack of contol over ender and context of answering + Question wording can have # mar ele on awe 1 Respunent eracy probe, 1 People tlk more en han hey wt 4 Tmpsile wo eheck serioumes of honesty of awe. 4 Respondent uncertainty a to what happetn co date The Pon and Con of Quetananrs Problems of data quality Data quality isa problem a several levels (ce below), but a fundamental problem is dhat queontnairs are often come pleted hastily and careealy. Ask youre bow much thonghe and trouble dl you take over the lst questionnaire ‘yon completed? And what was ie chat made you skate Trough it as quiey se posible? For thee reasons one ‘anot have mich confidence in some of the answers that are given (but you don’t know which), and some questions tay not be answered aa, Eve factual accuracy cannot be astmed. For example, etal ofan individuals medial history (When el you lst ce your GF? What was the problem?) are often answered wrongly, a nurse ad doctors well know. HF hi i true for ‘important’ details, how accurate ate people going to be abot les enportan Tae? 1. Typical te span rates sample caption pat tis depends on whether the respondents know you person: ally on whether the questionnaire is seen a interesting ‘and worthwhile to complete (and when did you lst se fone that wa Hike that?) and the amount of ime and trouble that ha oe expended to compete and return it Surprisingly ite thought unualy given to making ‘questionnaire ini rewarding, bat this may be a Jey factor. “Impersonal” quesionnaites typically attract. a respone rate of around 30 per cent, although fllow- p requests nay increase this by up to a third, Over 50 percent has ole accounted a good response A eaptive froup — students ina Ieeute all stall at a taining Ieeting ~can mean 4 response rate of nearly 100 pee ent, but such questionnaires are sloted i between bother activites, An unexpectedly poor response to quese tionnaires can be a salutary experience for dhe novice researcher Developing Queonnaire 2. Problns of mating rspndent. Few people are steongly motivated by questionnaires ales they can se ft a8 having personal relevance, e. gathering infrmation and opinions on job organization, satay snd salary, oF related to a topic of rea! importance. ‘The market it ‘questionnaire saturated. Even if 4 questionnaire come pleted and rerned, ew respondents wll really have Worked a the answers. This dimension i very apparent in a fcetorface interview (where motivation i much stronger), because you an se the efor the interviewer haa tomate to gives considered response 3, The ned for bast and rail ple questions, Opinions are divided a8 to how long a questionnaire shot be, xcept that it should be as short ax posible. Some are imposibly long, For example, one American queston- naire on experiences of childhood sexual abuse consisted af over 600 questions. A questionnaite maybe tov ight to betaken seriously, but four to six pages (depending on design and layout) is probably the maxim. In excep tional cases up to ewelve pages may be feasible, The need to simplify question, eg avoiding compound questions, for using jargon, is abo part of the prepiloting and piloting stages. Writing questions that are not minner ood, that are not ambiguous or inadequate fr the ‘opie, is sueprisngy diticul, 4. Misonderdendingr emma be cre. Linked tothe above, ‘one ofthe mont fustratng things forthe researcher iso Find thata question hasbeen ‘misunderstood - Of course this s largely the role of inadequate attention to the etal of questions inthe development stage. However Jhuman psychology being what ti, "misunderstandinge can never be ently elitsinated, although carcal plot ing can pick up questions that are ambiguous or mit leading. This is one area whee the interviewer hoe a nmargvable advantage: misundestandingr can be inmmediatly detected and corrected, the roe and Cons of Quennaies 5. Questionaire destopmeat is fen par. OF caine thi doesn’t have to he the cate, so how cam it Be # Weakest ofthe technique? The answer is given in the opening, paragraph of this chapter: that questionnaires ate 90 ‘easy to do quickly and badly dae, in a way, they insite ‘arlesnese More troublesome research metho da not neeraaily lead to better quality data, a8 any exp fenced supervisor of postgraduate research stents can tet, But technigues (lke experimental reach) ‘which are dificult understand and st wpy even in & basic form, do have a kind ofn-bule discipline which i one oftheir attractions. The other ie the rigiity of « ‘method that removes many uncertainties, which perhaps one reason why those who have mastered it ling to st That “hart reseatch so researc, tke ‘wing questionnaires, i easy todo badly, dificult co do swell That i tue ofall methods outside the traditional sient doxalogy 1. Ses information just by ekg ueions Well how ee do you da i The answer i tbat in a questionnaire you Eannot ~a major, bu usually urecognizd, liitation The alternative isthe fxcetoface interview ~ but dont you ask question in an interview? OF course, but ‘questions arent the only — or even the best way of setting people © tell you whae you want t know. ‘Questions presume that people have ready anewers there is also something conrolling, because interrog tary, about a questionand-answer approach, Skilled interviewers get people to talk, o relleet on their respons, to ster in particular direction ina varcty of ways, including, most powerfully, by an intersted silence. Oblique approaches like an appreciative cam- rent (“That must have been dial), expressing one's fown uncertainty (‘Pm not sure T undesstand that) of Feflecing hack what the interviewee hae aid. (you hough there was more t it than the fearans you were ng 4 Qyesonmaire ven) have the offer of directing and encouraging people beng interviewed without making the fel that they are beng pied, Avene rind ane wens sole x ov ogni’ uhm. Ths not a problem with eight factual questions (xcept for accuracy of recall Tt touch more of a probe when opinions are being tonght. Pople may not have dct ota, ony tot have retested onthe topic behind the questions the range of choice-answers may” acento dy
  • means ‘greater than’. But wha, you may as, iva p value? ‘This isthe kind of dual dha many statistical texts teat a ‘obvious But ifyou are completely new tothe subject you wil need to work Ubrough an explanation, 1p stands or probabil 2, Probabilcy ic most cally understood in terms of how ten (one in ten, one in & hundred, ete} a given rel could oceur by chance ‘The les often the result could cecur by chance the higher the probability that your result significant 4, Statisticians talk about “level of significance’ in these 5 In statistics p values are usally expeesed as decimal of 1. Once you get used toit this ina very economical way a expressing probability, For example: 05. = Shanes in 10, or! chance? (evens) 1 = Tenanceio 1d 1005 = Seances in 100 ort chance in 20 B01 = 1 etancein 1 2001 = 1 hance 0 tied son. Below the tine results are conventionally judge as more or lew gnificane “Toft thisin your mind, do the following exercise (answers given atthe end ofthe chapter) Staiical Analy of Coed Quoions What are dhe probate hoe signee eel? 025 = 1ehancein 02) =I ehancein— 0.02 = Lchancein— 8.008 = | chancein— ‘00001 = | chance in — “There isa simple pater, even iinialy you ate a bit blind “Tre now go back our ret 00.39, which doc even each the p=. evel we ca ee wh Sis emphat= ally not siguant 1 coll easly have ocured onetime in way chance alone. Now ths ners, beeasei we hhad just relied on our ‘eyeballing’ of the data on p. 72 we fright have bund owscles saying tht omen are more Ike to se anageris thas men’, when ek aguate tl ws that forthe numbers we have the diflerene is defies not ‘Spica Tsou satis that eb he impresion you Seton enees between small numbers, It is easy to Comething that snot there fan get fom red into your da A practical exercise ‘You may fe you have flowed ll hit rough and under- stood it But you don't telly understand a procedure until yourcan doit Tet assume that in the quetionnaire on the use of analgesics (hese we found there wasn't a sgnifiant sex Gifference between those who used them and thoxe who {in we get the following eels for brand fidelity Developing Questionaire Calculate chi square flowing the procedure already given [ansyer at the endl of he chapret. Limitations on the use of chi square the exec requeney in anyeellofa 2% 2 tables es than 1O)y0u have to apply a correction (known as Yate’ corte tion), We won't go inc tha hee, but you need to be aware ‘ofthat imitation. expected fequenccs are les than ven any cellin a2 x 2 cable you should not ae chi square a al, ‘obtained frequencies are very sll (most les tha 12, for example) chi square isimappropriate and you should ie Fishes test of exact probally. Vou would need to const a specialist statial text for tis. Insel snot difieult Understand, but it involves the computation of factorial (he fnctorial of Dis 1x 2% 3x4 5x6 x7 xB 49 x 10) Factoralsare written as 1! and won, and since isher'stese involves mulilying fetorials you soon get into very big numbers indeed. ‘This means you would have to tse ‘mpter with specialist software such as SPSS (Stains! ‘Program forte Swi Scieaes). Don't let tha discourage yous ‘you get results that are smal bt extreme they may wel ie signicane: For example ial} A sates specials, or indeed anyone uct usng SPSS, ‘would be able wo un this for you very quichly. Sesical Analyt of Cvs Questions Answers Palas 0.25" = J chanceind 02. = Lehancein3 0.02. = 1 ehance in 50 8.005. = 1 hance in 200 ‘1.0001 = 1 chance in 10,000, hi equre = 4788 ‘Whats the pve? » 9 Questionnaires as Part of a Multi-method Approach Questiounstes ae rarely adequate asa eesearch method on their own. Indeed, that i tue of every method, especially when you ate dealing with «complex, real-world situation, There are several dimensions to at adequate picture of any human activity. Diflerent methods have different, even if ‘overlapping, stengths and weaknesses. Ifyou we a range of ‘methods you can put tagethera more adequate picture. The ‘eur study execs dis approach (we Case Study Revarch ‘Mets ithe seis) When you have analysed the results ftom your question aire you will se it limitations, What you have ia Standanlized, descriptive set of data that raises more quese tions than it answers. In particular, by did respondents tele thee anne? Why it there this iflerence between foe group and another? An so. In addition, results rom ‘a quetionnaire have a thi, abstract quality, rather remote fiom dhe realty af people's ives. What it ike t bea stall sure in 4 baby general hospital, oF a teacher in a rural ‘comprehensive schol, ora care worker ia a eilden’s home? Tue basic research questions are camples when ate they no) then your data are going to look prety thin and ‘superficial iP all you can report are the results of « quese Uionnair. In a smalbacale seudy thie lack i going to be Particularly apparent Surveys that employ: questionnaires will often ince Developing a Questionaire what are known as area data (information from witen ‘eord: sta turnover, cages in the level of jo applica ‘ons, increases in the numberof ingle parent, staf absences and soon; whatever is available and appropriate to provide s context); and alto, and in partielan the results of semi= structored interviews with a small number of these who answered the questionnaire. Indeed, questionnaires some- Times end with an inition to volunteer fr an interview "The need for further methods providing eifercut kinds of «ata will become apparent shen the questionnaire research thas Deen earsed out. You will see that you need to do background research ow some topics policy documents, work records, government legislation, departmental regula: tions and s0 forth. Even more clears, you will ce which topics or questions could usefully be developed and explored in detail ina semistructured interview. If you sane 0 convey a picture ofthe reality of peoples lives and, more importantly, what their questionnaire responses might mean, then a face-o-fice interview is without parallels method. Tn other words, i dors¢ just illustra it alo ‘taints What does ‘semistructured’ mean? Ifyou look back at Table LL on p. 3 you will sce where it sands on the structured-unstructared continuum. You have dear que Sons you want answered but yow ask them in way that invites am open response; You prompt the interviewees when neeesary and you have to Kee then on rack aad Keep them ‘moving. There are good reasons for this, Uncontrolled for nder-

    You might also like