Professional Documents
Culture Documents
Data Capturing Strategies Used in Istat To Improve Quality: Conference of European Statisticians
Data Capturing Strategies Used in Istat To Improve Quality: Conference of European Statisticians
Data Capturing Strategies Used in Istat To Improve Quality: Conference of European Statisticians
Rossana Balestrino, Stefania Macchia, Manuela Murgia ISTAT Italian National Statistics Bureau Rome, Italy balestri@istat.it, macchia@istat.it, murgia@istat.it
1
CATI/CAPI offer already mature and well tested solutions so have a higher rate of consolidation CASI techniques are younger and more depending on the continuously evolving of IT solutions and network tools 2
in-house strategy
It consists in relying on a private company for the call centre, the selection of interviewers and to carry out the interviews, but in giving it all the software procedure, developed in Istat, to manage the data capturing phase: calls scheduler electronic questionnaire set of indicators to monitor the interviewing phase
8
Quality standards have been defined for: the data capturing phase the monitoring phase the secure transmission of data
10
A limited but exhaustive set of indicators to monitor the trend of contact results Ad hoc instruments to monitor particular aspects of the survey
13
14
The daily transmission is based on a secure protocol (HTTPS) and puts data on an Istat server, INDATA, placed outside the
firewall and devoted to data collection
15
Interviews length
1200 500 1348 543 10 56 13 20 903 2654
Response rates
92.6% 93.2% 94.7% 96.8% 95.8% 94.7% 99.8% 72.4%
Refusal rates
5.4% 4.9% 3.9% 2.2% 3.6% 4.8% 0.1% 16.0%
16
Nr of checking rules
195 205 324 122 52
Sample births survey 2001 Sample births survey 2004 University-to-work transition survey and perspectives 2004 Upper secondary school graduates survey 2004 Water System Surveys (preliminary survey) 2006
2,774
280
17
Checking rules in the data capturing phase with the in-house strategy
The number checking rules included in the data capturing phase (together with the number of variables) are surely significant indicators of the complexity of the survey questionnaire
This complexity has not negatively affected the response and refusal rates
because
18
the trade-off between the quality of data and the fluency of the interview has been taken into consideration
different treatments of the rules to detect errors have been implemented
19
The trade-off between the quality of data and the fluency of the interview
The consistency plans included in the electronic questionnaires comprised a great part, even if not all, of the rules proper of the edit and imputation plans avoiding, during the interview, a too frequent display on the pc-screen of a dialog window asking for the confirmation of the given answer
(including the complete edit plan in the data capturing phase would have guaranteed a high quality of the answer but would have definitely burdened the respondent and the interviewer, thus increasing the interruption rate)
20
2004
2001
2004
2001
94.7% 4.8%
85.4% 10.8%
95.8% 3.6%
94.0% 3.9%
23
No errors
From 1 to 2 errors From 3 to 4 errors 5 and more errors
13,013
5,742 1,183 470
63.8
28.1 5.8 2.3
63.8
91.9 97.7 100
12,245
9,029 1,582 406
52.6
38.8 6.8 1.8
52.6
91.4 98.2 100
Total
20,408
23,262
- 2001: 4.92% of raw data had to be corrected, during the edit and imputation phase - 2004: 0.81% (with the new strategy) had to be corrected, during the edit and imputation phase
25
26
CASI
prototypal experiences realised in the late 1990s current situation comprises several Web sites, located at Istat side and dedicated to the capture of surveys data for approximately 30 surveys
The need of designing a new environment and new rules aimed at introducing more standard solutions and effective security measures came out.
27
31
In synthesis
Both primary (single questionnaire, CSAQ = Computer Self Administrated Questionnaire ) and secondary data collection (collection of data) are dealt with.
34
System Architecture
Firewall Load Balancer Load Balancer
Web server
Web server
Front End
Back End
DB server
DB server
36
Central Directorate for Structural Surveys on Businesses Central Directorate for Short Term Surveys on Businesses Central Directorate for Surveys on Institutions
13
6 2
TOTAL
21
37
N. of treated surveys 10 8
1
PHP language - EXCEL questionnaire offline compilation PHP language - BLAISE questionnaire offline compilation
38
2005
2004 2005
10,000
45,000 68,000
1
2 2
75%
23% ...
2004
2004 2005
15,000
250 250
15
3 3
30%
100% ...
40
2
3 4 5 6 7 8
Yearly Survey on the Structure of Labour PHP language - EXCEL questionnaire - offline Cost compilation Yearly Survey on Telecommunication Enterprises Yearly Survey on structure and production of farms Quick Survey on certificates of balance accounts of Municipalities Quick Survey on certificates of balance accounts of Provincial Administrations Three-year survey on graduates (survey addressed to Universities) PHP language - EXCEL questionnaire - offline compilation PHP language BLAISE executable questionnaire - offline compilation Documentation and instructions for sending a file Documentation and instructions for sending a file PHP language - EXCEL questionnaire -42 offline compilation
43
Thanks
44