Customer Satisfaction Report

Sieu thi dien may Viet Long - www.vietlongplaza.com.
vn
EUROCONTROL
Customer Satisfaction Report
IANS
A Centre of Excellence in ATM Training
Report
Copyright Notice
2006 European Organisation for the Safety of Air Navigation (EUROCONTROL). All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise. Any other use is subject to prior written consent by EUROCONTROL. Request shall be addressed to: Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupry, L-1432 Kirchberg/Luxembourg.
The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you want and to make your stay in the Institute as enjoyable as possible. All Institute personnel are there to ensure that your stay at the Institute is successful. However, if you do have a complaint (or a compliment) please tell us. If you are not satisfied with the service we provide or you would like to propose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html, or contact IANS.complaints@eurocontrol.int directly.
ii
Copyright 2006 EUROCONTROL
Table of Contents
INTRODUCTION..................................................................................................................................... 4 ATM TRAINING ...................................................................................................................................... 5 ATC TRAINING .................................................................................................................................... 12 E-LEARNING ........................................................................................................................................ 13 STAKEHOLDER FEEDBACK.............................................................................................................. 14 COMPLAINTS ...................................................................................................................................... 15 CONCLUSION ...................................................................................................................................... 16
Printed on 30/11/2004
Produced by the Business and Administration Unit
Introduction
This is the 7th report that the Institute devotes to the assessment of the level of satisfaction of its customers. The study on which this report is based has been conducted in the framework of the quality improvement programme implemented at the Institute since end of 1996. Measuring customer satisfaction on a regular basis is one of the key elements of the quality strategy implemented by the Institute since 1997. Our objective is to understand exactly how our customers perceive the Institute and, based on their level of satisfaction, to improve our services and products. This report proposes a summary and an analysis of the data collected in 2005. A Student Evaluation Form filled in by the students at the end of a course. This evaluation form is used for both the ATM course participants and ATC course participants. An E-Learning Evaluation Form filled in by a student upon completion of an E-Learning Module. A generic Stakeholder Feedback Form filled in by various stakeholders. The form is available at http://www.eurocontrol.int/ians/feedback.html.
Customer Visits
IANS visits a sample of its customers on a yearly basis. The customer visits serve as a main source of feedback on existing products and a source of information for improvements and new products. The customer visits include visits to Air Navigation Service Providers and civil and military State Authorities.
The customer segments assessed in 2005

IANS applies a policy to measure the satisfaction of its customers with all its products and services. These include: The ATM course participants The ATC course participants The E-Learning Students Training Sponsors Consultancy Bodies. User Groups
Complaint Procedure and Complaint Form

IANS uses an anonymous form, available on its web-site for the collection of complaints related to its products, services and staff. A reference to the complaint procedure and associated form is made in all products and services. The complaint form is available at http://www.eurocontrol.int/ians/complaint.html.
Assessment Tools
The assessment is based on a set of assessment tools and measurement methods. During 2005 further work has been done to increase the integrity of the data collection.
Evaluation Forms
To measure the satisfaction of its customers with the products and services IANS uses the following evaluation forms:
ATM Training
The Satisfaction of Students with the ATM Training Products is measured through the use of a Course Evaluation Form.
Measurement Method
At the end of each (classroom) course, students are presented with an evaluation form containing a number of criteria to be assessed. Each criterion can be given a result between 1 and 4 meaning unacceptable (1), unsatisfactory (2), good (3) an excellent (4). The complete set of criteria are grouped in five main categories: Training Content Instructional Team Documentation Equipment Administration
The first three categories are directly related to the course. The last two categories are directly related to the supporting environment and organisation.
Figure 1 Student Evaluation Form
From Forms to Results

We map the marks of the students on a scale from 0 (all students mark Unacceptable) to +100 (all students mark Excellent). When all students mark Good then the result is 75 and when students mark, on average, between Good and Excellent then the result will be around 85. Likewise, if students fill in 'all-Uncacceptable' then the result will be 0. If students fill in 'all-UnSatisfactory' then the result will be 25.
Measuring Satisfaction
Marks from Students
Unsatisfactory 1
Satisfactory 2
Good 3
Excellent 4
Result
0 All 1
25 All 2
75 All 3
100 All 4
Figure 2 How we measure satisfaction
IANS aims to meet our objective of being a Centre of Excellence in ATM Training. For us this means we aim to achieve results on any criterion between 75 and 100. As an example: the overall average (all courses, all criteria, 3090 forms) for IANS in 2005 is 87.5 compared to 85 in 2004. The highest result on a single criterion is 95 (Subject Knowledge of the Instructor). The lowest result remains pre-course information (76.5 compared to 70 in 2004).
Results
The results presented below are extracted from the automated student satisfaction evaluation system. Results can be presented per category, per training domain and per criterion. We have selected the most significant results to be included in this report.
3500 3000 2500 2000 1500 1000 500 0
Size of the Population

This graph shows the number of evaluation forms received and processed by IANS. In 2005 the Institute continued to optimise the processing of the forms. This led to another significant increase of the number of processed forms in 2005.
2002 1937
2003 1915
2004 2624
2005 3090
Number of Forms
Figure 3 Number of Evaluation Forms
Composition of the Population

Students indicate on the evaluation form which type of staff they are.
1400 1200 1000 977 2004 2005 800 600 600 400 192 200 0 Administrative Managerial 223 345
1298
This chart shows that a majority of Students at IANS in 2005 are Operational and Technical Staff. These results are consistent with the results of the previous years.
824 693
Operational
Technical
Figure 4 Composition of the population (number of students)
Satisfaction
100 95 90 85 80 75 70 65 60 55 50
This chart shows the satisfaction of the students with the ATM courses. The result is obtained by averaging the results obtained on every criterion in the categories Training Content, Instructional Team and Documentation. A result of 87 indicates that, on average, students mark criteria between Good and Excellent. The increase in satisfaction has been achieved through detailed analysis of the results of each course and introducing corrective action wherever required.
2002 83
2003 84,5
2004 86
2005 87,5
Course Results
Figure 5 ATM Course Results
Segmented View
100 95 90 85 80 75 70 65 60 55 50 Content Instructor Documentation 2002 81,5 88 80 2003 83 89 80,5 2004 84,5 91 83,5 2005 86,5 92 85
This chart presents the results per measurement Category. These are Training Content, Instructional Team and Documentation. IANS has improved the results on each of these three categories during the last four years. IANS will continue to analyse the results and make improvements where possible and practicable.
Figure 6 ATM Course Results: per Category
Best Results
Customers often praise IANS for its staff and excellent infrastructure. This informal feedback is confirmed by the course evaluation results. The criteria Instructor Subject Knowledge and Classroom Equipment have consistently received the highest marks from the students during the last three years. IANS will continue to invest in the competence of its instructors by continuation training and competency assessment and it will maintain the building at its high standard.
100 90 80 70 60 50 Instructor Subject Knowledge Classroom Equipment
2002 92,5 88,5
2003 93 89
2004 94,5 90,5
2005 95,5 91
Figure 7 Criteria with Best Results
100 90 80 70 60 50 Pre Course Information Course Duration Canteen
Areas for Improvement

IANS continuously looks for opportunities to improve. Criteria with the less positive results are the Pre-Course Information and the Course Duration. In 2004 IANS has put in place an action plan to improve the Pre-Course information and significant progress has been made in this domain in 2004 and 2005. The optimum duration for each course will be reviewed on a courseby-course basis.
2002 61,5 71 73,5
2003 64 74 72,5
2004 70,5 76 75
2005 78 76 77,5
Figure 8 Criteria with less positive Results
In 2005 IANS changed the layout of the coffee area and installed new equipment in the kitchen to improve the flow of students.
Focus on Safety
IANS is heavily involved in the development and delivery of Safety Courses to support Safety Regulation and Safety Management. In 2005 the number of courses delivered in the Safety domain increased from 36 to 50 and the number of participants increased from 614 to 844. We are pleased to note that the results in this relatively new area for IANS equal the average obtained in other, more mature, training domains. It is a clear indication that the rigorous development approach at IANS leads to products that meet customer expectation.
100 95 90 85 80 75 70 65 60 55 50 Safety Overall
2003 86 84,5
2004 88,5 86
2005 90 87,5
Figure 9 Safety Domain Course Results
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Measuring Loyalty
In 2004 we introduced a new question on the evaluation form: Would you recommend the course to a colleague. In 2005, like in 2004, more than 90% of the students would recommend the course that they attended to a colleague.
Yes 2004 2005 91% 92%
No 1% 1%
Blank 8% 7%
Figure 10 Would recommend course to a colleague
10
10,00% 9,00% 8,00% 7,00% 6,00% 5,00% 4,00% 3,00% 2,00% 1,00% 0,00%
Measuring dissatisfaction
To measure dissatisfaction IANS uses the percentage of unsatisfactory marks on the evaluation forms. This chart shows the proportion of unsatisfactory marks on the evaluation forms for all criteria. The excellent results in 2005 are spread over the entire spectrum of criteria.
2002 0,91%
2003 0,90%
2004 0,83%
2005 0,59%
Percentage of 'Unsatisfactory' Results
Note that that an unsatisfactory mark on one criterion does not necessarily mean that the student is not satisfied with the whole product.
Figure 11 Level of dissatisfaction
11
ATC Training
The satisfaction of ATC ab-initio students is measured using the same evaluation as the system used for measuring the satisfaction of the ATM students. Satisfaction figures of ATM and ATC students must however not be compared. The context of these courses, although delivered in the same building, is completely different: The duration of ATM courses is maximum of two weeks whereas ATC have a duration of several months. The student population of ATM courses are staff members from service providers and member states. The student population of ATC courses are abinito ATC students
100 90 80 70 60 50 Training Content Instructors Training Documentation 2002 79,5 76,5 78 2003 79 73 69 2004 75 72,5 76 2005 78 77 78
Last but not least, the ATC ab-initio students are evaluated at the end of the course. At the end of the evaluation they either pass or fail. The outcome of the evaluation tends to have a significant impact on the satisfaction of the students. IANS believes that the critical success factor for ATC students is their validation rate (i.e. the proportion of students that obtain an ATC license). We do however present the satisfaction data in this report as it is part of our quality management tools.
ATC Student Satisfaction

This chart shows the satisfaction of ATC abinitio students. The last few years ATC ab-initio training has been characterised by high rotation of staff and continuous changes to the training content and structure as a result of variations in demand and new regulatory requirements. In 2005, IANS significantly improved the documentation and training of the ATC Instructors. Compared to 2004 the satisfaction of the students has increased for each segment.
Figure 12 ATC Student Satisfaction
12
E-Learning
IANS introduced satisfaction measurement for its elearning products in the middle of 2004. Students are given an opportunity to evaluate each e-learning module at the end of the module. The figures presented in this report cover all the modules on the EUROCONTROL IANS on-line TrainingZone, including the CFMU modules. In 2005, students filled in 2774 forms (800 in 2004) evaluation forms were filled in. 90% of the students identified to be satisfied with the modules and more than 88% of the students indicated that the module met the objectives. These figures indicate that the overall satisfaction with the e-learning modules is excellent.
1.
2.
3.
4.
5.
6. 7.
Why are you studying this module? Personal reasons (e.g. to further career, out of personal interest etc). My employer decided I should. It was a prerequisite for a class-room based course or programme. The module agree not sure-n/a disagree followed a logical sequence, O O O was not too easy or too difficult, O O O was clear and easy to understand. O O O The module was agree not sure-n/a disagree easy to use, O O O interactive and interesting. O O O In order to successfully access this module, did you have to change your browser (Internet Explorer) settings? install new software (flash plug-in, java plug-in, internet explorer)? disable a popup-killer or anti-intrusion programme (e.g. Norton Internet Security)? The module agree not sure-n/a disagree met the objectives set at the beginning,O O O allowed me to check my understanding. O O O Did you enjoy studying this module: O yes O no Will this module Help you directly in your current job? Help you in a future job? Increase your understanding of the industry without direct impact on your current/future job? Help you personally (e.g. out of personal interest)?
E-Learning Evaluation Form

A customised form was developed to capture student satisfaction with the e-learning modules. The form collects information related to the reasons for studying, the complexity of the module, the objectives of the module and the impact of the module on the students job.
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2004 2005 structure 92% 95% level 73% 80% clarity of language 86% 87%
Structure of ELearning Modules

This chart shows the percentage of students that answered yes on the following questions: Structure of the module meets expectations. Level of the module meets the expectations Module uses clear language. These results indicate that the structure and content of the E-Learning modules meet the needs of the students
Figure 13 Structure of E-Learning Modules
13
45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Self-driven Employer-driven Prerequisite 2004 43% 30% 27% 2005 28% 39% 33%
Origin of ELearning Students

It is IANS strategy to integrate e-learning in well defined training programmes defined and implemented by the students employer - and as pre-requisite modules for classroom courses. These results indicate that the proportion of students registered through their employer or studying as a prerequisite module has significantly increased compared to the number of students that study out of their own initiative. The number of students that see a direct impact on their job from studying e-learning modules has increased from 56% in 2004 to 68% in 2005.
Figure 14 Why did students study e-learning modules
14
Stakeholder Feedback
Training Consultation Group
The Stakeholder Feedback Form was distributed to the meeting participants at the end of each Training Consultation Group Meeting. The majority of the feedback indicates that the overall perception of the Institute lies between Good and Excellent (on a scale unsatisfactory satisfactory good excellent). Compared to 2004 the results related to communication (content, format and timeliness) have improved. We will continue to review and improve our communication with the members of the consultation group.
Other stakeholders
The stakeholder distributed to: evaluation form was also Participants of the training focus group. The training partners at IATA. The results indicate an equally high level of satisfaction of working together with IANS.
Customer Visits
The Customer Visits are a process in which IANS meets stakeholders in member states on a bi-lateral basis. It is a mechanism to collect qualitative feedback from various stakeholders. The countries visited in 2005 are: Austria, Belgium, BIH, Bulgaria, Greece, Hungary, Spain, Sweden and Turkey During the meeting of the Training Consultation Group in September 2005 it was agreed that IANS would include in its Customer Satisfaction Report information gathered during the Customer Visits. During these customer visits, stakeholders have in general expressed a high degree of satisfaction with IANS, its staff and the services it offers. The following points have been raised: OJTI: The delivery of OJTI courses through partnerships is often perceived as an expensive solution (by stakeholders receiving the services). The solution is not perceived as entirely satisfactory. ATSEP Training: Many of the visited stakeholders are currently in the process of defining basic training for Air Traffic Safety Electronics Personnel (ATSEP). Some collaboration with EUROCONTROL and other member states could avoid duplication of effort. On-Site Courses: In general the demand for on-site course (i.e. courses at stakeholder premises) exceeds the capacity of IANS. Although most stakeholders are aware of the E-Learning products of the Institute many of them have not yet been able to assess the feasibility to use the products available. ATC Training Notes: IANS should develop a policy to distribute CCC aligned ATC Training Notes.
Complaints
Every training note distributed to students and all pages on the On-Line Trainingzone contain a clearly visible reference to IANS complaint procedure. The procedure enables stakeholders to submit complaints in an anonymous way or an identified complaint through e-mail. The text is as follows: The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you want and to make your stay in the Institute as enjoyable as possible. All Institute personnel are there to ensure that your stay at the Institute is successful. However, if you do have a complaint (or a compliment) please tell us. If you are not satisfied with the service we provide or you would like to propose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html, or contact IANS.complaints@eurocontrol.int directly.
Although the procedure is clearly marked in all documentation, IANS did not receive formal complaints in 2005.
15
Conclusion
IANS aims to be recognised by its stakeholders a Centre of Excellence in ATM Training. We continuously improve our products and processes. We measure the satisfaction of our customers with the products that we provide. This document presented the results of the satisfaction measurement in 2005. The satisfaction with our ATM Training Products shows an upward trend in all domains for the fourth year in a row. The satisfaction of the ATC Ab-initio students with the ATC training started increasing again in 2005. In 2005 we have further exploited satisfaction measures for the E-Learning Products and for the Stakeholders that we work with. The results in these domains show a very high degree of stakeholder satisfaction.
END OF DOCUMENT.
16

Customer Satisfaction Report

Uploaded by

Copyright:

Available Formats

You might also like

Customer Satisfaction Report

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Customer Satisfaction Report

Uploaded by

Copyright:

Available Formats

Sieu thi dien may Viet Long - www.vietlongplaza.com.