[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.
209]
Original Article
A retrospective analysis of the NABH feedback
forms used for training purpose as a tool for
continuous quality improvement Abstract
Prashant Paschal, K.K Kalra,
Bhupendra Kumar Rana
National Accreditation Board
for Hospitals and Healthcare
Providers, ITPI Building, 5th Floor,
4-A, I P Estate, Ring Road,
New Delhi, India
Address for correspondence:
Dr. Bhupendra Kumar Rana,
National Accreditation Board
for Hospitals and Healthcare
Providers ITPI Building,
5th Floor, 4-A,
I P Estate, Ring Road,
New Delhi - 110 002, India.
E‑mail: bkrana@nabh.co
Access this article online
Website: www.nabh.ind.in
DOI: 10.4103/2319-1880.187764
Quick Response code:
© 2016 The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Published by Wolters Kluwer - Medknow
Introduction: Feedback is one of the crucial elements of learning and achievement,
but its outcome can be either positive or negative. Drawing on findings from different
trainings on Program On Implementation (POI) organized by NABH, this study reveals
the extent to which effectiveness can be accurately measured. The study argues
that the participants are in the best position to judge the effectiveness of feedback,
but may not always recognize the benefit it provides.
Aims and Objectives: The aim and objective of this study are: (1) To find out the
effectiveness of the NABH training programs and efficiency of faculties related to its
effect on learning, improvement, and achievement. (2) To find out ways to improve
the NABH form related to delivery, content, and its appropriateness.
Patients and Methods: All the feedback forms were collected from the participants
after the successful completion of the program and then the interpretations and
comments if any provided by them were viewed and analyzed.
Statistical Analysis: Responses were analyzed using average percentage method
which reflected the satisfaction level of participants as well as the appropriateness
of NABH training (POI) programs.
Results: The feedback forms obtained by the participants are analyzed in two
ways. (1) Detailed analysis of feedback forms of all the programs together.
(2) Program wise analysis.
Conclusions: The overall conclusion of the feedback analysis was that NABH needs
to take evaluation feedback much more seriously if they are to achieve the dual
benefits of greater accountability and more effective learning.
Key words: Evaluation, feedback, feedback effectiveness, self‑assessment
INTRODUCTION
Much staff time and effort go into producing assessment feedback, but very little effort is made
to examine its effectiveness. This study draws on findings from the analysis of feedback forms
addressing participant’s engagement with training programs conducted by NABH.
Feedback is an information about how we are doing in our efforts to reach a goal. It is a basic
teaching method essential for learning and improving performance.[1] Hence, training and its
regular evaluation would make NABH to stand in the limelight in achieving the objectives and
This is an open access article distributed under the terms of the Creative Commons
Attribution‑NonCommercial‑ShareAlike 3.0 License, which allows others to remix, tweak, and build upon
the work non‑commercially, as long as the author is credited and the new creations are licensed under
the identical terms.
For reprints contact: reprints@medknow.com
How to cite this article: Paschal P, Kalra KK, Rana BK. A retrospective analysis of the NABH feedback
forms used for training purpose as a tool for continuous quality improvement. J Nat Accred Board Hosp
Healthcare Providers 2016;3:32-8.
32
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]
Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement
imparting the knowledge of quality services being required in
health‑care industry.
METHODOLOGY
Feedback can only be effective when the participant understands
the feedback and he/she is willing and able to act on it. In
program‑on‑implementation  (POI) organized by NABH, the
likelihood of feedback providing unambiguous, categorical
feedback about the effectiveness of the program or how to
improve is very clear with most feedback requiring interpretation.
The feedback forms are distributed to the participants as a
part of course material. They provide their feedback in the
format (feedback forms) prepared by the NABH. All the feedback
forms were collected from the participants after the successful
completion of the program, and then the interpretations provided
by them are viewed and analyzed in regards to the following points:
1. The three criteria for the faculties, namely, knowledge
about the course content, teaching skills, and engagement
with the participants
2. The NABH feedback form was based on the understanding
and usefulness on different parameters such as appropriateness,
venue, food quality, presentation content, timely information,
and administrative arrangement which decide the effectiveness
and usefulness of these training programs
3. Logistics which includes venue arrangement, food quality,
and registration counter.
The participants are required to fill the feedback form based on
a detailed score matrix which includes 5 for excellent, 4 for very
good, 3 for fairly well, 2 for average, and 1 for poor. At last, all
the feedback forms are analyzed, and conclusion is drawn out
with respect to the number of feedbacks received.
The participants gave their feedback based on a grade matrix system
comprising 5‑point Likert scale. All the responses were analyzed using
average percentage method which reflected the satisfaction level of
participants as well as the appropriateness of NABH training (POI)
programs. In addition, the analysis is done against each parameter of
the NABH feedback form, for which average percentage is obtained
for the entire program and each program separately.
Levels of feedback
• Level 1: Participants observe the faculties  (without
judgment) ‑ Description of the observed behavior – easiest
for the learner to hear and accept
• Level 2: Participants’ personal reaction (not judgment)
• Level 3: Participants’ prediction  –  Insights based on
experience about the appropriateness, correctness, or
effectiveness of the training program.
Analysis of feedback forms
This study reveals the analysis of NABH training (POI) feedback
33 The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1
forms of different programs. Feedback forms of 16 different
programs comprising 559 participants (n = 559) were analyzed.
The categorical distribution is as follows: (A) Dental POI: 5; (B)
nursing POI: 6; (C) blood bank: 1; (D) and hospital: 4.
RESULTS
The feedback forms obtained by the participants are analyzed
in two ways. (1) Detailed analysis of feedback forms of all the
programs together. (2) Program wise analysis.
Detailed analysis
The analysis clearly shows that 90% of the participants are satisfied
and fully agree that NABH training programs are appropriate
and useful. Around 81% of the participants agreed that they
receive timely information from the NABH regarding the training
schedule, venue, timing, etc., some of the participants (around
7.6%) are not satisfied with the criteria of timely information.
In regards to the venue decided by the NABH for different
programs, it was observed that 94.6% of the participants
are satisfied and feel that venue is appropriate in terms of
their approachable location, facilities, etc., More than 93%
of the participants agreed and gave their responses in favor
of administrative arrangements done by the NABH. The
satisfaction level suddenly drops down in regards to registration
counter where only 67% of the participants are satisfied, and the
same reaction is being observed in case of presentation content
where only 48.7% of the participants are fully satisfied.
The majority of the participants felt that the NABH provides
quality food at these venues as 89% of the participants gave their
opinion in favor of the criteria. The concern area is nonresponse
obtained for different criteria. As per the percentage analysis
shown in Table 1 and Figure 1, each criterion has some percentage
of participants who did not gave any response which is observed
to be maximum in presentation content and minimum in venue.
Program wise analysis
Dental program
A total of 5 feedback forms comprising 117 participants (n = 117)
were analyzed and their results are calculated based on average
percentage method. The analysis was done for NABH training
programs as well as faculties who present the course content.
The results are shown in [Tables 2,3 and Figures 2,3].
Hospital program
A total of 4 feedback forms comprising 153 participants
(n = 153) were analyzed and their results are calculated
based on average percentage method. The analysis was
done for the NABH training programs as well as faculties
who present the course content. The results are shown in
[Tables 4,5 and Figures 4,5].
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]

Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement

Table 1: Percentage feedback of all programs together

Content Grade matrix (average in percentage)
Excellent/very good Fairly good Average Below average Poor No grade given
Appropriateness or usefulness 85.5 4.5 0 0 0 10
of the workshop for participants
Timely information from NABH 73 7.9 5.1 0 2.6 11.3
Venue 82.5 12.1 2.4 0 1 1.9
Administrative arrangements 90.4 3.2 0.4 0 0 5.9
Registration counter 57 8.9 2.2 0 3 28.9
Presentation content 47 1.7 3.6 0 3.4 44.1
Food quality 77.7 11.6 7 0 1.2 2.4

Table 2: Percentage feedback analysis for dental program

Grade matrix Appropriateness or usefulness Timely information Venue (%) Administrative Presentation
of the training program (%) from NABH (%) arrangements (%) content (%)
Excellent 57.20 56.41 45.29 47.86 39.31
Very good 37.60 34.18 40.17 43.58 17.94
Fairly good 3.41 5.12 11.11 6.86 1.73
Average 0 1.70 1.74 0.85 0.85
Poor 0 0.85 0.85 0 0
No grade given 1.79 1.70 0.85 0.85 40.17

Table 3: Percentage feedback of faculties for dental program

Grade matrix Knowledgeable (%) Well prepared (%) Responsiveness to
participants questions (%)
Faculty 1 Faculty 2 Faculty 1 Faculty 2 Faculty 1 Faculty 2
Excellent 81.81 68.18 77.27 72.72 72.72 72.72
Very good 13.63 18.18 22.73 22.72 22.72 27.28
Fairly good 4.56 13.64 0 0 4.56 0
Average 0 0 0 4.56 0 0
Poor 0 0 0 0 0 0
No grade given 0 0 0 0 0 0

Table 4: Percentage feedback analysis for hospital program

Grade matrix Appropriateness or usefulness Timely information Venue (%) Food Administrative Presentation
of the training program (%) from NABH (%) quality (%) arrangements (%) content (%)
Excellent 54.90 38.56 47.71 43.13 47.05 20.95
Very good 23.52 20.91 32.67 40.52 22.87 17.15
Fairly good 2.63 6.53 10.48 10.45 6.53 0
Average 0 4.59 4.57 0.68 1.34 0
Poor 0 0 0 0 2.61 0
No grade given 18.95 29.41 4.57 5.22 19.6 61.90

Table 5: Percentage feedback of faculties for hospital program

Grade matrix Knowledgeable (%) Teaching skills (%) Engagement with
participants (%)
Faculty 1 Faculty 2 Faculty 1 Faculty 2 Faculty 1 Faculty 2
Excellent 81.06 70.58 77.12 64.70 77.77 60.78
Very good 14.37 20.91 17.64 24.85 16.99 27.45
Fairly good 0 2.61 1.30 5.88 1.32 5.25
Average 0 1.33 0 0 0 1.30
Poor 0 0 0 0.65 0 1.30
No grade given 4.57 4.57 3.94 3.92 3.92 3.92

The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1 34
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]

Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement

Blood bank program DISCUSSION

One feedback form comprising 34 participants (n = 34) were
analyzed and their results are calculated based on average Understanding about effectiveness may require the use of input,
percentage method. The analysis was done for the NABH process, and output measures. However, meaningful evaluation
training programs as well as faculties who present the course requires a common conformity on the purpose of feedback. This
content. The results are shown in [Tables 6,7 and Figures 6,7]. is aided by the use of “efficient” tick‑box feedback forms which
focused on benchmarking and providing analytical justification
Nursing program
of the mark awarded. However, those unsure about the purpose
A total of 6 feedback forms comprising 255 participants (n = 255)
of feedback had not given any response and left the column
were analyzed and their results are calculated based on average
percentage method. The analysis was done for the NABH blank. Limiting feedback to justification of the grade system
training programs as well as faculties who present the course ranging from excellent to poor reinforced the belief among
content. The results are shown in [Tables 8,9 and Figures 8,9]. participants that feedback has no feed‑forward opportunity. The
findings suggest scientific investigation, but this seems to arise
Feedback Analysis
from the difficulty of determining effectiveness rather than a
100%
80%
reluctance to measure what is assumed to be positive.[2]
60%
40%
20%
0%
The key components of feedback[3]
The key components of feedback are tangible, and transparent,
actionable, user‑friendly  (specific and personalized), timely,
ongoing, and consistent.
Grade Matrix (Average in Percentage) Grade Matrix (Average in Percentage)
Grade Matrix (Average in Percentage) Grade Matrix (Average in Percentage)
Grade Matrix (Average in Percentage) Grade Matrix (Average in Percentage) A. Dental Program (NABH training feedback)
70
60
Figure 1: Analysis of all the feedback forms together 50
40
30
20
10
A. Dental Program (Faculties Feedback) 0
Appropriateness or Timely information Venue Administrative Presentation
Knowledgeable usefulness of the from NABH (in percentage) Arrangements Content
(in percentage) Faculty 1 training Program (in percentage) (in percentage) (in percentage)
(in percentage)
16% 19% Knowledgeable Excellent Very Good Fairly good Average Poor No grade Given
(in percentage) Faculty 2
16% 15% Figure 2: Analysis of dental feedback forms
Well Prepared
(in percentage) Faculty 1
16% 18%
Well Prepared B. Hospital Program (For NABH)
70
(in percentage) Faculty 2 60
50
40
30
20
10
0
Figure 3: Analysis of faculties for dental program Appropriateness Timely Venue Food Quality Administrative Presentation
or usefulness of information from (in percentage) (in percentage) Arrangements Content
the training NABH (in percentage) (in percentage)
Program (in percentage)
B. Hospital Program (Faculties Feedback) (in percentage)

Excellent Very Good Fairly good Average Poor No grade Given

Knowledgeable
(in percentage) Faculty 1 Figure 4: Analysis of hospital feedback forms
Knowledgeable
14% 19% (in percentage) Faculty 2
C. For Blood Bank Program (For NABH)
Teaching Skills 80
70
18% (in percentage) Faculty 1 60
50
16% Teaching Skills 40
30
(in percentage) Faculty 2 20
10
0
15% Engagement with Participants Appropriateness Presentation Timely Venue Food Quality Registration
18% (in percentage) Faculty 1 or usefulness of Content information Arrangement (in percentage) Counter
the training (in percentage) (in percentage) (in percentage) (in percentage)
Engagement with Participants Program
(in percentage) Faculty 2 (in percentage)

Excellent Very Good Fairly good Average Poor No grade Given

Figure 5: Analysis of faculties for hospital program Figure 6: Analysis of blood bank feedback forms

35 The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]

Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement

Table 6: Percentage feedback analysis for blood bank program

Grade matrix Appropriateness or usefulness Presentation Timely Venue Food Registration
of the training program (%) content (%) information (%) arrangement (%) quality (%) counter (%)
Excellent 70.58 58.83 61.76 47.05 64.70 38.23
Very good 20.58 35.29 17.66 35.29 23.54 29.42
Fairly good 0 0 0 14.70 11.76 0
Average 0 0 0 0 0 0
Poor 0 0 0 0 0 0
No grade Given 8.84 5.88 20.58 2.96 0 32.35

Table 7: Percentage of faculties for blood bank program

Grade matrix Knowledgeable (%) Teaching skills (%) Engagement with
participants (%)
Faculty 1 Faculty 2 Faculty 1 Faculty 2 Faculty 1 Faculty 2
Excellent 82.35 85.29 76.47 85.29 67.64 85.29
Very good 17.65 14.71 23.53 14.71 29.41 14.71
Fairly good 0 0 0 0 0 0
Average 0 0 0 0 0 0
Poor 0 0 0 0 0 0
No grade given 0 0 0 0 2.95 0

Table 8: Percentage feedback analysis for nursing program

Grade matrix Appropriateness or usefulness Timely Venue Food Administrative Presentation
of the training program (%) information (%) arrangement (%) quality (%) arrangements (%) content (%)
Excellent 60.39 42.76 48.20 29.46 29.49 19.22
Very good 19.61 23.92 37.25 41.06 37.19 5.88
Fairly good 0 5.49 7.84 10.62 10.14 0
Average 0 0.39 3.19 9.66 2.41 0
Poor 0 0.39 2.35 6.76 1.93 0
No grade given 20 27.05 1.17 2.44 18.84 74.90

Table 9: Percentage feedback of faculties for nursing program

Grade Knowledgeable (in Teaching skills (in Engagement with
matrix percentage) percentage) participants (in percentage)
Faculty 1 Faculty 2 Faculty 1 Faculty 2 Faculty 1 Faculty 2
Excellent 80.39 80 85.02 85.02 77.65 74.11
Very good 18.43 17.25 14.98 10.98 20 21.56
Fairly good 0 0.79 0 1.17 0.39 1.96
Average 0 0 0 2.83 0 0
Poor 0 0 0 0 0 0
No grade 1.18 1.96 0 0 1.96 2.37
given

Purpose, measurement, and engagement CONCLUSIONS

The analysis of feedback form explain its purpose of
among and between faculties and participants. Hence, while
faculties acknowledged that feedback provides an “indicator
of performance,” it is obvious that it plays a role in learning
and development and helps to move participants in the right
direction.
Comparison of feedback forms
We have compared the old & modified feedback form. The
detailed content are given in [Tables 10 and 11].
The overall conclusion of the feedback analysis was that NABH
needs to take evaluation feedback much more seriously if they
are to achieve the dual benefits of greater accountability and
more effective learning.
From the above feedback, the conclusions that were drawn include:[4]
1. Training environment must favorable
2. Course material should include more case studies and
practical trainings

The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1 36
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]

Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement

C. Blood Bank Program (Faculties Feedback) Table 10: Earlier version of feedback form
Rating
Knowledgeable
(in percentage) Faculty 1 5 4 3 2 1
18% 17% Knowledgeable Appropriateness or usefulness of the
(in percentage) Faculty 2 program for the participants
Teaching Skills Timely information from the NABH
14% 17% (in percentage) Faculty 1 Venue
Teaching Skills Administrative arrangements
18% 16% (in percentage) Faculty 2
Presentation contents
Engagement with Participants
(in percentage) Faculty 1 Faculty was
Knowledgeable
Name of faculty 1
Figure 7: Analysis of faculties for blood bank program
Name of faculty 2
Well prepared
D. Nursing Program (NABH Feedback) Name of faculty 1
80
70
60
Name of faculty 2
50
40 Responsive to participants’ questions
30
20
10 Name of faculty 1
0
Appropriateness Timely Venue Food Quality Administrative Presentation Name of faculty 2
or usefulness of information Arrangement (in percentage) Arrangements Content
the training (in percentage) (in percentage) (in percentage) (in percentage)
Program
(in percentage)

Excellent Very Good Fairly good Average Poor No grade Given

Table 11: Modified version of feedback form
Rating
Figure 8: Analysis of nursing feedback forms
5 4 3 2 1
Appropriateness or usefulness of the
program for the participants
D. Nursing Program (Faculties Feedback)
Presentation contents
Knowledgeable Timely information from the NABH
(in percentage) Faculty 1 Logistics
15% 17% Knowledgeable Venue arrangement (e.g., sitting
(in percentage) Faculty 2 and audio‑visual aids)
16% Teaching Skills Food quality
16%
(in percentage) Faculty 1 Registration counter (if applicable)
Teaching Skills Course faculty
18% 18% (in percentage) Faculty 2
Name of faculty 1
Engagement with Participants Knowledge about course contents
(in percentage) Faculty 1
Teaching skills
Engagement with participants
Figure 9: Analysis of faculties for nursing program Name of faculty 2
Knowledge about course contents
3. Session should not be lengthy and exhaustive and must Teaching skills
include video demonstration and mock drills with Engagement with participants
free time
4. The learning phase is evaluated by conducting test before Recommendations
and after training
We recommend that all faculties videotape their own classes
5. NABH should add facility planning criteria chapter within
at least once a month. Peer review is another strategy for
content, aims, and objectives for further information and
managing the load to ensure lot of timely feedback; it is
clarity.
essential, however, to train staffs to do small group peer
There was a recognition that more effort is needed in identifying review to high standards, without immature criticisms or
and prioritizing target audiences in specific training program, unhelpful praise.
and that attention needs to be given to this early on in the
process, not after the evaluation is completed. It was agreed Some recommendations for effective feedback are as follows:-
that “learning by actually doing” is often the most effective • Choosing the option which offers the best solution
feedback route. • Identifying possible courses of action

37 The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1
[Downloaded free from http://www.nabh.ind.in on Friday, December 27, 2019, IP: 106.220.112.209]

Paschal, et al.: A retrospective analysis of the NABH feedback forms used for training purpose as a tool for continuous quality improvement

• Weighing up the pros and cons of each alternative REFERENCES

• Defining the nature of the issue to be dealt with
implementing the policy. 1. Hewson MG, Little ML. Giving feedback in medical education:
Verification of recommended techniques. J Gen Intern Med
Financial support and sponsorship 1998;13:111‑6.
2. Hattie J, Timperley H. The Power of Feedback. Vol. 77. America:
Nil. American Educational Research Association; 2007. p. 81‑112.
3. Turnbull J, Gray J, MacFadyen J. Improving in‑training evaluation
programs. J Gen Intern Med 1998;13:317‑23.
Conflicts of interest
4. Kunche A, Puli RK, Guniganti S, Puli D. Analysis and evaluation of
There are no conflicts of interest. training effectiveness. Hum Resour Manage Res 2011;1:1‑7.

The Journal of National Accreditation Board for Hospitals & Healthcare Providers | Jan-Jun 2016 | Vol 3 | Issue 1 38