Download as pdf or txt
Download as pdf or txt
You are on page 1of 15

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/361580964

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS


OF INDIA

Article  in  Educational Assessment · June 2022

CITATIONS READS

2 107

2 authors:

Badrilal Gupta Pratibha Bundela Gupta


National Institute of Technical Teachers' Training and Research, Bhopal institute of professional education and research bhopal
88 PUBLICATIONS   54 CITATIONS    18 PUBLICATIONS   50 CITATIONS   

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Badrilal Gupta on 28 June 2022.

The user has requested enhancement of the downloaded file.


Indonesian Journal of Educational Assessment
p-ISSN : 2655-2892
e-ISSN : 2684-8074
http://ijeajournal.kemdikbud.go.i
d

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL


INSTITUTIONS OF INDIA

B. L. Gupta1 and Pratibha Bundela Gupta2


National Institute of Technical Teachers Training and Research, Bhopal, India
IPER, Institute of Management, Bhopal, India
E-Mail: badrilalgupta72@gmail.com

Received: 08/07/2021; revisions: 03/01/2022; accepted: 09/01/2022


Published online: 23/01/2022

Abstract. The implementation of outcome-based education in technical institutions (engineering,


polytechnic, pharmacy, management, and architecture colleges) compelled the teachers to use outcome-
based assessment tools and techniques to assess the learning of the students with reference to defined learning
outcomes. Rubrics are valid, reliable, and accurate assessment tools that are being used by trained and not
trained teachers (Professors, Associated Professor, and Assistant professors) in the assessment. This study
was conducted to critically examine the use of rubrics in technical institutes. The paper is based on primary
data gathered from 315 technical teachers from July 2020 to November 2020 using researchers designed
Google Form. The findings of the study are noted on the training of teachers in preparation and use of rubrics,
the purpose of using rubrics, abilities assessed using rubrics, types of rubrics prepared and used, the process
of preparing rubrics, major difficulties, and barriers to use of rubrics, the process of ensuring validity and
reliability of using rubrics, the impact of using rubrics, the decision was taken based on the analysis of results
of rubrics and suggestions to improve the quality and use of rubrics at institute level. Researchers have
concluded and noted suggestions for harnessing the full potential of rubrics. Areas for further study are stated.
Keywords: outcome-based assessment, holistic and analytical rubrics, impact of rubrics

INTRODUCTION attainment of learning outcomes in affective and


India has formally adopted the philosophy of psychomotor domains of learning. It is also a
outcome-based education in technical institutions in significant and direct tool to assess the quality of
the year 2013 (NBA, 2013). Later on, the same outputs, processes of learning, and inputs to
philosophy is adapted in higher education learning. The study presents the overall scenario of
institutions (NAAC, 2017). The curriculum is being training, awareness, types of rubrics, ability assessed
designed using outcome-based education using rubrics, the process of developing and using
philosophy. The faculty members of technical rubrics, difficulties, and barriers in the use of rubrics,
institutions are learning to implement an outcome- and impact of the use of rubrics in technical
based curriculum. They are using the rubrics in the institutions.
implementation of the curriculum. In outcome-based education assessment of
Assessment of learning outcomes plays a very learning outcomes have become important for
important role in ensuring the effectiveness of the certification purpose. The stakeholders are
implementation of an outcome-based curriculum. pressurizing the institutions to have an objective,
The assessment should be valid, reliable, credible, data-based, and transparent assessment
practicable, integrated with the learning process, system. The use of rubrics has become important
transparent, and productive. There are many because it is being used as a motivational tool for
learning outcomes in the curriculum which are learning, a diagnostic tool for diagnosing learning
important and commonly used in the world of work problems, an instructional tool for taking corrective
situations. Some of these learning outcomes cannot and preventive actions for learning, a feedback
be assessed accurately without using the rubric. providing tool for continuous improvement in
Therefore, the rubrics are one of the significant, learning, and certification tool for awarding grades.
valid, reliable, and accurate tools to assess the

20
A rubric is an important tool that is provide the range of marks for each grade create
predominantly being used in higher education in challenges for students and teachers for using them
general and technical institutions in particular for resulting in a waste of time and effort (Ho, 2019).
causing learning, diagnosing the learning problems The sliding scale rubrics are useful to improve
of the students and awarding the grades in all three the learning of the students in different stages of
domains of learning at the course level. It is a learning. These are used for improving the teaching-
versatile tool that is integrated with the learning learning process at the individual and school levels
process in different forms. The rubric has attracted (Jacobo, 2019).
the attention of many researchers to enhance its use The rubrics are defined as an appropriate tool
in technical education. for assessment. The analytical rubrics consisting of
separate criteria and descriptive rubrics consisting of
LITERATURE REVIEW ON RUBRICS four to five levels are studied. The criteria used in the
The research study to verify the reliability and rubric depend on the purpose. The numbers of levels
validity in assessment using rubric concluded that used in the rubric are three, four, and five. The
the use of rubrics enhances the reliability in the author stated that the value of the rubric lies in its
assessment of performance. It also increases the use in formative assessment. Appropriate criteria
validity of performance assessment if appropriately and performance levels are key to the effective rubric
designed and used by trained assessors. Rubrics are (Brookhart, 2018).
considered to be a good tool to promote, improve, A rubric is a tool that can help the student to
and consolidate learning and instruction. Rubrics guide their learning process and become the agent of
facilitate specific feedback providing processes and their learning. It is a good tool to become learner-
are useful for self-assessment (Anders Jonsson, centred (David C. Leader, 2018).
2007). The rubric is used in a variety of practices and
A model of collaborative rubric construction research studies. It is used for the holistic
was developed. This rubric construction process development of the students. It declares articulation
involves teachers and students in constructing the of standards of learning outcomes. Quoting Popham
rubric following four stages viz literature review and the author stated three features of the rubric:
Delphi method, rubric co-construction, use of co- measuring criteria, the quality definition for these
constructed rubrics, and model of collaborative criteria at a particular level, and scoring strategy.
rubric construction (Morton, Northcote, Kilgour, & The author suggested 14 rubric design elements to
and Jackson, 2021). make the rubric more replicable design, research,
The rubrics are promoted as educational and practice-oriented (Dawson, 2020).
tools to guide the students learning using good The results of the students are improved by
practices. Students are involved in construction 10% using the rubrics for learning and assessment of
of rubrics (Jason K. Morton, 2021) learning. (Simona Laurian, 2013)
An analytical rubric enhances objectivity and The rubric is defined as a multipurpose
reliability in performance assessment. The process scoring guide for assessing student product and
of preparing an analytical rubric comprises four performance. The rubric has great potential for
steps viz, categories, levels of performance, pre-use nontraditional, first-generation, and minority
review, and post-use evaluation (McCormick, students. Three steps to rubric development are;
2021). identify performance criteria, set performance
The use of an analytical rubric with a levels, and create a performance description. The
problem-based learning method increases the benefits of the rubrics are; learning targets are clear,
critical thinking skills of students (Nurhuda, 2021). a guide to instructional design and delivery, accurate
The feedback rubrics develop process skills in measurements, and helps in advance learning. The
students such as information processing, critical limitations of the rubric are it takes time to design, it
thinking, communication, and teamwork. (Doug may be too narrow or too broad, the developer may
Czajka, 2021) not be trained to design the rubric, the user may not
Peer review of the assessment process is an be proficient and comfortable using the rubric and
innovation in the field of assessment that was draw conclusions (Stevens, 2007).
implemented to achieve the attainment of outcomes, The rubrics are being used for various
accountability, satisfaction, a better understanding purposes in higher education. These purposes are
of rubric, and assessment (Grainger, 2021) increasing student achievement, improving
The rubrics have multiple purposes in instruction, and evaluating programmes, changes,
education. Good practices such as standardization and improvements in course delivery. The
of evaluation, objectiveness in evaluation, perception of students and the instructor is positive
guidelines for students work, and transparency of towards using rubrics. The researchers noted that
evaluation increases accountability, fairness, and there is a need to conduct more research studies on
reliability. Similarly, bad practices such as vague methods and analysis in different cultural and
descriptions in making a rubric, and failure to geographic areas, establishing validity and reliability

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 21
of rubrics, and closure focus on learning (Andradeb, 5. To determine the suggestions of teachers
2010). to improve the quality of rubrics and their
Assessing higher-order course outcomes and use.
programme outcomes rubrics is an important and
reliable tool. Rubrics serve as a transparent and METHOD
inspiring guide to learning. Rubrics are scoring, or It is an exploratory type of study
grading tools used to measure a students’ encompassing all the systematic stages of this kind
performance and learning across a set of criteria and of study. There was no similar study available on the
objectives. Rubrics communicate the expectations in topic, so, researchers selected the steps from the
the assessment. There are three components within exploratory type of study. The major steps followed
rubrics namely (i) criteria/performance Indicator: in this study are stated in Figure 1.
the aspects of performance that will be assessed, (ii)
descriptors: characteristics that are associated with
each dimension, and (iii) scale/level of Literature review
performance: a rating scale that defines students’
level of mastery within each criterion (AICTE,
Finalization of title and research objectives
2018).
The rubric is defined as a learning and
assessment tool, analytical and holistic, general and
task-specific types of rubrics are defined and the Research design & Instrument design
process of constructing the rubric, use of rubric and
benefits of rubrics are defined (Chowdhury, 2019).
The benefits of using rubrics are establish Data collection, presentation and analysis
shared expectations, evaluate students work
consistently, and efficient way to assess multifaceted
examples of students work. The use of rubrics
require time and effort to develop, need to include Findings & conclusion
descriptors, maybe less precise
(Assessment@Carolina, 2017).
The analytic and holistic rubrics are
Recommendation and areas for further study
compared and on validity, reliability, and benefits.
The findings from this study provide a Figure 1: Research Methodology
complementary perspective on the role and rationale
to use both holistic and analytic approaches to
marking (Carmen Tomas, 2019). Literature Review
The rubric is defined as an assessment tool A brief review of the related literature is
that can be used with different methods of stated in point no 2 of this paper. It was useful in
instruction. The components of the rubric and finalizing the research problem based on the
models of development are described (Muhammad, literature gap, selecting the population, deciding the
2018). sample, and designing the research instrument.
Discussion: There was no research study
conducted on the use of rubrics in technical Finalization of title and research objectives
education in India which encouraged use to Some research studies and conceptual papers
critically study the use of rubrics in various aspects are available on the rubric. These papers are
of technical education in India. covering problems such as the use of scoring rubrics,
assessment rubrics, appropriate criteria for designing
RESEARCH OBJECTIVES the rubrics, the role of rubrics in advancing and
Following research objectives are formulated assessing student learning, and review of rubric use.
to critically study the use of rubrics in technical There was no study found on the use of rubrics in
institutions: technical institutions with reference to outcome-
1. To determine the status of training of based assessment which led researchers to undertake
technical teachers on designing and using this study.
rubrics for various purposes.
2. To describe the process adopted to Population
prepare the rubrics. All the faculty members of technical
institutions constitute the population.
3. To describe the process used for using the
rubrics.
4. To determine the impact of using rubrics
on the learning of students and teachers.

22 IJEA Vol. 4 No. 2, December 2021: 20 - 33


Sample received from respondents in structured and semi-
A convenient and purposive sample of structured research instrument.
technical teachers is selected because the researchers
are not having a detailed profile of all faculty 1. Status of training of technical teachers on
members. The google link was communicated to designing and using rubrics
2000 teachers out of which 315 teachers from 152 Out of 315 respondents, 49% of respondents
engineering and polytechnic colleges responded. are trained in preparing rubrics. 59% of trained
respondents received one-day training, 15% received
Design of Research Instruments two-day training, and 26% of respondents received
There is no standard research instrument three or more days of training.
available as the study is being carried out for the first
time. So the researchers designed the quantitative Purposes of using rubrics
and qualitative, and structured and semi structured Purposes of using rubrics are indicated in
instrument following the steps shown in figure 2. table 1.

Table 1: Purposes of using rubrics


Literature review Purposes Respondents
Providing feedback to students 72%
Producing learning in students 71%
Objective wise research questions
Certifying the ability of students 68%
Diagnosing learning problems of the
67%
students
Identification of parameters for each question Decision making related to
64%
instruction
Improving communication 57%
Validation of parameters
Learning about preparing and using rubrics
72% of respondents have self-learned
preparing rubrics, 74% of respondents self-learned
Design of instrument
about using rubrics, and 51% of respondents
indicated that they have learned from their
colleagues.
Validation of research instrument
Faculty members training
Figure 2: Design of research instrument Maximum 30% of faculty members received
training and a minimum of 11% of faculty members
Validation of research instruments: received training in preparing and using rubrics.
Validation of research instrument is done on 43% of respondents recommended imparting
content and construct validity by 4 experts working training to untrained teachers in developing and
in the area of curriculum development and using rubrics in their institute.
assessment
Purposes for using the rubrics
Data collection The purposes of using rubrics are mentioned
The Google Form was used to collect the data in table 2.
from respondents. The link to the google form was
emailed to potential respondents with a request to Table 2: Purposes of using rubrics
fill the google form. Purposes Respondents
Seminar presentation 81
Presentation and analysis of data Learning process assessment 79
Learning product assessment 79
The data are presented in tabular form based Quality 77
on the percentage of responses. There were Presentation 77
irrelevant responses to semi-structured questions Working in a team 73
that have not been considered. There was a Problem-solving 73
duplication of responses in different wordings which Safety abilities 69
are combined and stated in the paper. Psychomotor domain abilities 69
Leadership 63
Sustainability 52
FINDINGS Maintenance 52
Findings are noted in a conscious and Good housekeeping 43
precious way that is based on the information

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 23
Discussion Process followed for developing rubrics
The teachers are rreceiving training and using The respondents offered the following views on
rubrics for various purposes related to teaching- the process of developing the rubrics:
learning, assessment and decision making for • Followed analytical process,
improving the whole process of education. The • Followed steps such as to define goal, choose
involvement of all the teachers is increasing. rubric type, define criteria and prepared
rubric,
2. Process adopted to prepare the rubrics • Prepared scoring rubric,
67% of respondents prepared rubrics. • Identify the criteria and use a hierarchy of
learning in each domain of learning,
Types of rubrics prepared • Used previous experience and common
64% of respondents prepared 1-5 holistic sense,
rubrics, 25% prepared 6-10 rubrics, 7% prepared 11- • Followed steps such as define goal of the
15 rubrics and 4% prepared 16-20 rubrics. It is found rubric, decide rubric type, decide criteria for
that 65% of respondents prepared 1-5 analytical assessment, define performance level,
rubrics, 18% prepared 6-10 rubrics, 12% prepared
• Used 4 point scale in rubrics,
11-15 rubrics and 5% prepared 16-20 rubrics.
• Used sequential process - define the goal,
choose rubric type, identify criteria, create
Validation of rubrics
performance level, and write descriptors,
48% of respondents stated that they have
• Used sequential processes such as decide
validated the rubrics prepared by them with
project titles, set aims and objectives and
colleagues.
steps to follow while preparing the project,
then prepared rubrics by deciding criteria,
Use of criteria for defining the performance
followed by descriptors on a scale of 5 points,
75% of respondents have used the criteria for
defining the performance, 85% of respondents • Used seven steps viz. review learning
indicated that they used necessary criteria and 74% outcomes, list performance criteria, describe
indicated that they used sufficient criteria for levels of quality for each criterion, develop a
preparing the rubrics. grid, add a descriptor or numerical score to
each performance level, practice using the
rubric, share the rubric with students,
Match between the number of criteria for
measuring the performance and time assigned to • Developed holistic and analytic rubrics using
observe the standard process,
80% of respondents indicated that the time • Used general logic and experience in
assigned to assess the criteria of assessment is assessment,
sufficient. • Taken idea from online sources and adopted
it,
Appropriateness of rating/classes of performance • At first, go through the outcomes of a task,
criteria then find the key parameters to achieve that
74% of respondents mentioned that there is task, use this information to select criteria,
the appropriateness of rating/classes of performance then assign the performance levels and score
criteria. based on the nature of the task,
• Discussed with trained persons and used
Appropriateness and clarity of description for each previous data of assessment, identified
class of performance parameters for evaluation and identified the
76% of respondents mentioned that they have descriptors for each criterion to decide the
checked appropriateness and clarity of description grade,
for each class of performance. • Used own thought process,
• Used internet sources and examples of
Difficulties experienced in preparing rubrics rubrics, first for a particular course, course
The difficulties are indicated in table 3. outcomes are written then it was mapped
with program outcomes and program specific
Table 3: Difficulties experienced outcomes, again a table is formed of
Major difficulties Respondents performance level and components, likewise
Scientific method not known 73% on 5 or 10 point basis a table is formed,
Defining the descriptors 68% • Followed a sequence viz decide purpose,
Validation 67% domain, activity, scale, validation,
Identifying the criteria 58%
Defining the scale 58%
• Generally, the tutorials are given to the
students and various assessment criteria are
fixed with the quality parameter and

24 IJEA Vol. 4 No. 2, December 2021: 20 - 33


depending upon the assignment given to The rubrics are used by course teachers,
students, rubrics are formed, students, peers, and external examiners for assessing
• Used logical and analytical skills, determine the learning outcomes, inputs, processes, and
the level of performance adjectives, outputs of the learning process in such situations
identification of assessment criteria, and which can not be used by other assessment tools and
performance parameters achieved in different techniques.
ways, The students are made aware at the beginning
• Identified skills required and essential as per of the academic session about the complete
course outcomes, involved the students for assessment scheme and rubrics.
whom it is designed and utilized the The rubrics are effectively and efficiently used
experience of training, for motivating the students for learning, diagnosing
• Developed by subject coordinators then the learning problems of the students, offering
checked by quality assurance committee, and constructive feedback for improving learning, and
discussed with accreditation expert of the final assessment for awarding marks in the course
institute to finalize, and certifying attainment of course outcomes.
• Modification of online ready-made rubrics, The teacher's commitment to valid and
study through google search, learn by doing, reliable assessment is essential for productively
identifying the criteria, and prepared using the rubrics.
informative rubric, At the institute level policy related to design,
• Used brainstorming, use and improve rubrics is required to ensure
• Received readymade rubrics from uniformity which is useful for comparison,
Maharashtra Board of Technical Education, documentation, and communication.
Mumbai, referring accreditation manual, Rubrics are very effective tools in assessing
checked with rubrics available with other the performance of the student in critical abilities
similar institutions, as well as on internet like safety, housekeeping, presentation, quality,
search, ethics, and the like.
Rubrics are consistently used to see the
• Many times, took opinions from industry
progress in learning related to the specific learning
persons for performance measurement, raw
outcome.
process without forming matrix but just plain
Respondents mentioned that they want to
worded document, and not satisfied with the
learn the correct techniques of preparing and using
process used.
rubrics. They are novices in the area but the
National Institute of Technical Teachers and
Creating awareness for teachers of other
Research, Bhopal induction training programmes
institutions
developed abilities for using the rubrics for various
72% of respondents created awareness for
purposes. In outcome-based education, rubrics have
teachers of other institutes for preparing rubrics for
a very high weightage. These should be used for
different purposes.
assessing practical, tutorials, assignments, and
presentations.
Rubrics developed for assessing outcomes in a Formative assessment using rubrics must be
particular domain made compulsory in the technical education system.
83% of respondents developed and used Respondents used rubrics for assessment of learning
rubrics in assessing the cognitive domain outcomes, in the project, giving constructive feedback to
74% developed and used rubrics for assessing students, evaluating the performance of the student,
psychomotor and affective domain outcomes. assessment in experimental methods, and holistic
approach in the assessment.
Discussion: teachers are developing different
types of rubrics for achieving various purposes.
There is no fixed method adopted by teachers to Purposes of using the rubrics
develop the rubric and to assure the validity and Teachers used rubrics for achieving purposes
reliability of the rubrics. stated in table 4.
3. Process for using the rubrics Table 4: Purposes of using the rubrics
Use of ready-made rubrics Purposes Respondents
83% of respondents used 1-5 holistic rubrics, Producing learning in students 82
17% used 6-10, 4% used 11-15 and 3% used 16-20 through the rubric
holistic rubrics. Certification of learning outcome 81
70% of respondents used 1-5 analytical Diagnosing the learning problem 75
rubrics, 20% used 6-10, 6% used 11-15 and 4% used Improving communication and 75
16-20 analytical rubrics. feedback
Decision making 75

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 25
designing rubrics for assessing psychomotor
The respondents expressed that and affective domain abilities.
• Rubric preparation decides clarity in • Rubrics are used for learning and assessment
assessment based on different parameters, purposes so negative criteria of assessment
• Need to update their knowledge on method should be avoided.
of preparing rubrics, • Rubrics are suitable for relative assessment.
• Rubrics used just for convenience to evaluate Scaling must be dynamic in rubrics.
but no particular domain has been identified
and selected, Respondents have stated difficulties in general
• Rubrics are used for creating interest in terms which are classified by researchers on system,
learning, analyzing and calculating design, implementation, and interpretation aspects.
attainment,
• Rubrics are used to let students know how System-related difficulty
they have scored for given practical, Institute is implementing the rubrics for
• They use the rubric to inculcate the desired formative and summative assessment but students
skills among the students, do not understand rubrics. More time is required for
• In architecture courses rubrics are useful, the preparation of rubrics, using rubrics, and
• Do not use readymade rubrics, analyzing the results in comparison to traditional
assessment. It is difficult to make rubrics for
• Use rubrics for assessing laboratory work and
numerical problems. Faculty members think that it
reduce grading time,
is not their job. It is a time-consuming process. The
• Use rubrics for steadiness in assessment and
validation process for finalizing the rubric is not
validation for skills developed,
clear. The scientific approach is not known. Difficult
• Used rubrics during conducting to prepare scoring rubrics.
extracurricular activities and after completion
of the session,
Design related difficulty
• In the fast-developing era, it is very essential Deciding the valid and reliable criteria,
to design and use rubrics that will diagnose descriptors, and scale with respect to learning
the learning problems, outcomes is very difficult. Proper articulation of
• Rubrics should be prepared to develop and criteria and descriptors is very difficult. It requires
assess learning outcomes creative and analytical ability. Making rubric
balanced in terms of marks allocation to criteria and
Difficulties experienced while using rubrics descriptors. It is a time-consuming process. Difficult
The comments of different respondents are stated to decide the levels in the rubrics. Validation of self-
below: prepared rubrics is difficult and takes time. Fixing
• It is difficult to obtain the correct criteria in criteria and rating scale. Defining and scaling the
the rubrics and calculate reliability. correct rubric.
• In many criteria even after proper definition,
different assessors will give different marks. Implementation related difficulty
So, preparing a rubric to get an almost similar Measuring the outcome and connecting it
assessment is a big challenge. with feedback to students is a difficult task. Students
• Faculty members require training and oppose the use of rubrics because they get fewer
practice for preparing rubrics on a case marks in comparison to paper and pencil tests.
assignments basis. Students are unaware of the concept of rubrics, the
• A proper thought process is required for process of using the rubric, and the purpose of the
making rubrics. A scientific method to define rubric in learning and assessment. Difficult to make
a rubric for a particular type of course is students understand the utilities of rubrics. Difficult
unknown to us. to measure the performance of students in a class of
• Preparing and using rubrics is a time- 60 students. Students are not giving importance to
consuming process. rubrics in achieving learning outcomes. The use of
• Rubric preparation for each subject is rubrics by all faculty members is difficult and time-
important to assess students’ performance. consuming. Some difficulties in diversified courses.
• Even though rubrics are used in outcome- It is a very confusing and time-consuming process.
based education, it is not implemented
properly for harnessing the full potential of Interpretation related difficulty
rubrics for producing and assessing learning Rubrics are subjective if the assessor is biased.
outcomes. Rubrics are judgmental in the assessment of affective
• The format for preparing the rubric should be domain abilities. Implementing the outcome of
standardized at the institute level. It is not rubrics in the form of feedback is difficult. All faculty
clear on a number of criteria to be used for members are not using the same approach for using

26 IJEA Vol. 4 No. 2, December 2021: 20 - 33


rubrics in the assessment. Analysis and description sufficient criteria, significant and direct
of results are difficult. Difficult to allocate the marks criteria, measurable and observable criteria,
to students. Some students are not bothered about fact-based, and easy to administer criteria.
assessment and marks in a course. The descriptors for each criterion are
Grievances: Respondents stated that there are designed using the principles which are used
some grievances of teachers related to an overload for deciding the criteria of rubrics. The rubrics
of work, non-availability of resources, lack of are designed following the standard process
recognition for using the rubrics, and non- at the institute level. The rubrics are validated
availability of guidance. They have indicated that a by peers to ensure content and construct
grievance handling mechanism should be developed validity.
for students related to the assessment of learning.
Reliability of rubric
Quantity of teachers use rubrics 76% of teachers used rubrics providing
The percentage of teachers and percentage of consistency in scoring.
teachers who use rubrics in their institute is Rubrics are multidimensional sets of scoring
indicated in table 5. guidelines that can be used to provide consistency in
evaluating students learning. The reliability of the
Table 5: Percentage of teachers who use rubrics rubrics depends on the type of rubric. When two
Percentage of Percentage of different graders use a reliable rubric on the same
respondents teachers use performance, they will give similar scores. This is
rubrics called inter-rater reliability. The rubric should also
20% 1-10% yield consistent results if one grader uses the same
11% 11-20% rubric to judge the same performance at different
12% 21-30% times. Rubrics spell out scoring criteria so that
5% 31-40% multiple teachers using the same rubric would arrive
6% 41-50% at the same score. If rubrics are well prepared
5% 51-60% considering all the concern domains of learning then
5% 61-70% it helps to ensure reliability. The reliability of the
9% 71-80% rubrics may be assured effectively only when it is
scientifically designed.
3% 81-90%
Respondents indicated that they use methods
5% 91-100%
such as using the rubric on a few students and
improving it, using the trial and error method to
11% of respondents stated that they have no improve reliability, validating it with colleagues and
idea about the number of teachers using rubrics, faculty members of other institutions, using
20% stated that they use rubrics for assessing the analytical tools, checking criteria, considering the
learning in projects. course outcomes to be measured, using scalable
measurement approach, field testing, continuously
Process of ensuring validity, reliability and practicing and improving, analyzing the outcomes,
usability increasing objectivity, using feedback of students
Validity of rubric after administering, using the cross-verification
• 56% of respondents used professionally approach.
designed and validated rubrics.
• 74% of respondents indicated that they have
checked the appropriateness of rating/classes
of performance criteria (or column headings Rubric as a self-assessment tool
of the rubric table). 81% of teachers encourage students to use the
• 76% of respondents indicated that they have rubric as a peer assessment tool.
checked the appropriateness and clarity of Rubric as a peer assessment tool
descriptors for each class of performance for 72% of teachers encourage students to use the
each criterion as stated in different cells of the rubric as a peer assessment tool.
rubric.
• 81% of respondents indicated that the Outcomes assessed using rubrics
performance to be measured is observable The respondents stated that they assess
• 80% of respondents indicated that it is learning aligned to course outcomes and unit
measurable. 80% of teachers used valid learning outcomes in cognitive, psychomotor, and
rubrics. affective domains using rubrics.
• Rubrics are scientifically designed Different teachers assessed different
considering the learning outcomes, using outcomes related to students’ project work, practical
action verbs, objective criteria, necessary and work, teamwork, overall growth, conceptual clarity

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 27
on a topic, communication skills, leadership, 44% respondents stated that they use
interpersonal skills, safety, quality, sustainability, professionally designed and validated rubrics.
design, drawing, critical thinking skills, model
building, report writing, research, and ability to Rubrics measure for which it indented to measure
introduce the topic. 79% respondents stated that rubrics measure
for which it intendent to measure.
Type of task assessed
Respondents mentioned a wide range of tasks Consistency in assessment among assessors
and activities in which they use the rubrics. They 76% respondents stated that there is a
used rubrics in project work, seminar presentations, consistency in scoring among assessors.
role-play, practical, and group discussions,
assignment, quiz, problem-solving, and working in Discussion: Teachers are using the rubrics
a team. They used rubrics for assessing input, for achieving various purposes. At the same time
process, and product outcomes. They used rubrics in they are facing some difficulties and barriers in using
formative and summative assessments. the rubrics.

Barriers in using rubrics 4. Impact of using rubrics on learning of students


It is found that 26% of respondents The impact of using the rubrics is classified
mentioned that as such there is no major barrier to on significant parameters and briefly described in
design and use the rubric in their institute. subsequent paragraphs.
Respondents have listed barriers they have faced in
using rubrics. Generic impact
• There are no proper guidelines and Respondents stated that use of rubrics resulted in
infrastructure to train the students in • accurate measurement and assessment of
activities other than curriculum, learning with respect to pre-decided learning
• Innovative concepts are lacking in the outcomes,
curriculum, • easy to assess the learning of the students,
• Student batch size in theory and practical • significant improvement in demonstration of
classes are very high, ability by students,
• Initially time-consuming process, • clear outcome-based analysis,
• Every course needs its own way, • effective feedback to students on learning,
• Lack of awareness about the benefits of • increased confidence for performance,
rubrics among students and teachers, • increase in uniformity and reliability in
• Use of rubrics is not accepted by other faculty assessment,
members, • achievement of course outcomes established,
• Lack of support from technical staff, • development of attitude for life-long learning
• Paucity of time, in students,
• Less support and no guidance from the higher • development of soft skills,
authorities, • best tools for assessing any type of task
• Lack of ability to use the rubrics, without any partiality and penalising,
• High expectations of teachers, • increased transparency in assessment
• Low maturity of students in assessment, • increased performance of students,
• Difficulty in the interpretation of results, • a better understanding of exactly what to
• Difficult to prepare, learn,
• Articulation of criteria and descriptors is • uniform policy for assessment,
difficult, • students motivation to learn,
• Traditional assessment scheme having no • reduced complaints related to assessment,
provision for rubrics, • involvement of students in the learning
• No encouragement, and appreciation for process,
using rubrics. • identification of areas for improvement,
• self-regulation for learning,
Awareness for teachers using rubrics • the clarity in assessment and reduction in
72% of respondents stated that they create random scoring,
awareness in teachers of their institute to use the • ease for teachers in progressive assessment,
rubrics for different purposes. • understanding in students about their
achievements and deficiencies,
Use of professionally designed and validated
• time management,
rubrics
• improvement in flexibility in assessment

28 IJEA Vol. 4 No. 2, December 2021: 20 - 33


• simplification in the assessment process, learning, affective domain learning, and
• improvement in quality of learning, psychomotor domain learning.
performance, and teaching-learning process, Respondents indicated that rubrics are being used
• uniformity in decision making irrespective of for achieving a wide variety of purposes related to
time, the learning of the students such as
• removing biases in assessment, • Identifying the learning gaps and mislearning
• increase in fairness in assessment, of the students,
• same yardstick for all students, • Improving learning skills of the students,
• increase in introspection, • Providing feedback for improvement in
• increase in objectivity, validity, and reliability learning,
of assessment, • Developing observational and assessment
• documentation of assessment process, skills in students through self-assessment and
• students earn better grades and feel less peer assessment,
anxious. • Using the student-centric teaching-learning
process,
Impact on the learning process • Improving the quality of rubrics based on the
Respondents stated that rubrics have experiences
impacted the learning process in aspects such as • Creating rubric bank at the programme level,
improvement in learning outcomes, increase in • Standardizing the process of formulation of
student motivation and commitment for learning, rubrics,
increase in ability, overall development of the • Overcoming barriers to innovation in
students, and increased competition for learning assessment,
among students. • Making rubric as motivational tool, learning
problem diagnosis, assessment and decision
Impact on learning outcomes making tool,
Respondents stated that rubrics familiarized • Reviewing curriculum based on the
students for achieving the highest level of outcomes, experiences of assessment,
fostered critical thinking, improved observational • Using the rubric as a guidance and
and communication skills, refined course outcomes. counselling tool,
• Increasing transparency in assessment
Impact on assessment
Systematic assessment integrated with the Experience sharing workshops
learning process, accurate assessment of learning 59% of respondents organized experience-
with justification, effective decision making related sharing workshops in their institute on rubrics.
to learning, personal biases of assessors reduced, There is no fixed frequency of organizing workshops
clarity in assessment has increased, the involvement but it is organized on yearly basis.
of students in assessment increased in self and peer
assessment. Modification of rubric after using
59% of respondents modified the rubric after
Impact on the feedback process using it.
Rubrics helped students to get immediate
feedback for improvement in learning, identifying Impact of rubrics as a self-assessment tool
strengths and weaknesses. Respondents stated that there is an
improvement in; encouragement of self-learning as
Analysis of rubric at institute and programme level they know the criteria of assessment at the beginning
60% of respondents stated that they analyse of the learning process, identifying the weak areas
the rubrics at institute and programme levels to which require more learning efforts, customized
improve the design and use of learning for achieving evaluation, positivity towards assessment,
various purposes. satisfaction of students in assessment, preparing
course plan for the next cycle of learning, formative
Significant decisions are taken based on the and summative evaluation of learning of students,
analysis of rubrics validity in assessment, providing information for
Decisions related to the preparation of learning and assessment, and improving the
assessment policy, assessment guidelines, and teaching-learning process,
documentation are taken at the institute and Respondents stated that there is an
programme levels. improvement in; observational and assessment
The use of rubrics is being made compulsory skills, feedback providing skills, analysis and
for assessing project-based and problem-based interpretation skills, reflective skills,

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 29
Respondents stated that there is an Suggestions Respondents
improvement in the design of learning assignments There should be a mentor teacher to
and practicals for the students, quality of the guide others in preparing and using 91%
the rubrics
teaching-learning process, transparency in Students should be trained in using
assessment, the involvement of students in the the rubrics for learning and 91%
assessment process which increases their interest in assessment of performance
learning, design, and use of rubrics, accuracy of Teachers should be encouraged to
91%
assessment, increase in introspection, earn better publish the tried out rubrics
grades, creating an ecosystem for learning, Institute should create a rubric bank 89%
providing 360-degree feedback for improvement and
recognition of achievements. Discussion: Teachers developed competency and
It is found that 14% of respondents have not proficiency to prepare and use the rubrics for
used the rubric as a self-assessment tool and 12% of assessing various learning outcomes of the
respondents have not observed the impact of the students. They have made many suggestions to
rubric from a self-assessment point of view. improve the process of preparing and using the
rubrics at institute level.
Impact of the rubric as a peer assessment tool
Respondents stated that there is an increase CONCLUSION
in; involvement of students in learning and The conclusions are drawn based on the
assessment process using rubrics, students resolve structured data and unstructured information
their learning problems on their own, peer feedback provided by respondents. It is concluded that:
is more acceptable than feedback given by teachers,
academic discipline of the students have improved, Training and development
positive or negative favouritism in awarding marks Teachers are formally trained in developing
reduced significantly, quality of learning, and using rubrics, other teachers are self-learning
transparency of assessment, time-saving related to and learning from trained teachers. Trained teachers
formative assessment, collaborative learning, trust are creating awareness for designing and using
in assessment, the learning capacity of individuals, rubrics. An environment for developing and using
comparison in achievement among students, rubrics for achieving various purposes related to
rationale, constructive, positive, appreciation, learning and assessment is being created in
progressive reliable assessment, improvement in institutions.
group learning, the confidence level of students in
assessing the performance of others, group learning, Purposes of using rubrics
team learning, cooperative learning, Teachers are using rubrics for achieving
communication skills among students, coordination various purposes related to learning and assessment
between instruction and assessment, observation, such as motivating students for learning, diagnosing
life long learning skills, time-saving and self- learning problems, taking self-feedback on learning,
satisfaction. receiving feedback from peers for improving
Discussion: The impact of the use of rubrics learning, decision making for improving the
is visible on many aspects of instruction and instruction, improving communication, and
assessment in technical education institutions. assessment of performance. They are using rubrics
in formative and summative evaluation. Rubrics are
5. Suggestions to improve quality and use of not being used for harnessing their full potential for
rubrics in the institute achieving various purposes by all teachers of the
The suggestions are stated in table 7. institute. Different respondents use it for achieving
different purposes.
Table 6: Suggestions to improve quality and use of
rubrics Abilities assessed using rubrics
Suggestions Respondents Rubrics are being used for assessing more
All teachers should be formally than 12 learning outcomes by more than 50% of
trained in preparing and using the 95% respondents. These outcomes are related to safety,
rubrics
There should be experience sharing
housekeeping, sustainability, quality, maintenance,
sessions on the development and use precision, professional ethics, communication and
94% presentation, leadership, and working in a team.
of rubrics every six months in the
institute Teachers are using rubrics for assessing cognitive,
There should be a guideline affective, and psychomotor abilities. They are using
document to develop and use the rubrics for students' project work, seminars, and
94%
rubrics of the teaching-learning
process
assessments. They are using rubrics with
There should be a rubric validation instructional methods such as projects, role-play,
92%
process in the institute

30 IJEA Vol. 4 No. 2, December 2021: 20 - 33


group discussion, practice, and workshops for the There are barriers to designing and using
learning and assessment of students. rubrics at the institute level. These barriers are
related to training, awareness, policy, resistance,
time, support and guidance, maturity of teachers
Types of rubrics and students in using rubrics, and assessment
Teachers prepared holistic and analytical scheme of the institute.
rubrics to assess cognitive, affective, and
psychomotor domain outcomes. They have used Impact of using rubrics
criteria for preparing the rubrics. They have used The use of rubrics impacted improvement in
observable and measurable criteria which are the learning process, learning outcomes, assessment,
necessary and sufficient to assess the learning and feedback. The impact of the use of rubrics is
outcomes. They have checked the criteria for observable on many dimensions related to the
appropriateness and clarity. They have validated the teaching-learning process, student learning,
rubrics with their colleagues. Teachers are using administrative decisions, satisfaction of students
appropriate ratings and descriptors for preparing and teachers, reduction in grievances related to
rubrics. assessment, and involvement of students in the
assessment process. The use of rubrics as self and
Process of preparing rubrics peer learning tools impacted improvement in the
Different teachers followed the different learning process and assessment process.
processes of preparing the rubrics. They have not
used a standard and scientific process for preparing Decision taken
the rubrics. They face difficulties such as defining Teachers have taken many decisions related
the descriptors, validation, identifying the criteria to the design and use of rubrics to improve the
and defining the scale. Some teachers prepare teaching-learning process and assessment process.
original, others copy and adopt. At the institute level, no policy decision is reported
Teachers are not following the scientific on the basis of an analysis of preparation and use of
method of ensuring validity, reliability, and usability rubrics by teachers. There is a scope to
of the rubrics at the institute level. Different teachers institutionalize the use of outcomes-based
are using their own methods for ensuring the assessment and use of rubrics in technical
validity, reliability, and usability of rubrics. Rubrics institutions.
are used as a self-assessment and peer assessment
tool. RECOMMENDATIONS
At the institute level, there is no system to The teachers are designing and using rubrics
ensure the validity and reliability of rubrics. for achieving various purposes related to improving
Therefore, teachers are using their own way in the learning process and assessment but it is not
ensuring validity and reliability. The teachers are being done at full scale and in a scientific manner.
weak in ensuring the validity and reliability of To harness the full potential of the rubrics for
rubrics. learning and assessment following
recommendations are noted.
Use of rubrics
In different institutions, holistic and Policy and guidelines
analytical rubrics are used by a different number of At the institute level, comprehensive policy
teachers. Teachers prepared and used 1-20 related to the design and use of rubrics should be
analytical and holistic rubrics for achieving learning formulated and implemented to ensure the quality
purposes and assessing the learning of the students. of assessment and uniformity in the assessment.
Teachers used outcomes related to cognitive, At the institute level guideline document for
affective, and psychomotor domains of learning. designing and using rubrics should be prepared to
They have used rubrics in teaching-learning ensure valid, reliable, and productive assessment. It
methods such as practical, industrial training, should be communicated to teachers and students
students project, group discussions, seminars, and and implemented to ensure quality in assessment.
role play. They used rubrics for formative and All teachers and students should be made
summative evaluation. Teachers used rubrics for aware of the policy and guideline document.
assessing performance in students' activities, tasks, At the institute level use of outcome-based
and performance. assessment should be institutionalized. In outcome-
based assessment, the significance of the use of
Difficulties and barriers rubrics for learning and assessment of learning
Teachers are facing system, design, should be projected. In addition to assessment by the
implementation, and interpretation-related course teacher, self-assessment, and peer assessment
difficulties in preparing and using the rubrics. should be given due importance as both are

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 31
powerful and direct methods of improving learning assessing activities, tasks and assignments, using
for students. rubrics in teaching-learning methods like role play,
At the institute level, various types of rubrics group discussion, seminars, practical, projects, and
such as generic and specific, holistic and analytical, presentations.
progressive scale and self-feedback providing for At the institute level, support and guidance should
common course outcomes should be prepared and be provided to teachers and students for using the
included in the assessment scheme. rubrics effectively and efficiently for productive
learning and assessment. At the institute level,
Training and development shortcuts for preparing rubrics should be prevented
et the result from the learning and assessment as it affects the validity and reliability of assessment
point of view for individual students and groups of in outcome-based education.
studentIn-house training capacity should be
developed in technical institutions to train the Research study
teachers to design and use the rubrics for achieving At the institute level, action research,
various purposes related to students learning, analytical research, and impact studies should be
assessment, and administrative purposes. conducted to improve the quality of the assessment
Teachers should be encouraged to use the process in all the academic programmes and
rubrics for achieving various purposes. They should research studies. The impact of preparation and use
be encouraged to assess a variety of abilities of of rubrics should be assessed to transfer the
students as being assessed by respondents. experience of preparing and using rubrics in sister
At the institute level, a workshop of teachers institutions.
should be conducted to guide and facilitate the
process of rubric design and to validate the rubrics. LIMITATIONS OF THE STUDY
Teachers should be given the opportunity to use the The study is conducted on a very small
rubric and interprs. sample size using convenience sampling and
purposive sampling itself is a limiting factor to
Experience sharing workshop generalize the findings. The respondents mentioned
At the institute level, experience-sharing that at the institute level no systematic policy,
workshops on rubrics should be conducted every guidelines and processes available to prepare and
year to enrich the collective learning on rubric use the rubric itself is an indication of the weakness
design, use, and analysis. of the rubric system.

Documentation 10 AREAS FOR FURTHER STUDY


At the institute level documentation of There is a scope for conducting a research
experiences related to design, use, and analysis of study on models of conducting training of millions
the rubric should be carried out for further learning of teachers within a one-year period to harness the
and improving institutional capabilities. full potential of rubrics in academics and research.
There is a need to evolve the process of continuous
Awareness improvement in the design and use of rubrics at the
The students should be made aware of the institutional level. As for validity, reliability, and
assessment scheme and the role of rubrics in the self- usability is a significant and important matter in
learning, peer learning, and assessment process. assessment so to develop a generic model for
Their doubts and apprehensions related to the use of ensuring validity, reliability, and usability should be
rubrics should be removed. The students should be developed.
trained to use the rubrics for self-assessment, and
peer assessment. They should be trained to take self-
feedback and offer feedback for improving their
performance.
Teachers should be guided and mentored by ACKNOWLEDGEMENT
experts to prepare and use the rubrics as they find it The authors would like to sincerely thank the
difficult to identify the criteria and descriptors. respondents for openly sharing their experiences
Teachers should be trained and encouraged related to the design and use of rubrics.
to ensure the validity and reliability of the rubric
with respect to the learning outcome. Teachers *****
should be encouraged to use the scientific method of
ensuring the validity and reliability of the rubrics. REFERENCES
AICTE. (2018). Examination Reforms. All India
Encouragement and incentive Council for Technical Education, New
Teachers should be encouraged to use the Delhi.
rubrics for formative and summative assessment,

32 IJEA Vol. 4 No. 2, December 2021: 20 - 33


(2017). Assessment@Carolina. The University of Process. Journal of University Teaching &
North Carolina at Chapel Hill. Learning Practice, 18(4), 1-13.
Anders Jonsson, G. S. (2007). The Use of Scoring Ho, Z. C. (2019). Good and Bad Practices in Rubric:
Rubrics: Reliability, Validity and The Perspectives of Students and
Educational Consequences. Educational Educators. Assessment and Evaluation in
Research Review, 2, 130-144. higher Education, 44(4), 533-545.
Andradeb, Y. M. (2010). A Review of Rubric Use in Jason K. Morton, M. N. (2021). Sharing the
Higher Education. Assessment & Evaluation Construction of Assessment Rubrics with
in Higher Education, 35(4), 435-448. Students: A Model for Collaborative Rubric
Brookhart, S. M. (2018). Appropriate Criteria: Key Construction. Journal of University Teaching
to Effective Rubrics, Volume. Frontiers in & Learning Practice, 18(4), 9-13.
Education, 3(22). Jacobo, D. M. (2019). Grading for Growth: Using
Carmen Tomas, E. W.-H. (2019). Modeling Holistic Sliding Scale Rubrics to Motivate
with Analytic Rubrics. Frontiers in Struggling Learners. Interdisciplinary
Education, 4(89), 1-19. Journal of Problem based Learning, 13(2),
Chowdhury, F. (2019). Application of Rubrics in the 1-28.
Classroom: A Vital Tool for Improvement in Martino Andreani, D. R. (2020). Shadows over
Assessment, Feedback, and Learning. Accreditation in Higher Education: Some
International Education Studies, 12(1), 61- Quantitative Evidences. Higher Education,
68. 79, 691-709.
Dawson, P. (2020). Assessment Rubrics: Towards McCormick, M. L. (2021). Constructing Analytic
Clearer and More Replicable Design, Rubrics for Assessing Open-Ended Tasks in
Research, and Practice. Assessment & the Language Classroom. Teaching English
Evaluation in Higher Education, 247-260. Learning as a Second or Foreign Language,
Retrieved from 24(4), 1-19.
http://www.tandfonline.com/doi/full/10.1 Nurhuda, N. S. (2021). The Effect of Problem-Based
080/02602938.2015.1111294 Learning with an Analytical Rubric on the
Muhammad, A. L. (2018). Rubrics as Assessment, Development of Students’ Critical
Evaluation, and Scoring Tools. International Thinking Skills. International Journal of
Journal of Academic Research in Business Instruction, 14(2), 665-684.
and Social Sciences, 8(10), 1417–1431. Nurhuda, N. S. (2021). The Effect of Problem-Based
NAAC. (2017). Accreditation Guidelines. National Learning with an Analytical Rubric on the
Assessment and Accreditation Council, Development of Students’ Critical
Bangalore. Thinking Skills. International Journal of
NBA. (2013). Self-Assessment Report. National Instruction, 14(2), 665-684.
Board of Accreditation, New Delhi. Simona Laurian, C. J. (2013). Effects of Using
Stevens, K. W. (2007). The Role of Rubrics in Rubrics in a University Academic Level
Advancing and Assessing Student Learning. Romanian Literature Class. 5th
The Journal of Effective Teaching, 7(1), 3-14. International Conference EDU-WORLD
David C. Leader. (2018). Student Perceptions of the 2012 - Education Facing Contemporary
Effectiveness of Rubrics. Journal of Business World Issues (hal. 431-440). Procedia
and Educational Leadership, 8(1), 86-99. Social and Behavioral Science.
Doug Czajka, G. R. (2021). A Novel Rubric Format
for Providing Feedback on Process Skills to
STEM Undergraduate Students. Journal of
College Science Teaching, 50(6), 48-56.
Grainger, P. (2021). Enhancing Assessment
Literacies through Development of Quality
Rubrics using a Triad based Peer Review

A CRITICAL STUDY ON THE USE OF RUBRICS IN TECHNICAL INSTITUTIONS OF INDIA.


B. L. Gupta, Pratibha Bundela Gupta 33

View publication stats

You might also like