Instructional

Illuminators:
Scoring Rubrics for
Speaking Skills
TESOL Arabia 2022, Dubai
Nara Avtandilyan
Community College of Qatar
 Objectives
● Rationale of my research
● Basic concepts of scoring rubrics
● Involving students in the assessment process
● The appeal of rubrics for teachers and students
● Research findings
● Q&A
“Speaking skills are an important part of the
curriculum in language teaching, and this makes
them an important object of assessment as
well.”
(Luoma, 2000)
 Research Questions

1. What are teachers’ and students’ perceptions of

the use of rubrics as a speaking assessment aid?
2. Can students use rubrics to self-assess
accurately?
• I don't really understand what we're
supposed to do for this project.
• What's the point of this assignment
anyway?
• Why didn't I get an A on this
presentation? I worked very hard on it…
• It's not my fault I didn't get a good
grade!
 Rubric Matrix

Rating Scale Rating Scale Rating Scale

Criteria Indicator Indicator Indicator

Criteria Indicator Indicator Indicator

“A rubric is a scoring tool that lists the criteria for a piece of work
and one that articulates gradations of quality for each criterion, from
excellent to poor.”
(Andrade, 2000)
Characteristics of a Good Scoring Rubric

CRITERIA

LEVELS OF PERFORMANCE

GRADATIONS OF QUALITY

CONTINUITY

RELIABILITY

VALIDITY
(Moskal & Leydens, 2000)
 Analytic Rubrics Holistic Rubrics

Components:
● Task description ● A single scale with all criteria to be included in the
● Assessment criteria evaluation being considered together
● Performance levels ● Based on an overall judgment of student work
(Adopted from Mertler, 2001)
Richardson (2001)
Involving Students in the Assessment Process

“The art of negotiable contracting consists of giving

students shared ownership in their own learning.”
Wiggins (1993)
Rubrics have the potential to act
as “instructional illuminators”.
Popham (1997)
 Why do teachers and students like Rubrics?
Teachers:
● Very practical
● Make scoring easier
● Accommodate
heterogeneous classes
● Provide feedback and
accountability to students
● Ensure scoring reliability
across multiple students
Students:
● Able to see criteria for
success
● Able to see strengths and
weaknesses
● Receive feedback
● Engaging and motivating
● Sense of stability across
multiple assignments
Andrade (2000)
 Methodology
Instructors distributed
scoring rubric to student-
participants in three
groups.

Group 1 Group 2 Group 3

(18 participants) (14 participants) (16 participants)
1) Ss are given a 1) Ss are given 1) Ss are given
scoring rubric to take scoring rubric to take scoring rubric
home as a guide for home as a guide for checklist to take
preparing for the preparing for the home as a guide for
upcoming presentation. upcoming preparing for the
2) Ss did self- presentation. upcoming
assessment using 2) Ss carried out presentation.
scoring rubric. peer evaluation 2) Ss carried out
3) Ss completed during the peer evaluation during
student questionnaires. presentation the presentation
3) Ss completed 3) Ss completed
student student
questionnaires. questionnaires.
 Results
Did you use the rubrics while get-
ting ready for the presentation? Know what aspects to
94%
4%2% work on

19% Understand teacher's

96%
assessment

52%
Know what to do 92%

23%

Be graded fairly 83%

sometimes very frequently

seldom not at all Prepare better 83%
not really

Figure 1: Use of rubrics Figure 2: Benefits of using rubrics 13

 Positive and negative aspects of using rubrics
 “Rubrics helped me to organize
and plan my presentation well,
as I concentrated on general
points important to the
presentation.”
 “I knew the aspects that
I must work on to get an
organized presentation.”
 “Rubrics are like a guide
providing information
about general requirement
of presentation.”
 “It completely clarifies what
someone has to do to prepare a
presentation, in what direction
to work on and to which points  “Using rubrics makes it
to pay attention to.”
 “Through rubrics I
knew what the
teacher expected of
me”.
 “What I liked more about
rubrics is that it helps students
to know in advance how the
presentation will be graded…”
 “Rubrics clarify the
process of assessment.”
easy to evaluate peers, as
well as to do self-
assessment.”
 “…It is just a form,
nothing more; a form
which helps students to
understand how the
instructor will evaluate
them.”
 Table 2a: Correlations  Table 2b: Students’ grades compared with those of
the teacher
Grade N -1 Total

Underestimate 7 (39%) 7 N=18

d

Exact 11 0 -Missing
(16%) response

Overestimated 0 0
Teacher Questionnaire and Interview Results
 Conclusion

 Rubrics are powerful tools for both teaching and

assessment.
 Rubrics help students become more thoughtful
judges of the quality of their own and others‘ work.
 Teachers need training in the field of language
assessment.
 Rubrics are positive addition to a teacher’s methods.
 Future Directions

• Involving students in creation

of scoring rubrics
• Impact of student self and
peer assessment through
scoring rubrics on teaching
techniques, classroom activities
and curriculum development
 References

• Andrade, H. G. (2000). Using rubrics to promote thinking and learning. Educational Leadership,
57, 5.
• Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of
pedagogy. ASHE-ERIC Higher Education Report, 27 (1). Washington, DC: The George
Washington University, Graduate School of Education, Human Development.
• Luoma, S. (2004). Assessing speaking. Cambridge. Cambridge University Press.
• Mertler, C. A. (2001). Designing scoring rubrics for your classroom. Electronic Journal: Practical
Assessment, Research & Evaluation, 7(25).
• Moskal, B. M. & Leydens, J. (2000). Scoring rubric development: validity and reliability.
Electronic Journal: Practical Assessment, Research & Evaluation, 7(10).
• Moskal, B. M. (2003). Recommendations for Developing Classroom Performance Assessments and
Scoring Rubrics. Electronic Journal: Practical Assessment, Research & Evaluation, 8(14).
• Popham, J. (1997). What’s wrong and what’s right with rubrics. Educational Leadership, 55, 72-
75.
• Richardson, E. (2001). Scoring performance assessments: checklists, rating scales and rubrics.
• Starr, L. (2000). Creating rubrics: tools you can use.
• Stix, A (1997). Creating rubrics through negotiable contracting and assessment.
• Tierney, R. & Simon, M. (2004). What's still wrong with rubrics: focusing on the consistency of
performance criteria across scale levels. Electronic Journal: Practical Assessment Practical
Assessment, Research & Evaluation, 9(2).
• Wiggins, G. (1993). Assessment: authenticity, context and validity: Assessing Student
Performances. Phi Delta Kappan, 75, 3, pp. 200-214. San Francisco: Jossey-Bass Publishers.
Thanks!
Any questions?
You can find me at:
● Narine.Avtandilyan@ccq.edu.qa
 Table 1: Template for analytic rubrics

Beginning Developing Accomplished Exemplary

Score
1 2 3 4
Criteria Description Description reflecting Description Description
#1 reflecting movement toward reflecting reflecting highest
beginning level mastery level of achievement of level of
of performance performance mastery level of performance
performance
Criteria Description Description reflecting Description Description
#2 reflecting movement toward reflecting reflecting highest
beginning level mastery level of achievement of level of
of performance performance mastery level of performance
performance
Criteria Description Description reflecting Description Description
#3 reflecting movement toward reflecting reflecting highest
beginning level mastery level of achievement of level of
of performance performance mastery level of performance
performance
Criteria Description Description reflecting Description Description
#4 reflecting movement toward reflecting reflecting highest
beginning level mastery level of achievement of level of
of performance performance mastery level of performance
Table 2: performance

Template for Holistic Rubrics

Score Description
Demonstrates complete understanding of the problem. All requirements of task are
5
included in response.
Demonstrates considerable understanding of the problem. All requirements of task are
4
included.
3 Demonstrates partial understanding of the problem. Most requirements of task are included.
2 Demonstrates little understanding of the problem. Many requirements of task are missing.
1 Demonstrates no understanding of the problem.
0 No response/task not attempted.
(Adopted from Mertler, 2001)