Professional Documents
Culture Documents
Unit 4.Testing and Assessment in EFL - Assessmet instruments
Unit 4.Testing and Assessment in EFL - Assessmet instruments
Assessment instruments
in TEFL
Index
Scheme 3
Key Ideas 4
4.1. Introduction and objectives 4
4.2. Making the learning goal visible 4
4.3. Objectives 7
© Universidad Internacional de La Rioja (UNIR)
4.4. KWL 9
4.5. Graphic organisers and checklists 11
4.6. Rubrics 17
4.7. Bibliographical references 25
In Depth 27
Test 29
Scheme
© Universidad Internacional de La Rioja (UNIR)
Assessment can, and should, be incorporated into the classroom in a way that
maximises learning while at the same time providing a framework that allows
teachers to understand where learning is at, and what more needs to be achieved.
Teachers can incorporate a number of tools that help them achieve this easily and
effectively. In this unit, we will look at Gagné’s nine-step instruction and using
objectives, graphic organisers, checklists, and rubrics. We will see the characteristics
of the different tools, how to adapt and construct them for use in our own
teaching/learning context and read about case studies where the tools have been
successfully implemented to the benefits of the students’ learning.
Recognise the characteristics of the different tools we can incorporate into the
classroom to facilitate assessment and learning.
Incorporate the different tools into our teaching practice.
Adapt the tools to the needs of our learners.
Construct the tools to the needs of our learners.
Take this analogy. You get in your car, and someone tells you that you need to get to a
certain destination, but that destination is not shared with you. How can you arrive there
if you do not know where it is you must get to? The same happens in a classroom when
the teacher decides the objectives of the lesson, but the students do not know what
these objectives are. The learner is rendered powerless because they cannot take an
active role. In this situation we have a very skewed power balance between the learner
and the teacher.
Figure 1. The teacher / student power balance in the traditional classroom. Source:
https://ca.wikipedia.org/wiki/Fitxer:Johnny-automatic-scales-of-justice.svg
As teachers, we can redress by making the learning process visible. In this unit we will
© Universidad Internacional de La Rioja (UNIR)
Gain attention.
Inform learner of objectives.
Stimulate recall of prior knowledge.
Present stimulus.
Provide guidance.
Elicit performance.
Provide feedback.
Assess performance.
Enhance retention and transfer.
In the figure below (figure 2) we can see how the steps have been presented as staircase.
The idea is that each step we take as teachers increases the effectiveness instruction
© Universidad Internacional de La Rioja (UNIR)
4.3. Objectives
Objectives are also referred to as student learning outcomes, precisely because we are
interested in the outcome, or result, of learning.
Objectives may have varying names, but they all basically provide the same function:
that of providing direction. And as with all learning, all assessment also needs a
purpose and a direction to be effective:
“All tests are for a purpose. A test that is made up without a clear idea of
what it is for, is no good” (Ingram, 1968; cited in Fulcher, 2010).
Provides direction.
Empowers the learners.
Motivates.
Increases metacognition and self-regulation of learning.
Enables self-evaluation.
© Universidad Internacional de La Rioja (UNIR)
As mentioned earlier, pupils should understand not only the learning objectives but
also how they are going to get there. Success criteria should be broken down into a
“Criteria and goal setting with students engage them in instruction and the
learning process by creating clear expectations. In order to be successful,
students need to understand and know the learning target/goal and the
criteria for reaching it. Establishing and defining quality work together,
asking students to participate in establishing norm behaviors for classroom
culture, and determining what should be included in criteria for success are
all examples of this strategy. Using student work, classroom tests, or
exemplars of what is expected helps students understand where they are,
where they need to be, and an effective process for getting there” (Garrison
& Ehringhaus, n. d., ¶11).
Objectives should be skill-based and linked to the skills being developed in the
lesson. They should not be a description of the activity.
Objectives are often expressed as ‘I can…’ statements.
They need to be measurable.
Be specific.
Avoid generic verbs difficult to quantify such as “understand”, “be aware of”.
Use active verbs (i. e. define, classify, design).
Check out the video Assessment quickies (MASaintGermain, 2009) in the In Depth
section to learn more about writing effective learning outcomes.
© Universidad Internacional de La Rioja (UNIR)
The acronym KWL stands for “what I know, what I want to learn and what I learnt”. It is
presented as a table that provides a visual representation of the learner’s previous
knowledge, objectives and learnt content in three separate columns:
Know activates prior knowledge and previously learnt content (beginning of the unit).
Want motivates by considering what the learner wants to learn.
Learnt reviews the learnt content.
The know and the want column are completed at the beginning of the learning cycle to
stimulate interest and motivate the student, whereas the learnt column is completed at
the end as part of the revision process to enhance retention.
Figure 3. The learning functions of the KWL and timing of each part within the learning cycle.
© Universidad Internacional de La Rioja (UNIR)
Figure 4. Speak Think Understand - KWL Chart from Critical Thinking. Rebekah Benson. Source:
https://www.teacherspayteachers.com/Product/Speak-Think-Understand-KWL-Chart-241002
© Universidad Internacional de La Rioja (UNIR)
Figure 5 How to Catch a Leprechaun Writing Prompt & Activities. Mrs. P's Special Education
Classroom. Source: https://www.teacherspayteachers.com/FreeDownload/How-to-Catch-a-
Leprechaun-Writing-Prompt-Activities-2445799
Lismayanti (2014) carried out research to determine the efficacy of using the KWL
with reading comprehension. Similar, smaller scale case studies can be carried out in
one’s own classroom to determine the effects tools such as the KWL can have on
students’ learning.
“There is a fact that most of the Secondary School students are still low
in comprehending reading texts. Therefore, the main objective of this
study was to see whether the use of KWL (Know, Want, Learned) strategy
was effective in improving the students’ reading comprehension
achievement in learning English as a Foreign Language. Non-equivalent
groups pretest-posttest design was used in this study. The population was
the eighth-grade students of SMPN 4 Palembang in academic year of
2011/2012 with a total number of 254 students. Out of this population,
40 students were taken as sample. There were two groups, each of which
consisted of 20 students. The data were collected by using multiple choice
reading comprehension test. The data obtained were analyzed by using
t-test formula. The finding showed that KWL strategy was effective in
improving the students’ reading comprehension achievement. The
effectiveness was indicated by the result of the Stepwise Regression
formula that the contribution of KWL strategy on students’ reading
comprehension achievement was 70.5 %.” (Lismayanti, 2014, abstract)
Graphic organisers and checklists are additional tools teachers can use to make
© Universidad Internacional de La Rioja (UNIR)
learning goals visible. They can be used in conjunction with sharing objectives and
rubrics with the student for effective assessment for and as learning.
Graphic organisers provide a visual representation of content and abstract ideas. They
are especially useful when the content involves sequencing or to connect abstract
concepts as a web of ideas. They can be adapted for use with very low levels to very
advanced levels. Below we can see two very different graphic organisers designed to
work with improving writing skills. The organiser in figure 6 is designed to help university
students write a research paper, while the Jack and the Beanstalk organiser is aimed at
children writing a letter (figure 7).
Figure 6. Guided note taking graphic organizer for a research paper. Weavers Social Studies Super
Store. Source: https://www.teacherspayteachers.com/Product/Guided-note-taking-graphic-organizer-
for-a-research-paper-2500146
© Universidad Internacional de La Rioja (UNIR)
Figure 7. Jack and the Beanstalk Graphic Organizer for writing a letter. Misty Mabry. Source:
https://www.teacherspayteachers.com/Product/Jack-and-the-Beanstalk-Graphic-Organizer-for-writing-
a-letter-83811
When using graphic organisers in class, Merkley and Jefferies (2001) recommend
teachers follow these steps:
While the Internet is full of models to download, it can be very enriching to design
graphic organisers in class. This can be a group assignment, as each team is assigned one
aspect to work with and has to design a graphic organiser to teach that material to the
rest of the class. Below we can see an outline of a didactic unit for designing graphic
organisers to help assimilation of content, Higher Order Thinking Skills and peer
assessment.
As a class, work out objectives. While these are guided by the teacher, it is
important the students feel they have some input in decision-making and are fully
aware of the learning objectives.
As a class, discuss the project and design a rubric to fit (see previous units). The
rubric can be written on the board or posted on the working wall so all students
are aware of assessment criteria and can consult it throughout the project.
In groups, students research their content. They design a web of ideas/graphic
© Universidad Internacional de La Rioja (UNIR)
organiser to explain their content effectively using images, labelling and short text
explanations.
Each team presents their project and the rest of the class and the teacher and
peers assess their performance using the rubric.
Merkley and Jefferies (2001) recommend the following four steps to create a graphic
organiser:
Analyze the learning task for words and concepts important for the student to
understand.
Arrange them to illustrate the interrelationships and pattern(s) of organization.
Evaluate the clarity of relationships as well as the simplicity and effectiveness of
the visual.
Substitute empty slots for certain words in order to promote students' active
reading.
Like KWL, graphic organisers can be used in research to determine if/how they aid
learning. Below we can see the abstract of research into the use of graphic organisers
to aid EFL reading comprehension:
English as a Foreign Language (EFL) students. A total of 340 first and third
semester students of non-English majors at a Chinese university
participated in this study. A DSGO completion test and a TOEFL (Test of
English as a Foreign Language) reading comprehension test were
administered before, immediately after, and 7 weeks following the
instructional treatment. The results showed that the DSGO instruction
Checklists
Whereas graphic organisers are used to organise ideas and assimilate content, checklists
are a list of required items or points to be considered. They are used as a reminder
either during learning to check what to include, or after learning to check all points have
been included.
Like graphic organisers, checklists can be used in a wide variety of teaching contexts, and
easily adapted to the learner. Below we can see two very different checklists, one aimed
at young children for writing (figure 8), and the other aimed at business EFL students
who, we can assume from the checklist, are probably adults.
© Universidad Internacional de La Rioja (UNIR)
Figure 8. Christmas, New Year’s 2020, Martin Luther King Jr. - Revise & Edit Checklist. Angelica's
Resources. Source: https://www.teacherspayteachers.com/FreeDownload/Christmas-New-Years-
2020-Martin-Luther-King-Jr-Revise-Edit-Checklist-1038885
For the purposes of this course, the in-class participation grade depends
on the following:
Doing the above will help you to develop your responses as listed below,
which will in turn affect your in-class participation grade positively.
Therefore, a student who demonstrates a high level of in-class
participation does the following:
Comparing the ease, or difficulty, with which we can read the necessary information on
each checklist, it becomes clear that visual information, with less text, is much easier to
use. Even taking into consideration the learner type that the second checklist is aimed at
(probably adult with business orientation), it is still much more difficult to access the
4.6. Rubrics
Rubrics, scoring scales or rating scales are used extensively in language assessment
to ensure consistency in evaluation criteria and scoring. They specify expectations,
subdividing each into what is considered an acceptable or unacceptable,
performance level, etc. Possibly the most well-known is the CEFR can-do
performance scales (we will see them in further units).
Normally, there are two types of scoring scales: analytical and holistic. Each one
complies with a different function and assessment context. Below we can see a
comparison of each.
© Universidad Internacional de La Rioja (UNIR)
Research suggests that analytical scales give more reliable results (Green & Hawkey,
2012, p. 303) This is possibly because they allow the rater (the person scoring) to
assign different scores for different features. Results are then added up for the total.
In table 1 we can see an example of an analytical rubric for fiction writing content.
CRITERIA 4 3 2 1
One of the plot parts
Both plot parts Neither plot
is fully developed
PLOT: What and Both plot parts are are addressed parts are
and the less
why fully developed. but not fully fully
developed part is at
developed. developed.
least addressed.
Both setting
One of the setting
parts of the Neither
Both setting parts parts is fully
SETTING: When story are setting nor
are fully developed and the
and where addressed but parts are
developed. less developed part is
not fully developed.
at least addressed.
developed.
Table 1. Example of an analytical rubric for Fiction Writing Content. Adapted from Teacher Vision,
(2001).
The student’s work is assessed as a whole, rather than broken down into its
component parts.
Holistic scoring scales are often considered more straightforward to use.
In table 2 we can see an example of a holistic rubric for the same content as seen in
table 1.
Table 2. Example of a holistic rubric for Fiction Writing Content. Adapted from Teacher Vision
(2001).
Neither rubric is better or worse. The use of either will depend on the learning
context. However, when peer evaluating, especially with children, it may be
© Universidad Internacional de La Rioja (UNIR)
preferable to use a simple analytical rubric that marks out of 10 (see table 3
for an example).
Using rubrics have numerous benefits for both the students and the teachers.
Assessment should NOT be a secretive process where the teacher believes sharing
evaluation criteria is akin to cheating and giving the students an unfair advantage. In
fact, the opposite is true. Assessment should be a transparent process that the pupil
is able to participate in. Rubrics facilitate this process (Nuere & Díaz-Obregón, 2018).
They aid objectivity, communication, facilitate feedback, and speed up grading. Let
us look at the benefits of using rubrics for both the students and the teachers:
challenging than simply ticking right or wrong in a closed answer test. By providing
a framework with level descriptors, it is much easier to award grades.
• Consistency. Rubrics increase the consistency of grading throughout the marking
of all the students.
• Reduced workload. Rubrics help speed up the marking process.
Once a teacher has used rubrics, the data collected can be used for teachers to analyse
classroom learning. Teachers can ask themselves questions such as (Stevens & Levi,
2013):
The reasons for sharing the rubrics with our learners are very similar to why we share
the learning objectives. Rubrics are based on student learning outcomes, and, as
mentioned earlier, they provide indicators and descriptors that break down the
objectives into more manageable parts where each student is able to see how to
achieve the objective. It empowers the learner and enables them to become an
© Universidad Internacional de La Rioja (UNIR)
active player in the learning process. By explaining and discussing the rubrics, we are
giving the learners the tools necessary to interpret the rubric, so feedback becomes
meaningful.
Empowering.
Becoming responsible for their learning.
Motivating.
Providing direction.
Encouraging self-evaluation. This stimulates metacognitive analysis and learning (see
previous units when we discussed assessment as learning).
Critical thinking.
Creating rubrics
When creating rubrics to use as an assessment tool in class, Stevens and Levi (2005, p.
29-30) recommends the four following key stages:
Stage 1: reflecting. In this stage, we take the time to reflect on what we want from
the students, why we created this assignment, what happened the last time we gave
it, and what our expectations are.
Stage 2: listing. In this stage, we focus on the particular details of the assignment and
what specific learning objectives we hope to see in the completed assignment.
Stage 3: grouping and labelling. In this stage, we organize the results of our reflections
in stages 1 and 2, grouping similar expectations together in what will probably
become the rubric dimensions.
Stage 4: application. In this stage, we apply the dimensions and descriptions from
stage 3 to the final form of the rubric. Use a grid format.
While many, many different types of rubrics exist, each adapted to the task at hand, an
easy formula to construct quick and effective rubrics is one which is based on 10 points.
There are five different components, each with a maximum of 2 points. If a component
is not observed, it is awarded 0 points. If it is partially observed, 1 point, and 2 points if it
is fully observed. It allows the teachers to give immediate feedback to the learners in an
easy to understand format as it does not involve complicated calculations and divisions
to know the mark out of 10.
This rubric is also ideal for group evaluations in project work, for example an oral
presentation. The teacher asks the group for their feedback and completes the rubric
© Universidad Internacional de La Rioja (UNIR)
In the following video, we will have a look in detail about the steps to follow to help
us implement rubrics and other assessment tools effectively in class to encourage
assessment for, as and of learning.
Green, A. & Hawkey, R. (2012). Marking Assessment: rating scales and rubrics. In
O'Sullivan, B. & Stoynoff, S. (Eds.) The Cambridge guide to second language
assessment. Cambridge University Press.
Teacher Vision, (9th January 2001). Fiction-Writing Content Rubric. Teacher Vision.
https://www.teachervision.com/fiction-writing-content-rubric
© Universidad Internacional de La Rioja (UNIR)
Quast, J. & Stanforf, A. (October 2018). Listening to sixth graders to shape positive
outcomes for the use of rubrics. Amble Magazine.
http://www.amle.org/BrowsebyTopic/Assessment/AsDet/TabId/180/ArtMID/780/Artic
leID/969/ReTooling-Rubrics.aspx
MASaintGermain (6th May 2009). Assessment Quickies #2: Writing Student Learning
Outcomes [video file]. https://www.youtube.com/watch?v=KA0DeysBX7M
© Universidad Internacional de La Rioja (UNIR)
Tobey Sanford (24th October 2013). Robert Gagné: The Conditions of Learning [video file].
https://www.youtube.com/watch?v=EOIGhyiCwpU
While this video is entertaining and easy to understand, it also offers a clear insight
into how Gagné’s nine-step instruction can be applied. It was designed by American
Graduate students. It is included here to provide a good example of group work
where each member has worked together and made a clear contribution to the final
product.
Teacher Vision (13th July 2007). Top 10 Most Popular Graphic Organizers. Teacher Vision.
https://www.teachervision.com/top-10-most-popular-graphic-organizers
2. Stevens and Levi (2005, p. 29-30) recommend the following stages when
constructing a rubric:
A. Applying, grouping and labelling, listing, reflecting.
B. Reflecting, applying, listing, grouping and labelling.
C. Reflecting, listing, grouping and labelling, applying.
D. Applying, listing, grouping and labelling, reflecting.