Designing an English

Language Test
Assessment in the EFL Classroom
7th semester
By Nickie Poulakis 1563201700289

Instructor: Trisevgeni Liontou

January 2019
 “Education is not the filling of a pail,

but the lighting of a fire.”

William Butler Yeats

Nickie Poulakis | 1563201700289

 Table of Contents

I. Introduction ............................................................................................................................................... 3

II. Progress Achievement Test ....................................................................................................................... 4

Part 1: Reading .............................................................................................................................................. 4

Part 2: Writing ............................................................................................................................................... 6

Part 3: Listening ............................................................................................................................................ 7

Part 4: Speaking............................................................................................................................................. 8

III. Rationale ................................................................................................................................................ 9

Part 1: Reading .............................................................................................................................................. 9

Part 2: Writing ............................................................................................................................................. 11

Part 3: Listening .......................................................................................................................................... 13

Part 4: Speaking........................................................................................................................................... 15

Conclusion ....................................................................................................................................................... 17

Appendices ...................................................................................................................................................... 18

Appendix 1 .................................................................................................................................................. 18

Appendix 2 .................................................................................................................................................. 19

Appendix 3 .................................................................................................................................................. 20

Appendix 4 .................................................................................................................................................. 21

Appendix 5 .................................................................................................................................................. 22

References ....................................................................................................................................................... 23

Nickie Poulakis | 1563201700289

 I. Introduction

Assessment is an essential part of Foreign Language Teaching, and we as teachers of English

should undoubtedly be familiar with its basic concepts, as well as with the design and evaluation of
an English language test. The current project works toward this goal, its aim being the design of a
four-section test covering the four skills – Reading, Listening, Speaking, and Writing.

The present paper presents a Progress Achievement test, contributing “to formative assessment”
(Hughes, 2003:14), as it establishes “how successful […] students […] have been in achieving
objectives” (13). It was designed for a group of four thirteen to fourteen-year old students of a small
language school in the periphery of Athens. They all share Greek as their mother tongue, but two of
the students are Greek-Polish bilinguals; they have been learning English for the last seven years.
Their level, which is quite homogeneous, is B2 (“Independent Users”) as specified in the Common
European Framework of reference for Languages (2001) and they are sitting the Cambridge First
for schools examination in May.

As Hughes argues, “it is better to base the content of achievement tests on course objectives rather
than on the detailed content of a course” (2001:15). Therefore, since the objective of this course was
to achieve a B2 level and ultimately succeed in the Cambridge FCE exam, the format of the test
designed was based on the format of the said examination.

Finally, it is also important to mention that at the beginning of the year, the students had been
profiled. Their interests included social media and watching films, among others, while all of them
had pet dogs. The following test has been designed taking these interests into account.

Nickie Poulakis | 1563201700289

 II. Progress Achievement Test
Name: Class:
Date: Grade: /100

Part 1: Reading

You are going to read an article about different jobs that were created thanks to the popularity of
social media. Six sentences have been removed from the article. Choose from the sentences A-F the
one which fits each gap (1-5). There is one extra sentence which you do not need to use.
Allocated time: 15 minutes

New Jobs Brought By Social Media

With the rise of the social media industry, it’s logical that a lot of jobs were born from it. ___1___
Social media jobs aren’t just about communicating with people online, and there are plenty of other
positions in this field. Now, let’s take a look at some jobs that have evolved since the creation of
social media.

Content Creation
A business needs to have an online presence, but how are they going to do that if they are not
producing any content? That’s how the content creator position came about. Content creation has a
wide range of job possibilities. For example, a company that sells handmade rings needs to have
pictures showcasing their models. ___2___ Besides working for a company, content creators
usually can write articles for news outlets, post videos on YouTube, add captions to Instagram
photos, and much more.

Social Media Manager

Companies are always looking for someone that can manage their social accounts. ___3___
Working as a manager can be stressful, but working from home is great! Sometimes this job may
imply dealing with customers on social media, and answering typical questions.

Hackers or Security Specialist

4 No, we’re not talking about typical Hollywood hackers. A security specialist works in a similar way
as a hacker does. But instead of releasing essential documents, they find security breaches and fix
them. It’s essential for a website to hire someone qualified for this job; many malicious softwares

Nickie Poulakis | 1563201700289

 can try to steal the customer’s information. ___4___

App Developer
Developing an app requires a large team of people; someone that can manage the social accounts
and give the right exposure to the app, a programmer that knows how to deal with coding, and a
security specialist ensuring that there aren’t any information leaks. This is a humble field, but it’s
something that is worth taking a risk for. No one ever knows if the app will go viral, and suddenly,
the developer could become the next Mark Zuckerberg.

Final Thoughts
___5___ The good thing is, it created a lot of jobs for millennials, and anyone can do them if they
have the right skills. Getting started can be difficult, but if someone is destined to become famous,
they will. Content creation, app developing, and security-related jobs are some of the most well-
known and important online.

A. This includes analyzing the audience, creating content with a team, posting at the right time,
and so much more.
B. Social media has its pros and cons.
C. Although this is a harder job, it is something that can be very successful if it goes viral.
D. Although social networks are present in everyone’s life, people often forget that they can
make a living out of it, or just don’t think about it as an option.
E. This job didn’t exist before because people weren’t used to online shopping. And also,
giving personal information online wasn’t usual for an ordinary person.
F. So, they will have to hire someone that organizes the photos, adds the information to the
website, and someone that posts them on the social networks.

____/25 points

(5 points per answer)

Nickie Poulakis | 1563201700289

 Part 2: Writing

Your teacher has asked you to write a story for the school English-language magazine. The story
must begin with the following sentence:

Kate was ready to unlock the door, when all of a sudden she heard a strange noise.

Your story must include:

 A visitor
 A ring
Write your story. (140-190 words)
Allocated time: 40 minutes

______/25 points

Nickie Poulakis | 1563201700289

 Part 3: Listening

You will hear an interview of Michael Hingson, a blind man whose dog guided him out of the
World Trade Centre in September 2001. For questions 1-5, complete the sentences with a word or
short phrase.
(Approximately 15 minutes)

On September 11th 2001, Michael was working on the (1) _____________ of the World Trade
Centre.
At the time of the explosion, Michael’s dog, Roselle, was (2) _______________ under his desk.
Because of his job as (3) _____________________, Michael was familiar with evacuation
procedures.
Roselle guided Michael and David to the (4) __________________, 1,464 steps away from the
ground floor.
As Michael and the group got closer to the ground, they saw people who were panicking and the
staircase was full of (5) ______________________.

______/25 points

(5 points per answer)

Nickie Poulakis | 1563201700289

 Part 4: Speaking

You and your partner are members of the student council and you are talking about your upcoming
field trip. Discuss how the destinations below might both appeal to and benefit students. Give
reasons and examples. Then, decide which destination you will visit.
Approximately 15 minutes

go to a
big city
abroad

How might these

a village in destinations both
visit an
the benefit students island
countryside
and appeal to
them?

go to the
mountain

_______/25 points

Nickie Poulakis | 1563201700289

 III. Rationale

Part 1: Reading

In this part of the test, learners have to read an online article related to jobs created because of the
rise of social media. This text, the original version of which can be found on
https://socialmediaexplorer.com/media-journalism/new-jobs-brought-by-social-media/,was selected
because it falls within their area of interest, as determined by the student profile created at the
beginning of the year. Text authenticity was also important for this choice, since it helps produce
learners able to communicate effectively and competently in real life contexts.

Furthermore, they have background knowledge on the topic, but not too much for it to be deemed
common knowledge. Therefore, both top-down and bottom-up processes are likely to be employed
as the test takers draw information from the text and combine it with information and expectations
that they already have (Celcea-Murcia, 2001).

The text is at the appropriate level of difficulty so as to be understood by the test-takers and to elicit
level-appropriate items as well; according to the CEFR Companion Volume with New Descriptors
(2017:62), B2 users are expected be able to “obtain information […] from highly specialised
sources within [their] field”, “with a large degree of independence” (60). So concerning the
vocabulary in the text, research has shown that in order for the student to be able to understand a
text with ease, they have be familiar with at least 95% of the words. The English Vocabulary Profile
(see Appendix 1) was thus used to determine the level; it was judged appropriate for B2 learners,
apart from a few words (malicious, imply, humble, destined, prominent). “Prominent” was replaced
by “well-known and important”; background knowledge and context made up for the rest of them.

As far as the task is concerned, test-takers will have to fill in a total of five gaps with the five
sentences that have been extracted from them – a multiple matching exercise. The instructions were
carefully written so as not to confuse them; simpler vocabulary was therefore used. There is also a
distractor – an extra sentence that does not need to be used – in order “to avoid the danger that once
all but one choice has been made, there is only one possible final choice” (Alderson, 2000:219).
The distractor is a sentence that was taken out from the beginning of the fifth paragraph, which does
9 not contain a gap – that way the extra sentence would be written in the same style as the rest of the
text and definitely not be suitable for the other gaps, since each paragraph is about a different job.
As for the rest of the design, the sentences that would be extracted were selected depending on the

Nickie Poulakis | 1563201700289

 presence of reference pronouns, repetition, synonyms and linking words or phrases that would help
the test-taker to restore them because, as Alderson (2000: 218) points out, the difficulty with these
types of tasks is “to ensure that no choice is possible unintentionally”. After all, it is a task that,
according to Munby’s taxonomy of reading skills (1978), requires that readers understand relations
beyond sentence level.

10

Nickie Poulakis | 1563201700289

 Part 2: Writing

For White (1990), writing tasks have four minimum requirements; clarity, validity, reliability, and
interest. Based on Cushing Weigle’s (2002:90) clarification of these requirements, the current
writing task was created to be, first and foremost, clear. The prompt and instructions contain
vocabulary that is slightly below the level of students; meaning is thus accessible to everyone. In
addition, they are neither unnecessarily long, which could create confusion among test-takers and
possibly detract their attention from the main focus, nor too short, as they provide enough
information and guidelines for the students to fulfil the task.

The writing task is also valid, as it gives learners the opportunity to showcase their abilities
whatever their level. According to the CEFR Illustrative Descriptors (Council of Europe, 2017:74-
75), creative writing is a type of written production that can elicit responses ranging from A1 to C2
level. A test-taker might “narrate a story, […] describing feelings and reactions in a simple
connected text” if their level is B1; s/he may “write a clear, detailed description of a real or
imaginary event or experience marking the relationship between ideas in clear connected text, and
following established conventions of the genre concerned” if their level is B2; or even write an
imaginative narrative, “[incorporating] idiom and humour” if they are over-achievers and their level
is C1.

Furthermore, the task designed is reliable, as the writing products will be evaluated according to
the same rating scale – the same one they will be marked by in their upcoming exams, the
Cambridge writing rating scale for B2 level (see Appendix 2). That way, it will be ensured that test-
takers will be graded with fairness, by a clear and concise scoring rubric that will be the same for
everyone. Since the scale criteria add up to 20, each grade will be weighted to 25 points.

Finally, the task will most likely be interesting for both the target test-takers and the
teacher/evaluator. One of the students’ interests communicated to the teacher through the process of
student profiling was watching films, with preference to different genres such as romance, action,
adventure, and even horror. So all students are likely to be engaged in this type of writing task and
find something to write. Apart from the motivation that a task designed taking their interests into
account entails, it also activates schemata, “a body of knowledge to which all writers have equal
access”; Kroll & Reid (1994:235) stress that “student-writers perform most successfully when the
11
writing topic taps into their background knowledge”.

Nickie Poulakis | 1563201700289

 In essence, the writing task was thus designed so that it reflects a real-life situation; a student (who)
is to narrate a story (purpose/genre/what) to the school magazine (to whom) to be read by their
schoolmates (by whom). It was well thought out so that it would be neither too general, risking
misinterpretation of the teacher’s intention – hence the starting sentence and two elements the writer
should include – nor too rigid, restricting the students so much that they cannot come up with any
output. The starting sentence leaves the writer the freedom of narrating any kind of story – romance,
adventure, even horror for Stephen King lovers – while the elements that should be included can
have multiple interpretations (a ring could be a phone call, engagement ring, loot or heirloom).

12

Nickie Poulakis | 1563201700289

 Part 3: Listening

In the listening task, the test-takers will listen to an authentic in origin oral text (Geddes & White,
1978); an interview with Michael Hingson, a blind man who was in his office at the World Trade
Centre on the day of the 9/11 terrorist attack and whose guide dog successfully led him out of the
building before it collapsed. As intrinsically motivating tasks generally ensure active participation,
this recording was selected, from the website https://www.bbc.co.uk/sounds/play/w3csv27j, because it is
an unusual story about a dog, which falls within the students’ interests, according to their profiles – they are
all fascinated with learning about their favourite pet’s achievements.

Furthermore, they have background knowledge on the topic of the 9/11 terrorist attacks, but not too
much for it to be deemed common knowledge or too sensitive an issue that might promote negative
responses. Buck (2001:124) asserts that “there is ample theoretical evidence that background
knowledge is important in listening comprehension, and there is research evidence to suggest that it
does affect test performance”, citing Chiang and Dunkel (1992), Long (1990), and Schmidt-
Rinehart (1994). In addition, for Buck (2001:150) “texts which have events described in a linear or
temporal order”, like the recording in question, “tend to be easier than texts which have a non-linear
structure”. The same applies for “texts in which the important protagonists […] are more easily
distinguished”. We are concerned with the text being “easier”, in order to compensate for the
increased difficulty of an authentic text and the slightly more advanced vocabulary that it contains.

Last but certainly not least, according to the CEFR Illustrative Descriptors (2017:55), B2 users are
expected to be able to “understand standard spoken language, live or broadcast on both familiar and
unfamiliar topics […].Only extreme background noise, inadequate discourse structure and/or
idiomatic usage influence the ability to understand. [They] can follow extended speech […]
provided the topic is reasonably familiar, and the direction of the talk is sign-posted by explicit
markers.” The recording is a broadcast interview, on a fairly familiar topic, without significant
background noise, inadequate discourse structure, or idiomatic usage, while explicit markers help
the listener follow the interview.

As far as the task is concerned, a selective listening task was designed, which asks of the listeners to
employ their skill of extracting specific information from the recording and fill in the gapped
sentences. In order to do that, they will need to employ bottom-up strategies, recognising key words
13
and key transitions in discourse, as well as top-down ones, associating the information they hear
with prior knowledge in order to infer the topic of the discourse and the outcome of the event. They
will first be given thirty seconds to read the items and then the recording will be played twice, with
Nickie Poulakis | 1563201700289
 thirty more seconds in between so they can write any answers they have already come up with. That
makes it a total of twelve minutes approximately; ten minutes’ time-on-task – since I cropped the
original recording on account of its length, using Audacity (see Appendix 3) – and about two
minutes’ wait time. The instructions were also carefully written so as not to contain any unknown
words or other kind of challenge; besides, comprehension of instructions is not what is being tested.

As for the items, it was important to create gapped sentences requiring “information that is relevant
to the main theme” (Buck, 2001:151), which was the story of the guide dog leading his blind owner
out of the World Trade Centre, and not the terrorist attack itself; so that was something that had to
be dealt with carefully. Another challenge faced lied in the contradiction between Sheerin’s view
(1987:128) that “we should not rely exclusively […] on questions that require exact recall of verbal
detail” and Buck’s claim (Buck, 2001:151) that “[t]asks that require recalling exact content tend to
be easier than tasks which require extracting the gist, or making a summary”. A compromise was
made, with the design of a gapped task that does not ask for the exact word or phrase in the
recording.

14

Nickie Poulakis | 1563201700289

 Part 4: Speaking

Hughes (2003:23) highlights the salience of “communicative language testing […] [and] measuring
the ability to take part in acts of communication. For Nunan (1993:59), a communicative task is “a
piece of classroom work which involves learners in comprehending, manipulating, producing or
interacting in the target language while their attention is principally focused on meaning rather than
form”. Since speaking is the most frequently used skill of all, it is essential that its assessment be
done in a real-life context. Therefore, they are put in a situation where they are members of the
student council having to discuss about their options for the end-of-year field trip before finally
reaching a consensus as to where they will go.

This task was designed to simulate a situation that the students are familiar with, and thus have the
necessary schemata to come up with appropriate content easily and effectively. Moreover,
according to Brown& Yule’s (1983) distinction of different types of conversational talk, this is both
a factually-oriented task, as it calls for comparison of the different options, and an evaluative one,
since the text-takers are to make explanations, justifications, and decisions. As a result, they will
use what the CEFR identifies as macro-functions, such as comparing and justifying, and micro-
functions – namely agreeing or disagreeing, suggesting, signalling non-understanding, etc. – to
showcase their functional competence.

A challenge that was encountered was to decide whether it would be a one-candidate situation or a
paired task. Eventually the latter was opted for, due to the fact that in the former case, “the
interlocutor initiates all phases of the interaction and asks the questions, whereas the role of the
examinee is to comply and answer.” However, “in other types of interactions, such as discussions or
conversations, the rights and responsibilities of the participants to take the initiative are more
balanced”. (Luoma, 2004:35) Besides, a paired task is what they will encounter on the day of their
FCE exam, so it seemed like the most suitable choice.

Nevertheless, this type of task has a significant drawback. Luoma (2004:37) points out that “the
examinees’ talk is almost inevitably influenced by the other participant’s personality,
communication style and possibly also language level. The concern is that all test takers may not
get an equal opportunity to show their speaking skills at their best.” However in this case, the target
test-takers know each other, and they have had plenty of practice together in speaking tasks, so this
15
negative point will not have a big impact on their performance.

Nickie Poulakis | 1563201700289

 This task was designed to be open-ended so that the students would have “more potential freedom
[…] for responding to it”, although some guidelines are offered through the instructions. This way –
in parallel to written production – the task will be more “valid”, as test-takers can respond and
produce output according to their abilities and their level of language competence.

As for evaluation, the Cambridge Speaking rating scale for the B2 level will be employed (see
Appendix 4).

16

Nickie Poulakis | 1563201700289

 Conclusion

All in all, it is essential that the teacher/assessor adapt every test to the target test-takers. Undoubtedly, the
presence of schemata, or background knowledge, enhances test-takers’ performance. From the part of the
test-maker, s/he has to apply the same criteria and use the same scoring scales for everyone. This will lead to
a testing situation that is to the students’ best interest.

17

Nickie Poulakis | 1563201700289

 Appendices

Appendix 1

18

Nickie Poulakis | 1563201700289

 Appendix 2

19

Nickie Poulakis | 1563201700289

 Appendix 3

20

Nickie Poulakis | 1563201700289

 Appendix 4

21

Nickie Poulakis | 1563201700289

 Appendix 5

Key

Reading

1-D

2-F

3-A

4-E

5-B

Distractor: C

Listening

1) 78th floor

2) asleep/sleeping

3) an office manager

4) emergency stairwell/staircase

5) fumes/smoke and noise

22

Nickie Poulakis | 1563201700289

 References
Alderson, J. C. (2000). Assessing Reading. Cambridge: Cambridge University Press.

Brown, G., & Yule, G. (1983). Discourse Analysis. Cambridge: Cambridge University Press.

Buck, G. (2001). Assessing Listening. Cambridge: Cambridge University Press.

Celcea-Murcia, M. (2001). Teaching English as a Second Or Foreign Language. Boston: Heinle & Heinle
Publisher.

Chiang, C. S., & Dunkel, P. (1992, Summer). The effect of speech modification, prior knowledge, and
listening proficiency on EFL lecture learning. TESOL Quarterly, pp. 345-374.

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning,
Teaching, Assessment. Strasbourg: Council of Europe.

Council of Europe. (2017). CEFR Companion Volume with New Descriptors. Strasbourg: Council of Europe.

Cushing Weigle, S. (2002). Assessing Writing. Cambridge: Cambridge University Press.

Geddes, M., & White, R. (1978). The Use of Semi-scripted Simulated Authentic Speech in Listening
Comprehension. Audiovisual Language Journal, pp. 137-145.

Hughes, A. (2003). Testing for Language Teachers. Cambridge: Cambridge University Press.

Kroll, B., & Reid, J. (1994). Guidelines for Designing Writing Prompts: Clarifications, Caveats, and
Cautions. Journal of Second Language Writing, pp. 231-255.

Long, D. R. (1990). What you don’t know can’t help you: An exploratory study of background knowledge
and second language listening comprehension. Studies in Second Language Acquisition, pp. 65-80.

Luoma, S. (2004). Assessing Speaking Skills. Cambridge: Cambridge University Press.

Munby, J. (1978). Communicative Syllabus Design. Cambridge: Cambridge Univeristy Press.

Nunan, D. (1993). Introducing Discourse Analysis. London: Penguin English.

Schmidt-Rinehart, B. C. (1994, Summer). The Effects of Topic Familiarity on Second Language Listening
Comprehension. The Modern Language Journal, pp. 179-189.

Sheerin, S. (1987, April). Listening comprehension: teaching or testing? ELT Journal, pp. 126-131.

23 White, E. M. (1990, May). Language and Reality in Writing Assessment. College Composition and
Communication, pp. 187-200.

Nickie Poulakis | 1563201700289