TESTING

INSTRUCTIONS / DESIGN
POSITIVE POINTS

 INSTRUCTIONSare clear / an example is given/ the questions are written above the pictures
 INSTRUCTIONSthe instructions are clear for the teacher
 RUBRICClear and simple instructions and prompts. This helps reliability
 FORMATof the task is simple / the layout is clear / there are key words in the task to guide the learners
through the text
 FORMATThere is minimal text to read and very little writing required in the answers (LISTENING)
 FORMATThe learners know that they will have the chance to listen to the recording twice (LISTENING)
 VISUAL/IDEAS/QUESTIONS PROVIDEDthis provides stimulus / content. Application: It tests SS ability to
communicate rather than her ability to invent the contents / her imagination.
 VISUALS Picture prompts mean learners don’t need to understand written prompts:
 QUESTIONSOpen ended so learner can write whatthey like
 PROMPTS SUPPLIED Prompts allow writer to focus on writing rather than on worrying about what to write:
Application: the prompts will allow learners to use some of the language studied (conditionals and articles)and
so the test has content validity
 DESIGN The design of the task is simple / the layout is clear / there are key words in the task to guide the
learners through the text
 Part 1 eases the learners into the test by asking about relatively simple things, and the tasks increase gradually
in difficulty. Application: If the learners are nervous about their speaking ability, Part 1 should give them time
to relax. The fact that the tasks increase gradually in difficulty should give them time to build confidence
 Having Task 2 on card makes the instructions very clear. Application: This ensures that the learners won’t
misunderstand the topic and will know exactly what they have to do.
 A relatively realistic, real-world task. Application: The writing task may reflect the real-world duties or experience of the
learner and therefore has validity

POSITIVE APPLICATIONS

 LEARNER REACTIONThe learners will feel reassured/confident because of the clear format / won’t panic
because theyknow they will hear it again (LISTENING)
 FORMAT/INSTRUCTIONS The learners will know what to do in the task (even though they maynot be familiar
with this test format) / the students don’t need to remember the questions
 FORMAT SS will not be tested on her ability to think of ideas but rather on her language/writingskills / she’ll be
able to show her true abilities / she’ll know what to do
 IDEAS PROVIDED The learners do not have to think of ideas / ideas and language at the sametime / they won’t
do badly because they lack ideas
 FORMAT SS will be able to perform to the best of her ability without being hindered by thetest/task format /
unknown lexis / If SS goes off track or gets stuck, she has other opportunities
  PICURES SS can relate to thepictures in ways which are relevant to theirown life experiences / ideas
 PICTURES If SS has low reading level, she will be able to respond to the pictures
 PROMPTS SS will not have to hunt for ideas on what to write about
 SPEAKING VS LEXIS It is a test of SS speaking ability rather than her knowledge of lexis so it willbe useful as a
speaking needs analysis


NEGATIVE POINTS

 VISUALSThe pictures may not be clear

 VISUALS The pictures may elicit limited language e.g. present continuous
 NO EXAMPLES/GUIDANCE No example of / guidance on what kind of sentence to write e.g. describe the
picture?
 DESIGN The task can be completed with a very short – even bullet-pointy – piece of writing. This may not
reveal much in depth about the candidate‟s writing skills.
 The examiner is taking all the initiative of asking questions, so it’s not a very typical interaction. Application: If
the learners have difficulty initiating conversation, the format doesn’t test that.
 The examiner could be seen as a figure of authority to whom the learner will defer. Application: The learners
may be less inclined to demonstrate features of spoken discourse such as turn-taking, interrupting and
initiating a topic if they view the teacher as a figure of authority. They might take a more passive role in Parts 1
and 3.
 The learners are required to talk from notes in part 2. Application: Talking from notes in Part 2 might distract the
learners from their audience, and could also mean that their pronunciation worsens.
 Part 3 is a continuation of Part 2. Application: Due to the element of continuation between Parts 2 and 3, the start is not
as ‘fresh’ as it could be here, meaning that if the learners were unable to come up with much in Part 2 they might also
struggle to come up with something to say in Part 3, or might be stressed by the impression that they are talking about
the same topics, which may negatively impact on their performance.

NEGATIVE APPLICATIONS

 TEST FORMAT The learners may be hindered by the format of the test / will not be able to participate
 VISUALS SS may be confusedabout content to write
 VISUALS SSmay not feel she can demonstrate her range of language
 NO EXAMPLE/GUIDANCE SS may be confused about what to write
 It may be difficult for the teacher to design the listening component of the course
2

MARKING
POSITIVE POINTS

 OBJECTIVE MARKING The marking is practical / objective

POSITIVE APPLICATIONS

 The results of the test will be (more) reliable / efficient to mark

NEGATIVE POINTS
  SUBJECTIVE MARKING The results will depend on person assessing
 SUBJECTIVE Range of different answers may make answers difficult to mark in terms of focus of marking/
Subjective element in marking may lead to unreliability

NEGATIVE APPLICATION

 SUBJECTIVE MARKING The learners may be placed in the wrong class

 SUBJECTIVE SS may not get the mark shedeserves and possibly be misplaced

REALIABILITY
POSITIVE POINTS

 ASSESSMENT MODE students talking together. Application: the teacher has the opportunity to focus on
assessment/increases scorer reliability.
 ASSESSMENT MODEMonologue so teacher does not need to act as interlocutor and assess his speaking at the
same time / Application: the teacher has the opportunity to focus on assessment/increases scorer reliability.
 CLEAR FRAMEWORK. the question provides a very clear framework for J to build her answer upon.
Application: the SS is not going to struggle to come up with ideas for the content of most of the email, which
will allow her writing to be assessed rather than her imagination
 TASKS→DATAThe number of parts / range of tasks to this diagnostic test, Application: SS will be able to judge
which situations she finds easier / more difficult torespond to and so be more aware of her own needs.
 VARIETY OF TASKS. answering direct factual questions with short answers in Part 1; delivering a planned
monologue in Part 2; taking part in a discussion about more abstract topics in Part 3. (You needed to describe
at least two of these to get the marks.) Application: The variety of task types gives the learner a better chance
of performing a task that he or she is better at and provides the teacher with a wider range of data upon which
to base the speaking component of the course.
 DATA Quite a good range of language tested (verb forms, articles, preposition of time etc.) Apllication: The
teacher will get useful information about the learners’ progress/abilities / will be able to assess the learners’
language accuracy / will be effective in highlighting SS progress in learning the grammar studied on the course.
 OPEN ENDED QUESTIONS: The questions are generally open-ended. Application: This gives learners the
chance to produce extended utterances and demonstrate their speaking skills.
 DATA. Some imaginative writing is asked for, i.e. coming up with issues to be addressed in the meeting
Application: This could allow the learner to show off her writing skills / for the SS/her employer The test should
provide evidence of SS overall writing ability interms of organisation / ability to select relevant content / use of
fixed phrases / cohesion etc. / teacher should be able to write a useful report on the basis of the writing.
 FAMILIARITY The topics do not require any specialist knowledge and will be familiar to most learners.
Application: J will be able to concentrate on showing what language she can produce and not waste time
trying to understand the contexts.

NEGATIVE POINTS

 TYPE OF INTERACTION Each student’s performance may be affected by that of their partner /higher level /
more confident speaker may dominate the discussion / some learners maynot say much. Application: learners
may not show their full abilities
 ASSESSMENT MODEThe teacher has to act as an interlocutor / assess three students at the same time
Application: It will be difficult for the teacher to assess all the individuals / the test lacks practicality / scorer
reliability
  LIMITED RANGE OF LANGUAGE The learner is tested on a limited range of language (e.g. not tested on future
verb forms, conditional forms, modals, functions of persuading / negotiating). Application: The teacher may
not get an accurate idea of their abilities/progress /learners won’t get a sense of progress

 The open-endedness of the questions in the tasks means that learners may avoid using language they are
not comfortable with. Application: If learners avoid using certain language then the teacher will not gain a full
picture of what they are capable of producing when performing such tasks
 The imaginative writing required is relatively unstructured and unsupported, i.e. providing an unnamed,
unknown office manager with points about an imaginary company to be discussed at a meeting. Application: It
is possible that J will lose points for not being able to think of good topics here
 The task can be completed with a very short – even bullet-pointy – piece of writing Application: This may not
reveal much in depth about the candidate’s writing skills

TIME
POSITIVE POINTS

 The learners are given time to write down their answers

NEGATIVE POINTS

 The test is too short to test the learners’ listening ability

 May be time consuming / impractical tomark

NEGATIVE APPLICATIONS

 SS may have to wait forresults

VALIDITY
POSITIVE POINTS

 INDIRECT / INTEGRATIVE TEST of (speaking skills) / it tests the learners’ speaking skills and language
knowledge / does not test any other skills. Application: tests language in use rather than declarative
knowledge in a grammar exercise / it’s an indirect/integrative test of language / valid as formative/summative
assessment
 DIRECT TEST, testing skills rather than knowledge. Application: This will enable the teacher to gain a good idea
of a skill the learners will need in order to function socially in the UK.
 DIRECT TEST of writing skills, i.e. it tests performance (what J can do with her knowledge). Because
communicative ability is a combination of various sub-skills and types of knowledge, it can be argued that it is
more valid to test it by means of an integrative test such as this. Application: J will be judged on how she uses
the language communicatively, which is appropriate as she uses English in this way in her job.
 CONTENT. Focuses on interactional social English. Application: Interactional social English is necessary for
anyone staying in the UK.
 SPEAKING SUBSKILLSit tests a variety of speaking sub–skills, e.g. turn-taking, initiating, responding, time-
creating devices, paraphrasing, asking for clarification etc (APPLICATION?)
  WRITING SUBSKILLS The writing will test SS ability to use writing sub-skills such as paragraphing, logical
progression of ideas, layout, salutations etc. Application: J will feel she is being tested on a skill that she needs
to develop
 MECHANICAL SKILLS Tests punctuation, letter formation, writing on the lines Application: ‘mechanical’ skills
will be tested relevant to students in a second language learning situation
 FUNCTIONAL LANGUAGE The functions required in both tasks (explaining, acknowledging, offering, requesting,
giving directions) are appropriate. Application: functional language tested is useful for SS in terms of her work
needs
 CONTEXT/CONTENT The test is clearly work and business orientated, increasing face validity. Application: The
test will show SS/employer how she will perform at work / what she needs to continue working on after the
course has finished. (Predictive validity)
 STYLE Semi-formal – neutral style is appropriate and so has validity as a test. Application: The learner will see
the test as valid as he uses a semi-formal neutral style with his customers at work
 TASK TYPE The test has (face) validity / (content) validity as there is an interactive conversation/ discussion and
monologue / it tests speaking sub-skills for these task types. (APPLICATION?)
 LANGUAGE The test requires language to talk about change, problem solving, comparing and giving opinion.
Application: The test indicates to the learner that the course will be relevant to his professional needs / the
test contains content which he might say when socializing in a work context
 LANGUAGE The task allows for a range of language / functions / grammar Application: Replicates authentic
language use e.g. describing, giving information so has facevalidity / allows learner to demonstrate language
OR grammatical OR lexical OR pronunciationOR discourse skills (a minimum of 1)
 LANGUAGE The items that the learners are listening for are well spread out in the transcript Application:
learners will feel motivated to do the test / will have faith in the test/course / see the relevance of the test
(because it’s testing their listening skills)
 LEXIS Lexis and phrases used sometimes relevant to social English. Application: SS will feel this test accurately
tests her progress in the element of language that her course is based around.
 APPROPIATENESS OF TEST TYPE Simulated conversation patterns relevant to social English. Application:
questions / statements for personal / social interactions = may appear relevant to SS (has face validity).
 CONTENT course is organised around grammar and so is this test. Application: SS will feel test is relevant / SS is
more likely to perform well if she thinks the test is relevant
 CONTENTIt tests the candidate‟s knowledge of acceptable conventions for businessemails. Application:
SS/employer will see the relevance of the test in terms of SS needs
 AUTHENTICITY A relatively realistic, real-world task. Application: The writing task may reflect the real-world
duties or experience of the candidate and therefore has validity
 AUTHENTICITY It replicates authentic language use and relates directly to the genre J needs for work (email)
Application: The task should have face validity for J as she can see its relevance to her writing needs

NEGATIVE POINTS

 SUBSKILLSIt only tests one listening sub-skill / listening for specific information/detail. Application: the teacher
will not get the information he needs to plan a course effectively
 SUBSKILLS It only tests sentence level writing / there is no test of more complex features of writing
(paragraphing, complex sentences, linking etc.). Application: the teacher cannot assess how effective the
course has been on the learners
 SPEAKING SITUATIONS All the interaction is face-to-face so telephone skills / conference calls are not tested.
Application: The test is not entirely relevant to learner’s needs regarding types of speaking he needs to do in
his work (i.e. face-to-face interaction / speaking on the phone)
  TESTS SPEAKING / LISTENING SKILLS / OTHER LANGUAGE SYSTEMS The test is wider in focus than the
course /it doesn’t only test grammar / it’s an indirect / integrative test of grammar / it’s a direct test of
speaking skills. Application: SS may feel the course did not prepare them well / lack faith in the teacher and
course
 LANGUAGE AVOIDANCE Free choice of language means that students can avoid unknown language / the
learner may choose one of the topics because he feels confident using the language needed to talk about it.
Application: the teacher cannot get a govfdod idea if the learner has learnt the content on the course.
 TASK APPERANCE It does not look like a serious/grammar test so it lacks face validity. Application: The
learners may be demotivated because this is not how they expect grammar to be tested / they may not take it
seriously.
 ACCURACY The type of interaction/a discussion does not lend itself to testing grammatical accuracy /learners
don’t need to be accurate in this type of discussion / it does not test written accuracy. Application: The learner
wmay feel the test lacks revelance / lose faith in the teacher
 APPROPRIATNESS OF TASK A memo to a colleague lacks content validity because it is unlikely that SS would do
this in English. Application: candidate may not be motivated to write / candidate may be put off their course of
study / institution
 VS GROUP All the tasks are one-to-one whereas, at work, SS participates in group discussions which reduces
the validity of the test / it lacks good predictive validity. Application: The employer/learner won’t get an
accurate idea of how they will perform at work.
 AUTHENTICITY Language is unnatural / unrealistic – often display language. Application: the test lacks the
predictive validity to give the teach an idea of how they will perform in the real world when encountering
natural / practical language
 FUNCTIONAL LANGUAGE No test of the functional language that SS says she needs Application: SS result in
the test will not reflect progress she has made in the area of functional language
 RELEVANCE Despite having some validity for J it doesn't investigate the 'dealing with customer complaints'
aspect of her job. Application: SS may not feel the course will address her needs
 STYLE The style is not totally appropriate for J: informal since to a friend, whereas at work emails to clients and
agents are likely to be more formal. Application: This makes it a less valid test of the language J needs.
 DETAILED INSTRUCTIONS support the candidate in providing clues to content and organisation. Application:
This will not be replicated in the real world so could be argued to make the test less valid for J
 WORD LIMIT The word limit is very low, which means that J will not be able to produce much language.
Application: J may feel that she is unable to express herself sufficiently and therefore question the validity of
the test.
RELEVANCE None of the topics relate directly to working in a hotel or dealing with guests/customers.
Application: J may feel the test lacks validity and not be motivated to perform to her best

LEVEL
POSITIVE POINTS

 LEVEL The test is an appropriate level / there are simple short sentences in the recording / the complexity of
the language is appropriate
 LEVELThe task is appropriate for a wide range of levels/most levels of learners because it is open-ended
 LEVEL Appropriate for the learner’s level / achievable / sufficiently challenging
 LEVEL May produce varied sample of language(vocab, grammar, spelling etc.) so maybe good for placement
 LOWER LEVELSSuitable for low levels because can be completed with limited language

POSITIVE APPLICATION
  CORRECT CLASSThe learners will be placed in the right level
 APPROPRIATE LEVELShe will feel the course is likely, as a result, to be at her level.
 LOWER LEVEL/WEAKER LEARNERSwill still be able to do the test. Can usecompensation strategies.

NEGATIVE POINTS

 LOW LEVEL/WEAKER LEARNERS may not understand the lexis in the instructions/questions
 RANGE May not test range of language sufficient for placement at all levels / may induce error avoidance
(poor predictive validity)
 LEVEL Level is (too) challenging for SS
 It is a fairly, simple, under-challenging task for an intermediate learner; much of the writing could be achieved by
repeating the email or question (e.g. “Yes, the staff already know about the visit”) – there is little demand on the writer
to convey complex or difficult ideas, structure text, transfer and process text from one source to another, etc.
Application: The teacher may not get a clear picture of the range of J’s writing abilities

NEGATIVE APPLICATIONS

 SS may be misplacedbecause she hasn’t been allowed todemonstrate enough

 may produce an unreliableresult / may be demotivating for SS

FAMILIARITY
POSITIVE POINTS

 TOPIC(Shopping) is a general/familiar/relevant topic / lexical area of shopping/accessible topic/ prompts to

provide ideas
 TOPICThe test is contextualised in personal experience and opinions / not reliant on knowing obscure lexis /
the test begins with an easy topic.
 IDEAS PROVIDED / CHOICE OF TOPICThere is an appropriate range of topics to choose from / the topics are
general/familiar/personal
 TASK TYPE - (a discussion) is a familiar/group task
 TASK TYPEA monologue is relevant to a business context as it is similar to giving a presentation / doing ‘small
talk’ with clients.
 APPROPRIATENESS OF TEXT TYPE / TASKA letter to an unknown person uses similarlanguage/organisation as a
business email / is similar to a business email
 TASK TYPE/FORMATis familiar/recognisable / it’s an engaging/unpressurised way to test grammar
 On the whole the topics are general. Application: The general nature of the topics means that the learners
should be able to find something to say about them.

POSITIVE APPLICATIONS

 LEARNER PERFORMANCEThe learners will be able to perform to the best of their abilities / shouldn’t be
blocked by unknown vocabulary or complex grammar (LISTENING)
 LEARNER PERFORMANCE Students have a chance to demonstrate abilities / be able to perform to the best
oftheir abilities
 CONTENT/TOPIC The learners will have something to say / be able to relate to the content / will be interested
in the topic / will want to discuss the topic\
 FAMILIARITY OF TOPIC this will make it easier for SS to complete the task
 TASK TYPE/FORMATLearners should know how to approach the task
 APPROPRIATENESS OF TASK TYPE SS/employer will feel able to trust the outcome of the test
NEGATIVE POINTS

 TASK TYPE Learners may not be familiar with the task type (LISTENING)
 TOPIC / LEXIS RELATED TO THE TOPIC OF (TRAVEL). It is a general business topic / there is no opportunity to
assess the learner’s use of specific travel topic-related lexis

NEGATIVE APPLICATIONS

 LEARNER PERFORMANCEThe learners may not show their real abilities

 TOPICS SS may not be interested in the topics and so find it difficult to speak, especially in the monologue

BACKWASH
POSITIVE POINTS

 POSITIVE BACKWASH in both accuracy of grammar and type of language

 MOTIVATION TO STUDY SS may be encouraged to study hard because she is aware there is a test every two
weeks. (Positive backwash)
 PREPARATION To prepare for this part of the test J would need to practise writing emails. Application:
Preparing for the test in this way would be relevant for her work (positive backwash/ washback).

POSITIVE APPLICATIONS

 when preparing, SS may beexposed to the language she needs.

NEGATIVE APPLICATIONS

 IMPACT ON COURSE DESIGN It may be difficult for the teacher to design an appropriate course to meetthe
learner’s needs
 IMPACT ON COURSE DESIGNSS may not be exposed tolanguage she needs during the course –
negativebackwash

FRESHSTART
POSITIVE POINTS

 FRESH STARTSthere are opportunities for fresh starts / The fresh starts make the test more reliable.
Application Learners will not get stuck if they cannot respond to one of the topics
 There are three main fresh starts across the tasks, and three more within Part 3. Application: If the learners
feel they aren’t doing well in a section / on one question, they know that they have the chance to perform
better in the next one

NEGATIVE POINTS

 LIMITATION OF TASKThere are no fresh starts / the learner is only given one opportunity to speak /short or
limited task

2015-2008 done