Testing Speaking

Arman Argynbayev
MA in TEFL
Suleyman Demirel University
arman.argynbayev@sdu.edu.kz
 Cambridge CCSE Test of oral interaction
OPERATIONS
EXPRESSING: likes, dislikes, preferences, agreement/disagreement,
requirements, opinions, comment, attitude, confirmation,
complaints, reasons, justifications, comparisons.
DIRECTING: instructing, persuading, advising, prioritizing
DESCRIBING: actions, events, objects, people, processes
ELICITING: information, directions, clarification, help
NARRATION: sequence of events
REPORTING: description, comment, decisions and choices
 Informational skills
Candidates should be able to:
• Provide personal and non-personal information
• Describe sequence of events (narrative)
• Make comparisons
• Give instructions
• Present an argument
• Express need, opinion
• Apologize, make suggestions
• Complain, speculate, analyze
• Paraphrase, summarize
• etc.
 Format 1: Interview

Limitations:
• The candidate speaks as to a superior and is unwilling to take
the initiative.
• Only one style of speech is elicited, many functions (such as
asking for information) are not represented.
 Interview sample questions
• Can you explain me how/why …?
• Can you tell me what you think of …?
• I’m sorry, but I don’t quite follow you.
• Is there anything you’d like to ask me?
 Elicitation Techniques

Questions and requests for information

Yes/no questions should be avoided.
‘Can you explain me how/why…’
‘Can you tell me what you think of…’

Pictures
Single pictures are particularly useful for eliciting descriptions.
 Format 2: Interaction with fellow candidates

Advantages:
•Better performance due to equality of the interlocutor
•Candidates feel more confident

Limitations:
•Performance of one candidate might depend on the other
 Elicitation Techniques
Interpreting
Speaker 1: Monolingual speaker of L1 (Kazakh)
Speaker 2: monolingual speaker of the FL (English)
Speaker 3: Interpreter

Discussion
Discussions of a topic or in order to come to a decision.

Role play
Candidates assume a role in a particular situation.
 Format 3: Response to audio or video-recordings

Advantages:
• Large numbers of candidates are tested simultaneously in a
lab.
• All candidates are given the same task.

Limitations:
• Impossible to follow up candidates’ responses. (Aptis)
 Tips on planning and conducting speaking tests
1. Make tests as long as possible. (Not less than 15 minutes for
proficiency test. Not less than 5 minutes for placement test.)
2. Include as wide a sample of specified content (functions) as
possible in the time available.
3. Plan the test carefully. Start with easy general questions and
continue with more complicated ones.
4. Give the candidate as many ‘fresh starts’ as possible. More than
one format should be used. It is desirable for candidates to
interact with more that one tester. Within a format there should
be as many separate ‘items’ as possible.

5. Select interviewers carefully and train them. Interviewers need

to be sympathetic and flexible characters, with a good command
of the language themselves.
6. Use a second tester for interviews.
7. Topics must cause candidates NO difficulty in their L1.
8. Carry out the interview in a quiet room with good acoustics.
9. Do not talk too much.
10. The test should be carried out at the level of the candidate.
11. Put candidates at their ease. Begin with requests for personal
details (not too personal), remarks about the weather, etc.
Testers shouldn’t be seen to make notes on the candidates’
performance during the interview. Transitions between topics
and between techniques should be made as natural as
possible.
 Techniques not recommended

• Prepared monologue (assigned as homework)

• Reading aloud (it’s reading and pronunciation test rather than

speaking assessment)
 Criterial levels of performance

Test at intermediate level

Accuracy Grammatical/Lexical accuracy is generally high, though
some errors which do not destroy communication are acceptable
Appropriacy Use of language is generally appropriate to
function. The overall intention of the speaker is always clear.
Range A fair range of language is available to the candidate.
He is able to express himself without overtly having to search
for words.

Flexibility The testee is able to take the initiative in a

conversation and to adapt to new topics or changes of
direction – though neither of these may be consistently
manifested.

Size Most contributions may be short, but some evidence of

ability to produce more complex utterances and to develop
these into discourse should be manifested.
 Proficiency descriptors
Accent
1. Pronunciation frequently unintelligible.
2. Frequent gross errors and heavy accent.
3. “Foreign accent” requires concentrated listening and
mispronunciations lead to occasional misunderstanding.
4. Marked “foreign accent” and occasional mispronunciations
which do not interfere with understanding.
5. No conspicuous mispronunciations, but would not be taken
for a native speaker.
6. Native pronunciation, with no trace of “foreign accent”.
 Proficiency descriptors
Grammar
1. Almost entirely inaccurate except in stock phrases.
2. Constant errors frequently preventing communication.
3. Frequent errors causing occasional irritation and
misunderstanding.
4. Occasional errors but no weakness that causes
misunderstanding.
5. Few errors with no patterns of failure.
6. No more than two errors during the interview.
 Proficiency descriptors
Vocabulary
1. Inadequate for even simplest conversation.
2. Limited to basic personal and survival areas.
3. Choice of words sometimes inaccurate, limitations of vocabulary
prevent discussion of some professional and social topics.
4. Professional vocabulary adequate to discuss special interests.
5. Professional vocabulary broad and precise; general vocabulary
adequate to cope with complex practical problems.
6. Vocabulary as accurate and extensive as that of an educated
native speaker.
 Proficiency descriptors
Fluency
1. Speech is so halting and fragmentary that conversation is virtually
impossible.
2. Speech is very slow and uneven.
3. Speech is frequently hesitant and jerky; sentences may be left
uncompleted.
4. Speech is occasionally hesitant, with some unevenness caused by
rephrasing and groping for words.
5. Speech is effortless and smooth, but perceptively non-native in speed
and evenness.
6. Speech on all professional and general topics as effortless and smooth
as a native speaker’s.
 Proficiency descriptors
Comprehension
1. Understands too little for the simplest type of conversation.
2. Understands only slow, very simple speech on common social and touristic
topics; requires constant repetition and rephrasing.
3. Understands careful, somewhat simplified speech when engaged in a
dialogue, but may require considerable repetition and rephrasing.
4. Understands quite well normal educated speech but requires occasional
repetition and rephrasing.
5. Understands everything in normal educated conversation except for very
colloquial or low-frequency items, or exceptionally rapid or slurred
speech.
6. Understands everything in both formal and colloquial speech to be
expected of an educated native speaker.
 Interview (IELTS: Part 1)
Let’s talk about your home town or village:
• what kind of place is it?
• what’s the most interesting part of your town/village?
• what kind of jobs do the people in your town/village do?
• would you say it’s a good place to live? (why?)

Let’s move on to talk about accommodation:

• tell me about the kind of accommodation you live in?
• how long have you lived there?
• what do you like about living there?
• what sort of accommodation would you most like to live in?
 Monologue (IELTS: Part 2)

Describe something you own which is very important to you. You

should say:
• where you got it from
• how long you have had it
• what you use it for; and
• explain why it is important to you.

You will have to talk about the topic for 1 to 2 minutes. You have
one minute to think about what you're going to say. You can make
some notes to help you if you wish.
 Interaction with fellow candidates - Sample card 1

WHAT MAKES A GOOD FRIEND?

You are going to talk to a partner about what you value in your friends. Look at the
suggestions below:
support shared interests kindness
a shoulder to cry on fun to be with
honesty
other …

1. Do you think it’s better to have one or two really close friends, or a wider circle of
less close friends?
2. What are the qualities in yourself that you think friends value?
3. There is an English saying “Blood is thicker than water”, meaning that family
relationships are more important/reliable than relationships with friends. Do you
agree with this?
 Interaction with fellow candidates - Sample card 2

MOBILE PHONES
Look at the statements below. Tick the ones you agree with:
“I hate it when phones ring at the theatre or cinema.”
“If you have a mobile phone you never feel alone.”
“It’s really dangerous to drive and phone at the same time.”
“I feel safer with a mobile phone.”
“I hate them – people look stupid walking around talking on the phone!”
1. Exchange your opinions about mobile phones with your partner. Talk about reasons
why people have them. What advantages do they have over conventional phones?
Are there any disadvantages?
2. What limits (if any) should be put on when and where mobile phones can be used?
3. In what ways, for better or worse, is technology changing how we communicate with
each other? What about future developments?
 Assignment
1. Work in groups of three.
2. Administer two speaking tests. Use an audio recorder on your
mobile.
3. Work in groups of three. Conduct two tests, using the following
techniques: “Interview” and “Interaction with fellow candidates”.
4. Compare the techniques and discrepancy between them as well as
between the evaluation of the raters.
5. Present the results at our seminars.
6. Use different questions and topics in two tests but make sure that
they are of the same level of difficulty.
 Assignment (Part 1)
Interview
1. Student A is a tester. Make sure that Students B and C do NOT see the
questions before the test.
2. Students B and C are testees.
3. Conduct the test. Record it. Use an audio recorder on your mobile.
4. All three students evaluate the performance of students B and C using
the recording and rubrics.
5. Do not show each other the scores WHILE grading because you might be
influenced by grading of your peers.
6. Compare the discrepancy between the raters (inter-rater reliability).
 Assignment (Part 1) Report Sample

Evaluation Criteria Rater 1 Rater 2 Rater 3 Discrepancy

Fluency and coherence high
Lexical resource minor
Grammatical range and accuracy high
Pronunciation minor
AVERAGE ___ / 10 ___ / 10 ___ / 10 high
 Assignment (Part 2)
Interaction with fellow candidates
1. Student A is a tester. Make sure that Students B and C do NOT see the
questions before the test.
2. Students B and C are testees.
3. Construct a card and conduct the test. Record it. Use an audio recorder
on your mobile.
4. All three students evaluate the performance of students B and C using
the recording and rubrics.
5. Do not show each other the scores WHILE grading because you might be
influenced by grading of your peers.
6. Compare the discrepancy between the raters (inter-rater reliability).
 Assignment (Part 2) Report Sample

Evaluation Criteria Rater 1 Rater 2 Rater 3 Discrepancy

Fluency and coherence high
Lexical resource minor
Grammatical range and accuracy high
Pronunciation minor
AVERAGE ___ / 10 ___ / 10 ___ / 10 high
 Assignment (Part 3)
Compare the discrepancy between the two techniques and do
analysis. Which technique seems to be more reliable and valid?
Why?

Raters Test 1 Test 2 Discrepancy

Rater 1 high
Rater 2 minor
Rater 3 high
AVERAGE ___ / 10 ___ / 10 minor