Research

Review Presentation
English Language Testing & Assessment

Massaya Rachawong
PhD Student ID: 64010191004

ELT
MEd & PhD Programme
English Language Teaching
 1. Introduction of the research
2. The intended purposes of the assessment and
decisions made based on assessment outcomes
3. The characteristics of the construct
4. The characteristics of the test-takers and assessment
OUTLINE context
5. The characteristics of the test or assessment task
6. The scoring method of the test or assessment task
7. The test development procedures based on the
research under review
8. The method(s) of evaluating the test quality
9. My opinion about the research under review
 Measuring L2 Oral Pragmatic Abilities for Use in Social Contexts:
Development and Validation of an Assessment Instrument for L2
Pragmatics Performance in University Settings
Introduction of the
Naoki Ikeda
research
Submitted in total fulfilment of the requirements of
the degree of Doctor of Philosophy
September 2017

Linguistics and Applied Linguistics

School of Languages and Linguistics
Faculty of Arts
The University of Melbourne

3
 The intended To assess L2 students’ pragmatic abilities
purposes of the utilized in university contexts
assessment and (oral pragmatic features, integrating how speakers manage the
extended discourse)
decisions made based
on assessment
outcomes

4
 The construct of L2 pragmatic abilities utilized
The characteristics of for language activities at university

the construct

oral pragmatic features,

integrating how speakers manage the extended discourse

5
 Test Takers: 67 international L2 students in
Australian English-medium education
-university students (N=44)
The characteristics of -language school students (N=23)
the test-takers and
The test takers’ English proficiency levels ranged
assessment context from an equivalent of IELTS5 level to near-native
level.

6
 Oral role play tasks:
-the dialogue task
The characteristics of -the monologue task
the test or
assessment task Used to elicit the target features of pragmatics
from test takers.

7
The scoring method The test takers’ task performances were scored by the
raters according to the rating criteria and descriptions
of the test or developed for the study.
assessment task

8
 Rating Criterion and
Descriptions of Pragmatically
Competent Test Takers

9
RATING
RUBRICS
(EXAMPLE)

Each feature classified the test

takers at four band levels with
score 4 (maximum) and score 1
(minimum).
The test development
procedures

11
Final Version of the Test Tasks

12
 Firstly, using the test scores, quantitative analyses were carried
out to explore test reliability and inter-rater reliability on
Classic Test Theory using SPSS version 23.
The method(s) of Secondly, four facets (test taker, rater, test task, and rating
evaluating the test criterion) were specified in the multi-faceted Rasch
measurement analyses employing the computer program,
quality FACETS (Linacre, 2015).

Test reliability indicated by Cronbach Alpha was examined for

each task.

Each test taker’s total score for one task was calculated by
adding all the scores on the rating criteria assigned from three
raters.

13
Test Item Reliability
 This research provides very clear concepts of
the research design in development a test.

My opinion about the It is very useful for those who are interested in
the field of language testing and assessment.
research under
review

15