Job Description

Job Title: Assessment Research and Analysis Manager

Job Level (GJC): 5

Reports To (Job Title): Head of Assessment Research

Job Function: Publishing/Content

Job Family: Commissioning/Acquisitions

Career Stream: Individual Contributor / Professional

Division or Group Function: ELT

Country and Location: UK

Job Purpose – in brief

The purpose of this job is to support the activities of the Research and Analysis team in the design and
continuous validation of English language tests. The role is responsible for documenting, carrying out,
improving, and automating research and analysis processes.

Key Accountabilities & Relationships – what I need to do and who I need to engage
• Prepare and manage large data sets, such as test taker response data, assessor rating data and
expert judgments in formats (tabular and text) such as Excel, CSV and Power BI engaging with the
Research team.
• Manipulate data in order to provide insights in answer to identified research questions, engaging
with the Research team.
• Carry out validation processes such as pretest analysis, live seeding and quality assurance script
production, engaging with the Research, Delivery, and Content teams.
• Maintain and update process documentation engaging with the Research team.
• Maintain and expand automated processes in Python, such as automated pretest analysis and
automated seed analysis, engaging with the Research team.
• Create regular reports on test performance, combining data from multiple sources, for internal
and external audiences.
• Co-ordinate research activities such as CERF alignment activities with third parties, including
freelance consultants.
• Contribute to research objectives (qualitative and quantitative) to validate our assessments,
engaging with the Research team.
• Contribute to research design, such as researching test performance, engaging with the Research
team.
• Contribute to internal and external research publications (e.g. academic journals), engaging with
third party researchers, consultants, and the Research team.
• Communicate research insights to internal audiences such as Sales and Marketing, and to external
audiences, including test stakeholders such as teachers, students, test score users and Ministries of
Education in remote and in-person presentations.

Success Measures – how I succeed

Job Description
• Data management will be error free and meet deadlines
• Processes carried out will be error free and meet deadlines.
• Improvements to process will be identified and documented in such a way that documentation can
be used independently by new users without training.
• Processes will be automated in Python so that they are more efficient and easier to carry out than
a manual alternative
• Communications with both internal and external stakeholders will be timely, clear and efficient.
• Presentations of research insights to any type of audience will receive positive feedback.

Knowledge, Skills & Attributes – what I need to know

Essential Desirable
Master’s degree or published papers in language assessment, computational
X
linguistics, applied linguistics, or a related field
Ability to handle large data sets accurately and efficiently X
Programming capability in Python, R, or similar language X
Ability to use Excel to manage and manipulate data X
Knowledge of English Language Teaching assessment X
Familiarity with Rasch measurement concepts X
Ability to use software to conduct Rasch analysis (e.g. WINSTEPS and FACETS) X
Familiarity with a range of statistical techniques used in social sciences X
Knowledge of natural language processing, educational data mining, or learning
X
analytics
Working knowledge of with PowerBi or SQL X
Resources – what is available to me

Document Management
Drafted by Colin Finnerty Date 2023-01-27
HR BP review by Jo Wilkins Date 2023-02-02
Reward evaluation by Date