Institute Information:

This is a draft of a final achievement test designed to assess students who have completed a
half-year English course, consisting of three one-hour classes per week. The language
school offering this course provides a range of programs, including business English,
conversational courses, English for health care, and English for adult beginners. The course
for which this achievement test is designed is intended to meet the needs of students who
are planning to participate in a student exchange program. The course aims to equip
students with the necessary grammar structures and vocabulary to study and live abroad.
Passing the final exam certifies students at level B1 in the English language, which is the
required level for applying to a student exchange program in Central Europe.

The cost of the half-year course and the final exam, which can be added to a professional
curriculum vitae, is 20,000 pesos. For this fee, the institution guarantees a proper learning
environment, including sufficient chairs and tables for each participant, a projector, and
speakers for practicing and testing listening exercises. Additionally, each group is assigned
two teachers to ensure better monitoring of students' progress and the final exam’s
reliability since the speaking will be assessed by two teachers (Hughes, 1998). The course
is offered in two schedules, morning and afternoon, allowing teachers to pilot the exam and
students to become familiar with the exam format. Each group receives a draft of a real
exam, which is analyzed when the results are tallied and edited before being applied as the
final exam for the other group. This also improves the reliability of the exam because it
helps to reduce errors in the design (Hughes, 1998).

Students’ Group Information:

The group is composed of bachelor's and master's degree in different specializations
students between the ages of 20 and 30, who want to participate in an exchange program
and need to achieve level B1 in English to be accepted. The course accommodates a
maximum of 18 students, and the minimum requirement for the course to be offered is 10
students. At the beginning of the course, candidates take a diagnostic test, and based on
their results, they are placed in different groups. Students taking this course to obtain a B1
certificate are those who scored enough to be at level A2 when starting the course. Those
who did not score high enough were recommended a more intensive course with more
hours per week or a one-year course.

Content Information:
The exam is designed to assess students' abilities to attend lessons in a foreign country,
travel around and use English in everyday situations such as shopping and socializing. The
learning objectives of the course, designed to take students to level B1, are shown in the
following chart:

Learning objectives Speaking Writing Listening Reading

Communicate personal details
Use appropriate strategies to express
preferences, make suggestions and
negotiate the details of a meeting, such
as the location, time, and duration.
 Explain and justify opinions
Expressing ideas and arguments clearly
and effectively
Describe experiences, events, wishes and
aspirations.
Follow clear simple directions
Identify the main idea on familiar
matters regularly encountered in work,
school and leisure time.
Use appropriate vocabulary and sentence
structure
Take notes of specific information such
as numbers and names of people, streets
and other places
Address people formally
Identify the purpose and tone of a text,
such as informative, persuasive, or
entertaining.
Understand and use synonyms,
antonyms, and homonyms correctly.
Use pronouns, prepositions, and
conjunctions correctly.
Scan a text to locate specific information.
Evaluation of situation and correct use of
grammatical tenses
Make connections between the text and
their own experiences or other texts.

To ensure the quality of the assessment, test items must align with the learning objectives of
the course. This alignment ensures that the test measures the intended knowledge and skills
and promotes transferability as students can apply their learning to real-world situations
beyond the classroom (Hughes and Palmer, 2008).

Exam Structure:
The exam will assess students’ productive as well as receptive skills, measuring all four
language skills: writing, speaking, reading, and listening. Assessing all four language skills
in a language exam can help reduce negative backwash and improve the test's validity
(Hughes,1998). Measuring multiple skills allows for a more comprehensive assessment of a
student's overall language ability, ensuring the test accurately reflects their proficiency level
in the language being tested. Including multiple skill areas in the test can also increase the
authenticity and relevance of the test tasks, helping to ensure the test measures the skills
that students are likely to use in real-world contexts (Hughes,1998).

As mentioned above, the students will become familiar with the test format through a
sample test first. The results from the sample test will be reviewed and analyzed to uncover
any mistakes in the test design. A piloting phase is an important step in the test
development process, allowing for evaluation and improvement before the exam is used for
its intended purpose. This improves the validity, reliability, and fairness of the exam
(Hughes,1998; Mendoza, 2014; Banerjee, 2016).

The test will take place during the last week of the course, allowing test-takers and teachers
three hours for test administration. The reading, listening, and writing parts will take half an
hour each, in that order. The speaking exam will take an hour and a half. Some students will
take the speaking exam on the same day as the writing exam, while the rest will come on
the third day for the speaking part.

The receptive skills will be assessed on the first day, while the productive skills will be
assessed on the second and third days. This approach aligns with Bloom's taxonomy, which
emphasizes the order of skills that students should develop. In the reading and listening
sections, students will mainly be tested on their ability to understand and analyze. In the
writing and speaking sections, students will be asked to evaluate and create, which are the
skills at the top of Bloom's taxonomy and will be tested last (Anderson and Krathwohl,
2001).

In the reading part, students will be asked to read two texts, one shorter than the other, to
make the test progressively more difficult and to test scanning and skimming, which are
both important reading strategies. Having at least two items that test reading skills ensures
that the test measures cognitive complexity and depth of knowledge (Anderson and
Krathwohl, 2001). By providing test-takers with unrelated texts, they are given a fresh start
with each new text. This can help reduce the carryover effect of mistakes made on earlier
texts, which improves the reliability of the test results. Students will have 10 and 20
minutes to read the texts and answer the questions.

The writing exam will be also divided into two sections. In the first one, students will be
required to react to a Facebook post from their university, while in the second section, they
will write a longer email expressing their thoughts on the university's studying program.
Each section will be allotted 10 and 20 minutes, respectively. The listening exam will also
be divided into two sections, with tasks increasing in difficulty.

The speaking exam will consist of three sections, with the first two being individual and the
last being a group conversation. At the start of the speaking exam, the examiner will
address each candidate individually for about three minutes. During this time, candidates
will introduce themselves, answer personal questions, and express their opinions on a given
topic. For the group conversation, students will be divided into teams of two or three and
engage in a five-minute conversation which improves the interactivness of the exam. If the
maximum number of students in the course is eighteen, they will be divided into six groups
of three, with five minutes allotted for group conversation and three minutes for individual
conversation, totaling eighty-four minutes. Students taking the speaking exam on the same
day as the writing exam will be given a 5-minute break.

To ensure accurate measurement of students' skills and content validity, the exam tasks will
align with the content information outlined above. To assess a broad range of skills and
knowledge, the test will use several types of items, including multiple-choice, matching,
fill-in-the-blank, short answer, and open-ended questions. Multiple-choice items will
measure lower-level cognitive skills such as recall and comprehension, while open-ended
items such as essays or short-answer questions will measure higher-level cognitive skills
such as analysis and evaluation (Hughes, 1998).

Writing and speaking will be assessed directly based on authentic tasks and communicative
purposes, such as asking for directions, agreeing to meet a friend, and writing a message.
Answer sheets will be used to protect the anonymity of the students and improve the
validity of the exam (Bachman, Palmer, 1996).

Result Analysis:
The exam is high-stakes and will significantly impact students' future opportunities. When
it comes to scoring the test, rubrics and scoring criteria are essential tools to ensure
consistency, reliability, and fairness in grading (Bachman, Palmer, 1996). Analytic rubric
will be used to evaluate the overall quality of a student's response in the writing and
speaking parts of the test. In both cases, the rubric will be focusing on use of grammar and
vocabulary, organization, and fulfillment of the task such as relevance to the topic and
length of the answer and how developed is the answer.

The receptive skills will be graded with the answer key, which will include all options that
might be accepted as correct. Each item in the listening and reading section will have a
value assigned to it depending on the difficulty of the question. All parts of the exam will
carry equal weight, 25% for reading, listening, writing, and speaking. This equal weighting
will encourage students to practice all skills equally and have a positive backwash effect.

The minimum passing grade will be seventy percent of the total possible points from all
parts, with grades of C, B, and A awarded for scores between 70-80%, 80-90%, and over
90%, respectively. Feedback will be sent to students via email, including information on
their strengths and weaknesses and suggestions for improvement. Upon passing the exam,
the student will be asked to schedule an appointment to collect their certificate. The results
of the exam can also be used to identify areas where students need additional support and
lead to improvements in the course program.

The limitations of the test include the time constraints, which may affect fairness for
students taking the speaking exam immediately after the writing exam (Banerjee, 2016).
Another limitation is the time of day when students take the exam, as energy levels may be
lower after attending university lessons or other duties they might have before taking the
test in the evening schedule. However, the practical solution ensures that the exam is
conducted during the time when students and teachers are available.

Bibliography:

Anderson, L. W., Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and
assessing: A revision of Bloom's taxonomy of educational objectives. Addison Wesley
Longman. https://www.uky.edu/~rsand1/china2018/texts/Anderson-Krathwohl%20-%20A
%20taxonomy%20for%20learning%20teaching%20and%20assessing.pdf
Bachman, F. L., & Palmer, A. S. (1996). Test usefulness: Qualities of language tests. In
Palmer, Bachman (Eds.), Language testing in practice: Designing and developing useful
language tests (pp. 17-43). Oxford University Press.
https://octovany.files.wordpress.com/2013/12/language-testing-in-practice-bachman-
palmer.pdf

Banerjee, H. L. (2016). Test fairness in second language assessment. Working Papers in

TESOL and Applied Linguistics, 16(1), 54-59. https://doi.org/10.7916/D8NG62KG

Hughes, A. (1998). Test techniques and testing overall ability. In A. Hughes (ed.), Testing
for language teachers (second edition, pp. 59-74). Cambridge University Press

Hughes, A. (1998). Achieving a beneficial backwash. In A. Hughes (ed.), Testing for

language teachers (second edition, pp. 44-47). Cambridge University Press.

Hughes, A. (1998). Reliability. In A. Hughes (ed.), Testing for language teachers (second
edition, pp. 29-42). Cambridge University Press.

Hughes, A. (1998). Validity. In A. Hughes (ed.), Testing for language teachers (second
edition, pp. 22-29). Cambridge University Press.

Mendoza Ramos, A. (2014). La validez en los exámenes de alto impacto: Un enfoque desde
la lógica argumentativa. Perfiles Educativos, 37(149), 169-186.
https://www.redalyc.org/pdf/132/13239889010.pdf