Final Test Project

203 Grammar Final Test Project

Introduction
Grammatical knowledge is one of the major components outlined by Bachman and
Palmer (2010) in their definition of Language Knowledge. Bachman and Palmer (2010) state
that grammatical knowledge falls under organizational knowledge and can be broken down into
three primary areas: knowledge of vocabulary, knowledge of syntax, and knowledge of
phonology/graphology. According to Bachman and Palmer (2010), Grammatical knowledge is
involved in producing or comprehending formally accurate utterances or sentences (p.44). In
addition, Jones (2012) states grammar is not only a set of abstract rules, but is fundamental to
the creation of meaning, or grammatical ability (p.248). So, L2 communication is not only
comprised of grammatical knowledge, but also grammatical ability, or knowing when and how
to say what to whom (Larsen-Freeman & Anderson, 2011, p.115).
Grammatical knowledge is reflected in explicit knowledge of rules and formulas, whereas
grammatical ability is found in the degree of communicative competence a learner demonstrates.
In ESL teaching practices, grammatical ability in addition to rule knowledge is required for the
production and reception of formally accurate communication, and is therefore important to both
grammatical instruction and assessment.
Due to this distinction between knowledge and ability, assessing a learners grammatical
ability in an authentic situation remains a difficult goal to achieve in comparison with the ease of
eliciting knowledge of rules and forms through grammar translation (a method used to teach and
assess L2 knowledge since medieval times) (Purpura, 2012). However, both grammatical
knowledge and ability are important to assessment because many learners and teachers feel that
grammar is the most salient and significant part of language learning (Larsen-Freeman & CelceMurcia, 1999; Purpura, 2004; Jones, 2012).
Finally, grammatical ability and knowledge should both be assessed in L2 learning
through techniques that allow testers to make distinctions among these components, when
possible, so that assessments can be used to provide more precise information to users of test
results (Purpura, 2011, p.82). Grammatical assessment techniques can involve objective-type
tasks, for example, selected response tasks (such as multiple choice, grammatical judgment
activities, lexical list activities) (Purpura, 2004). Additionally, subjective techniques for gathering
information about grammatical knowledge and ability such as limited-production tasks (like gapfilling or cloze activities) and extended-production tasks (such as essays, stories, problemsolving activities) can be used to design grammar assessments (Purpura, 2004).
I will outline the process of creating a grammar assessment for an intensive Academic
English Program at INTO Colorado State Universitys (CSU) upper-intermediate grammar class,
203 Grammar. In this paper, I will outline the test purpose and type, scores, TLU domain,
construct definitions, design, test task descriptions and the test creation process. I will then detail
the piloting process and test results using statistical analyses. Then, I will discuss specific
information in terms of the reliability and validity of this assessment. Lastly, I will reflect on this
test creation and piloting process.
Description of the Test
Purpose of test
The purpose of the test is to demonstrate that students in the INTO 203 Grammar class
have mastered most or all of the material specified in the class objectives and are ready to
advance. The grammatical ability goal of the class is to, Raise awareness of grammar points
taught at the upper intermediate level and equip learners with greater grammatical proficiency
for communicative use in other skill area classes (INTO CSU, 2014). The knowledge objectives
are as follows:
present perfect and present perfect progressive
past perfect and past perfect progressive
expressions of future including simple future with will and be going to, simple
present, present progressive, future progressive, and future perfect
modals of polite request, necessity, advisability, probability/certainty (INTO CSU,
2014).
The scores are used in conjunction with scores given for homework, in-class activities,
quizzes and a cumulative mid-term exam to determine whether a student has firmly grasped the
material and should advance. The 203 Grammar final, which tests all of the points taught in the
term is 15% of the overall percentage grade (INTO CSU, 2014). The overall score according to
INTO CSU that a student must achieve in the 203 Grammar class to advance is 79.6%.
For many students, this assessment is a low to moderate stakes test with the possibility
for high-stakes decisions to be made from it. It will determine if they can move forward within
the Academic English Program, but it can also determine whether or not their government
continues funding their education at INTO CSU. Although INTO CSUs policy states that
students may take any INTO class three times before they are asked to leave the program,
students are motivated by scholarship awards from their home countries to pass their classes the
first time so they can gain admittance into universities (INTO CSU, 2014). In addition, INTO
CSU students final goals typically include passing the TOEFL or IELTS exams. Students want
to pass these exams because their scores are used as part of the admissions decisions into
English-speaking universities. Therefore, studying for the 203 Grammar final can also help
students prepare for the TOEFL or IELTS.
Type of test and Interpretation of scores
The 203 Grammar final is a summative exam at the end of the course that is
designed to measure achievement. The scores are criterion-referenced because the final exam is
based on the syllabus/curriculum objectives and the scores are used to compare test-takers to the
criterion or cut score established by the proficient performance standardto obtain a description
of the specific knowledge and skills each student can demonstrate (Miller, Lynn & Gronlund,
2009, p.40-41). According to Huitt (1996), there are four major dimensions which can be
considered when distinguishing a criterion-referenced and a norm-referenced test. They

dimensions are purpose, content, item characteristics and score interpretation. Considering these,
my purpose with this assessment was to determine whether each student [had] achieved specific
skills or concepts; the content of the assessment was aimed to [measure] specific skills which
make up a designated curriculum; the item characteristics were created to [test each skill] by at
least four items in order to obtain an adequate sample of student performance and item
difficulty was not varied, but based on activities students should be familiar with and able to
master; and lastly, the score interpretation [compared each individual] with a preset standard for
acceptable achievement. The performance of other examinees [was] irrelevant (adapted from
Popham, 1975). According to these dimensions, this 203 Grammar assessment most closely
aligns with a criterion-referenced test, therefore shall be considered as such. This final is used to
evaluate how well grammatical knowledge and ability were learned by an individual during the
203 Grammar class.
TLU Domain Specifications
The Target Language Use (TLU) domain is academic English at the university level.
Most, if not all, students studying at INTO CSU wish to pass the TOEFL or IELTS so they may
be admitted to an English-speaking university to begin their undergraduate or graduate studies in
their chosen majors. One such task demanded in this TLU domain is the ability to write personal
statements for college applications. Within this domain, this task requires knowledge of the
perfect aspects, past, present and future tenses and modals (all grammatical points covered in 203
Grammar). The task of personal statement writing in this TLU domain usually requires students
to write an essay about their past achievements, future goals and personal strengths that they can
bring to a particular program. For characteristics of this domain, see Appendix A.
Construct definition
The test is based on the INTO CSU syllabus for 203 Grammar. The language constructs
and subconstructs that are to be assessed include grammatical knowledge and ability. These
constructs are based on Bachman and Palmers (2010) Language Knowledge, the discussion
between grammatical knowledge and use from above and the INTO CSU 203 Grammar syllabus.
See the figure below:
Language Constructs
Organizational Knowledge
Grammatical
Textual
Grammatical Knowledge
Knowledge of productive
vocabulary
Knowledge of syntax
Knowledge of forms and
functions
Cohesion and organization
Present perfect and present perfect progressive (vs.

simple present)
Past perfect and past perfect progressive (vs. simple
past)
Expressions of future including simple future with will
and be going to, simple present, present
Progressive, future progressive, and future perfect
Modals of polite request, necessity, advisability,
probability/certainty
Pragmatic Knowledge
Functional
Sociolinguistic
Grammatical Ability
Ability to use grammatical
forms and functions in a
meaningful and
communicative way using
ideational functions
Ability to understand and

produce cohesive, wellorganized and clear
language in the academic
register
Understanding connections between grammar forms and

meaning in both language production and
comprehension
Producing accurate and more complex grammar in
written contexts
Figure 1: 203 Grammar Constructs and Subconstructs. This figure outlines the constructs and subconstructs within
the overall grammar construct.
The aspects that are tested, but not being assessed (or abilities that are assumed) include
graphology and receptive vocabulary knowledge (so, if a students does not understand a word in
the instructions or the writing prompt, they may ask the instructor for a definition). In addition,
unless the instructor cannot read the students written work, graphology is not included in the
assessment constructs for 203 Grammar. Pragmatic knowledge is assessed, however, students
creative content (or imaginative functions) is not being assessed. Finally, abilities that are not
included in the assessment are speaking and listening abilities (there is no oral or audial
component to this 203 Grammar final).
Table of Specifications
Miller, Linn and Gronlund (2009) define a table of specifications (ToS) as a tool used to
ensure that classroom tests and assessments measure a representative sample of instructionally
relevant tasks (p. 142). The (ToS) for 203 Grammar was created to help guide the
development of a grammar assessment that reflects the amount of time spent in class on a
particular course objective and uses task types familiar to students during instruction. According
to Bachman and Palmer (2010), a ToS includes a description of the overall structure of the
assessment and the specifications for each type of task to be included in the assessment (p.144).
To create the 203 Grammar ToS three steps are involved: first, the objectives of the class must be
made clear and listed in the ToS across the top; second, the course content, topics or outcomes
must be listed and provided along the vertical axis of the ToS; lastly, the ToS can be developed
when comparing the objectives to the course content, topics or outcomes specified (Miller, Linn
& Gronlund, 2009, p.143-46).
To create the 203 Grammar ToS, I first examined the syllabus that is used to teach the
class. Based on my two terms of personal experience teaching the class, I then organized the
syllabus content by the approximate amount of time spent on each point of grammar (or on each
outcome stated in the syllabus). Then, by calculating the total amount of time spent in the
classroom (minus the total amount of time spent on taking quizzes, reviewing and taking the
mid-term and final), after calculating the approximate time consisting of actual instruction and
classroom activities, I calculated the time spent per course content area. This provided me with
the following table of time of instruction spent in the classroom per grammatical point:
Content areas
% of instruction time spent on each
7
content area
27%
20%
Future
13%
Modals
28%
Vocabulary
12%
Total:
100%
Figure 2: Time Spent on 203 Grammar Content Areas. This figure shows the percentage of time of instruction spent
Present prefect/present perfect progressive
Past perfect/past perfect progressive
on each content area in the 203 Grammar class.
Using this table as a starting point, I then created a ToS using the syllabus-stated
objectives, the course content points and these time percentages to calculate the number of items
needed to successfully reflect the instruction tasks in my assessment. I used the information
about the time of instruction/classroom activities spent on an objective to inform my decision
about how many task items to include for that particular objective; in other words, The
percentages in this table indicate the relative degree of emphasis that each content areawill be
given (Miller, Linn & Gronlund, 2009, p.77). Based on my experience, however, although the
203 Grammar syllabus (INTO CSU, 2014) states that one of the objectives is for students to
produce grammar in various speaking and writing contexts, it would be inaccurate to include
speaking in the ToS because during the instruction time, although students spoke in class,
speaking was not assessed nor used as a technique to elicit grammar in the classroom. So, I
removed the speaking portion from the objectives and did not include it in the ToS.
Included in the ToS, across the top line are the objectives for the course. The vertical
column to the left outlines the course content. There are a total of 35 items on the test. Within the
four objective areas of (recognizing/comprehending grammar forms; understanding forms and
meaning in production and comprehension; producing grammar in writing contexts; knowledge
of vocabulary) there are five content areas (present perfect/present perfect progressive (vs.
simple present); past perfect/past perfect progressive (vs. simple past); expressions of future; and
modals). Within each cell of the ToS the number of items per content area and objective are
indicated. In addition, the numbers in italics below the number of items indicate which task on
the test each item is. For example, in the content area Modals and the objective area
Producing grammar in writing contexts there are six items and on the test part 4 questions 1-6
correspond to this objective and content area of the test. In addition, the number of items for each
content area and each objective are indicated in the column and row titled Total # of items. The
percentage of items on the test per objective and per content area is also indicated in the column
and the row % of items. (Please see Appendix B for ToS.)
Description of test tasks
The instructions for any assessment tell the test takers how they are to proceed in taking
the assessment, the types of tasks they are going to encounter, the ways in which they are
expected to respond to these tasks, and how their responses are going to be scored (Bachman
& Palmer, 2010, p. 373). The tasks and instructions for the 203 Grammar final exam are all
things that should be familiar to the students taking the test. The instructions and tasks are
presented in the target language, English. They are written only, so they are visual (if a student
asks for clarification from the teacher, then there is an audial component). The instructions are
brief in parts 1-4 and lengthier for part 5, the short answer section. Examples are provided for
students for each subsection. Although no example is provided for the short answer item, the
instructions inform students of the three areas they will be graded on in their response. Each
section of the test is marked clearly so that students know when they are finished with one task
(such as fill-in-the-blank or cloze). The test sequence ranges from structured items where
students must answer supply items or select items to a less structured item in the form of an
extended response item. This sequence was chosen so that students begin the assessment with
items that should be quicker and easier to complete (because they have had ample in-class
practice with these items) and gradually becomes more challenging as students warm-up their
grammatical knowledge and ability.
The input for the assessment is visual because of the reading and writing and is languagebased in the target language (English). The input is supplied in short chunks already familiar to
students, allowing them a shorter processing time. The supply and select items are used to elicit
grammatical forms. Then there is a prompt to elicit an extended production response in which
students should be able to use multiple tenses/aspects and modals in a cohesive, coherent
paragraph. The speed is normal: students should be able to process the input at the same speed at
which input was processed in class. Lastly, the test is reproduced, not live.
Approximately 90% of the language of the input comes from the Academic Word List
and the 1-2K word lists. The last 10% of the words were off-list words, or words from the offword list, but should be recognized by students because they are words used in classed (such as
vocabulary) or names of their peers/teacher. The textual characteristics are short, cohesive and
clear. The input pragmatics are functionally heuristic because they are used for teaching and
learning, for problem solving, and for the retention of information (Bachman & Palmer, 2010,
p. 47). The input is also ideational in section 5 because they ask students to use the target
language to inform, to express or exchange information about ideas, knowledge or feelings
(Bachman & Palmer, 2010, p. 46-47). The input is academic in the formal register and uses
Standard English. The topical characteristics of the input are academic and based on information
students were exposed to during instruction time such as timelines and daily life themes.
In terms of the expected response, the format is visual for writing and it is languagebased in English. The length of expected response varies from 1 or 2 words to full sentences to a
paragraph. For example, in part 1, section 1, students are expected to provide one-word
10
responses whereas in part 5 they are expected to produce a minimum of eight sentences. The
types of responses are selected, supply, limited production and extended response.
Organizational characteristics of the response include grammatical knowledge of perfect and
future tenses and aspects, modals, the ability to read and write using various language features
with a focus on form, usage and meaning. Textually, students responses should be cohesive,
coherent and well organized. Pragmatically, the expected responses for parts 1-4 are heuristic and
ideational. For part 5, responses should be ideational and creative, use the academic and formal
registers and genre, Standard English dialect and draw upon personal and cultural references
(such as personal goals, methods they used to achieve those goals, etc). Topics should be
informed by the academic information in the class.
The relationship between the input and expected response is non-reciprocal, in that a
students response will have no effect on the input. The scope of relationship is narrow because
the processing time and amount of input is limited. The input should be familiar material and
responses are made based on a relatively limited amount of input, such as no more than a few
sentences to a short paragraph (Bachman & Palmer, 2010, p. 81). The relationship is direct for
parts 1-4 where students only need to interact with and respond to the input provided. However,
part 5 is indirect because students need to use their personal experiences to answer the question
well. (See Appendix C for the Test Task Characteristics.)
Students have 60 minutes for the assessment. Ten minutes in the beginning are allowed
for the students to look at the test as the instructor talks about the directions for each section. The
remaining 50 minutes are for students to complete the 203 Grammar assessment. The
recommended time spent on each task by test-takers is shown in Figure 3 below:
Section
Part 1
Part 2
Recommended time spent for test-takers (minutes)

5
5

Part 3
Part 4
Part 5
Total
11
20
10
10
50
Figure 3: Time Per Section. This figure shows the recommended time for test-takers to complete each section of the
final assessment.
The time for each section was determined based on my empirical observations of how much time
students needed to complete similar tasks in-class.
Item writing & editing, development of rubrics
The writing, editing and development of the items and rubric involved analyzing the
types of tasks used in the classroom and assigned for homework. The first version of the test was
long and included some tasks that students were not familiar with. Four instructors edited that
first version. Three work at INTO CSU, have taught the 203 Grammar class and are familiar with
the course content and one works at CSU in the TEFL/TESL graduate program. Through
discussions with the three INTO CSU instructors, some items were deleted and others were reworded to elicit grammatical responses that made distinction between the different content areas.
This was important feedback, because at the 203 Grammar level, more than one grammatical
form can be used accurately to convey the same meaning (for instance, in the cloze, students can
use either simple past or past perfect to talk about a finished event in the past and both answers
would be correct). In addition, the length of the test was altered as some items were discarded.
The rubric for the writing prompt in section 5 was developed based on an analysis of a
short and informal score reporting sheet used at INTO CSU. This sheet tells students their score
out of 10 they received on a writing activity. The score sheet indicates that students receive six
points for accurate grammatical usage, two points for mechanics (spelling, punctuation) and two
points for topic development (staying on-topic, providing details and information). To elaborate
this rubric and include enough details so that any teacher could use it to grade section 5, I
adapted and synthesized information from the TOEFL and IELTS independent writing rubrics. I
chose these rubrics to inform my rubric because they are the standardized tests, which evaluate
12
students academic English proficiency. They are also the tests most frequently taken by students
and used by universities for admissions purposes (which is a goal of the students who attend
INTO CSU classes). The rubric was edited to provide more specific language after being
reviewed by one teacher at INTO CSU and one teacher at CSU.
Pilot Test Procedure
Participants
There were nine participants who took the pilot 203 Grammar final. All participants
were studying at INTO CSU in the 203 Grammar class at the time of administration. They
ranged in ages from 18-28 with an English level of low- to mid-intermediate. Their L1s were
Chinese (two students) and Arabic (seven students).
Administration
The test was administered during class to students as a practice exam two days before
their actual 203 Grammar final exam. The instructor passed out the exam. (Although provided on
the exam, students were not asked to fill in the CSU Honor Pledge since this was a practice
exam). The instructor informed the students that they should treat this practice exam as if it were
their actual final exam, therefore, there should be no talking or sharing of information. Then the
instructor went through the instructions for each section. Students were then given 50 minutes to
complete the assessment.
Scoring procedures
The overall score that a student must achieve to pass a class at INTO CSU is set by
INTO CSU as 79.6%. Although my intentions were to set the cut-score of this exam to the same
number, none of the students achieved this score. Therefore, since this test was administered as a
practice test, the cut-score was revised to 70%. This score was chosen because it is commonly
accepted as an indication of average achievement. In addition, Colorado State University states
that a C (or approximately 70%) is passing (Colorado State University, 2008). The 203
Grammar final was scored by one rater (a previous teacher of 203 Grammar). For section 1
subsection 1.1-1.3, each answer was given 0.5 points for a correct answer and 0.0 for an
13
incorrect answer for a total of 5 points on section 1. For sections 2 (5 points), 3.1-3.2 (12 points)
and section 4 (7 points), each correct answer was given 1.0 point and each incorrect answer 0.0
points. Using the writing rubric, the essays were given a score of 0-10. To find a score, the rater
assigned a score for each section of the rubric (grammar, mechanics and task completion), then
added them together. Using cross multiplication, a score out of 10 was calculated. For example,
if a student got a 10 for grammar, 10 for mechanics, but a 7 for task completion, they would
receive a 9/10 on section 5 because 10+10+7= 27 out of a total 30. Then by cross-multiplying
27/30 = x/10, the score (rounded to the nearest whole number) is found. A score report sheet was
then used to record the scores of each exam. (See Appendix D for the pilot test, Appendix E for
the answer key and Appendix F for the writing rubric and Appendix G for the score report sheet.)
Test Results
Item statistics
The purpose of item analysis on criterion-referenced assessments is to show how well
each item is contributing to the pass/fail decisions that are often made with [criterion-referenced
tests] (Brown, 2003, p. 20). The items for the 203 Grammar test were analyzed using Microsoft
Excel spreadsheets. Item analysis was conducted through B-index findings. According to Brown
(2003), the closer to 1.00 the B-index is, the more reliable the item. A B-index number for what
is closer to 1.00 is not defined, however, Brown (2003) states that the items with the highest
values should generally be kept (p. 22). Therefore items with a B-index of 0.65 or higher will be
considered items that could be kept on this test. Further discussion of the item statistics is found
below in the Discussion section of this paper. (For the complete item analysis, see Appendix
H.)
Descriptive statistics
Descriptive statistics are used to give simple, overall pictures of data (Flahive, 2014).
Using the mean, median and mode, as well as the standard deviation (SD), data about the general
performance of an assessment can be evaluated. The descriptive statistics for the 203 Grammar
14
assessment are below in Table 1:

Table 1
Summary of Descriptive Statistics
Mean
Median
SD
Raw Score
21.33
23
7.57
Reliability and standard error of measurement
The reliability for the 203 Grammar final was calculated using SPSS (Statistical
Package for the Social Sciences). SPSS was used to find Cronbachs Alpha, which is the most
widely used of the many methods for estimating internal consistency reliability (Chapelle, 2012,
p. 2). Cronbachs Alpha can be used for both dichotomous (or items that are scored right or
wrong) such as in sections 1-4 on the 203 Grammar assessment or polytomous scores (or items
that are graded on a scale) such as the short answer item in section 5 (Chapelle, 2012). In
addition, Cronbachs Alpha is an indicator of test reliability and, It is expressed as a value that
can range from 0, meaning variation in test scores is completely random, to 1, indicating that
variation is completely reliable, that is, attributable to one factor (Chapelle, 2012, p.2).
Another measurement to evaluate reliability is the standard error of measurement
(SEM), which estimates the amount of variation to be expected in the scores, from a test, if it
were given to the same student on more than one occasion (Miller, Linn & Gronlund, 2009, p.
119). Therefore, a low SEM would imply that a test is a reliable measure of students
performance, whereas a higher SEM would imply that there are too many variations in a
students performance and therefore the scores do not reflect a test-takers true abilities. A
summary of reliability information can be found in Table 2:
Table 2
Summary of Reliability
N of Items
Cronbachs Alpha
SEM
35
0.874
2.69
Description of masters/non-masters
The description of masters and non-masters can be formed using information from the
B-index raw scores. Masters were able to fully complete the assessment in the given amount of
time and earn a score of 70% or higher. Masters were able to show that they possess grammatical
15
knowledge and grammatical ability that allows them to gain an average or above-average passing
score, according to CSU standards. Three students were found to be masters. Non-masters did
not complete the assessment and earned a percent score of 69.9% or less. These students showed
a below-adequate understanding of the grammatical knowledge and ability objectives as outlined
in the syllabus. Six students were found to be non-masters.
Discussion
Critique of item performance
Miller, Linn and Gronlund (2009) state that, Item-analysis data from small samples
are highly tentative (p.361). Considering this, the B-index information collected from the pilot
administration is not a good representation of this assessment. Additionally, since the assessment
was administered in-class as a practice exam, I believe that students did not regard the exam
seriously and, therefore, many answers were incomplete and the motivation to perform well was
low. These circumstances, I believe, contributed to the low B-index values for the majority of
items on this assessment. For instance, section 1, item 6 has the highest B-index for this section
at 0.67 suggesting that of the ten items in section 1, item 6 is the best indicator of pass/fail
decisions based on this assessment. In addition, because three out of three students who passed
the exam answered the question correctly and only two out of six who failed answered it
correctly, this is further evidence that this item may be a good indicator of pass/fail decisions.
However, item 19 in section 3 subsection 1 also has a B-index of 0.67. Yet, this item is not a
good indicator of pass/fail decisions because zero out of three of the passing students answered
this question correctly, while four out of six of students who failed answered this item correctly.
Overall, items number 6, 25, 33 and 35 are candidates for good items to remain on the test, while
all other items (out of 35) received B-index scores significantly lower than the 0.65 cut-off point.
Although the B-index would indicate that most items are unreliable items, I believe a larger
sample size administered in a higher-stakes environment would give a broader statistical picture
16
of the B-index values for each item.

According to the descriptive statistics, the mean scores indicate a low central tendency
to deviate from the average score (Flahive, 2014). The mean raw score was 21.33 out of 39 (or
55%), which is a failing score. The median, which describes the point in the distribution at
which half of the cases fall above and below, was 23 (or 59%, which is a failing score) (Flahive,
2014, p.3). This information was included because there was an outlier in the data. The outlier
received a low raw score of 7.5 out of 39 points. Since this score affects the mean and SD, the
median was included in the statistics because it is less sensitive to such data points (Flahive,
2014, p.3). Additionally, the median score indicates low performance on the pilot test by most of
the participants. This is also supported by information from the B-index that shows that only four
out of nine students earned a score above the median.
The SD is the most widely used and most informative measure of score distribution or
variability (Flahive, 2014, p.3). According to Miller, Linn and Gronlund (2009), the SD
indicates the amount that each score deviates from the mean (p.504). The SD for this
assessment was 7.57, a high SD indicating that students scores deviated significantly from the
mean of 21.33. This signifies high variability within the scores and shows that students scores
were all over the place in terms of consistency. This indicates poor individual and/or item
performance. Additionally, the data do not appear to be normally distributed, as it is
recommended that the SD be 20-30% of the mean, whereas the SD on this assessment is slightly
above this percentage at 35% of the mean indicating that this assessment needs revision and/or a
larger sample size (Flahive, 2014, p.6).
Evaluation of Test Usefulness
Reliability
The criteria for test usefulness include reliability, construct-related and consequential
evidence for validity, practicality, authenticity and interactiveness. The test results presented
above help explain the reliability, validity and overall usefulness of the 203 Grammar
17
assessment.
The reliability of this test, according to Cronbachs Alpha, is fair: it is close to 1.00 (at
approximately 0.84). This suggests that the internal reliability of the test is strong and the
variations in scores can probably be attributed to a small number of factors. In addition, Miller,
Linn and Gronlund (2009) state, Teacher-made tests commonly have reliabilities between .60
and .85 [and] are useful for the types of instructional decisions typically made by teachers
(p.132). So, this assessment appears to have reasonable reliability in terms of curricular decisions
to be made for students in the INTO CSU program. In addition, the 2.69 for the SEM implies
that a students true score on the test would be within about 2.70 points. This seems low and
therefore may be a good predictor of students actual abilities and knowledge.
Validity
The construct-related evidence for validity can be evaluated in terms of authenticity as
the assessment relates to the TLU domain; interactiveness, or fairness (Bachman & Palmer,
1996); the outlined construct definitions, and description of test tasks.
According to the TLU domain, which was defined as academic, I believe this 203
Grammar assessment reflects the TLU domain as well as an intermediate assessment can. Both
the input and the expected response for the assessment are evaluated at the upper-intermediate
grammar level and the task to write about themselves in section 5 is reflective of the TLU task
described for writing a personal essay. In terms of interactiveness, this test was reviewed by
multiple professionals in the TEFL/TESL field. The first version was found to have some items
that required varied cultural knowledge (from cultures outside the US), so those items were
eliminated. The final pilot version, however, appears to be less bias and does not negatively
activate affective schemata. In addition, the instructions were revised to include simple,
accessible language at a level upper-intermediate ESL students should be able to comprehend
and respond to effectively. The test tasks were all tasks that students were familiar with and had
18
had ample practice responding to both in and out of class. In addition, the construct definitions
outlined were all reflected in the assessment. Grammar knowledge was assessed in the sections
where students have to consider form and function (such as section 1). Grammatical ability was
assessed in section 5 where students needed to combine both their grammatical and textual
knowledge with their functional and sociolinguistics knowledge to produce a paragraph about
their achievements. This test measured outcomes only listed in the constructs and the items were
specific to material in the course content.
The impact of the test or how it will affect the stakeholders (identified as students and
teachers) should help both students and teachers. The 203 Grammar final is high-stakes and
students must demonstrate a sufficient level of mastery in the class (79.6%) to advance in the
program or they risk losing their place at INTO CSU, their funding and their chance at taking the
TOEFL/IELTS and gaining admittance into an English-speaking university. However, this test
does not necessarily determine a students final grade in the class, because it is only 15% of the
grade. Depending on how a student performed throughout the class, the impact of this
assessment could be low, while still being informative for a student and a teacher about a
students readiness to advance and their mastery of 203 Grammar course content.
Finally, this assessment is practical. It is paper and pencil-based and since it is based on
a syllabus used by all 203 Grammar instructors, it should be able to be used by all 203 Grammar
classes.
Overall estimation of whether or not the test achieved its purpose
According to the reliability and validity information, I estimate that this test, if taken
seriously by students, would achieve the purpose of being a summative achievement test.
However, based on the B-index and the poor performance of many of the items demonstrates
that a larger student sample and statistical analysis is required to make a final judgment of this
assessment. Many items may need to be revised, while other items might prove to be more
19
reliable and valid than this pilot test indicates. Yet, this assessment went through multiple
revisions and was examined by four professionals and reflects the TLU domain testing only the
outlined constructs. Therefore, I feel this assessment accomplished its purpose as a 203 Grammar
achievement summative assessment.
Reflection on personal significance of test
In conclusion, I will end with a reflection about this project. I felt that going through
the test creation process was useful. It made me aware of the consequences of poorly written
instructions can have on test-takers. I am also more aware of the importance of knowing
objectives and how course content relate to those objectives and then how to translate those into
an assessment. Lastly, I feel that knowing what constructs to assess and how to assess them is
paramount in task/test creation and I feel better prepared to go into the field of TEFL/TESL able
to create or adapt assessments that will more accurately evaluate specific constructs.
There were multiple limitations for this pilot testing. The small number of participating
students was a limitation in that the scores were spread out greatly, but only among 9 test-takers.
Additionally, with a SD of 7.57, there was a significant level of variability in the performance of
test-takers, which ultimately affected the results. In addition, I found with this project other
limitations were time, willingness (on the part of teachers and students) and motivation of testtakers. Time was a problem because INTO CSU runs on a tight schedule and making the time to
administer this test was difficult. Willingness was a limitation because the instructor wanted to
change parts of the pilot to better fit her own classroom. This highlights an even larger limitation
that is: INTO CSU is currently attempting to standardize its assessments, however its curriculum
is not entirely standardized. Therefore some classes cover all the material on a syllabus and some
do not. Willingness and motivation of students was also an issue because I feel that students did
not approach this assessment as high-stakes because it was given as a practice exam for only a
few points, and therefore they were not motivated to perform to their best ability. Additionally, I
20
would approach this test in the future by using z-scores to analyze each item to better understand
the reliability and validity of each item.
Finally, I feel that this process helped me understand the necessary steps to create a
reliable and valid assessment that has positive and constructive impact on stakeholders. I will
apply this information to future assessments I administer and create in the TEFL/TESL field.
References
Bachman, L. & Palmer, A. (1996). Language testing in practice. New York, NY: Oxford
University Press.
Bachman, L. & Palmer, A. (2010). Language assessment in practice. Oxford, England: Oxford
University Press.
Brown, J.D. (2003). Criterion-referenced item analysis (The different index and B-index).
SHIKEN: The JALT Testing & Evaluation SIG Newsletter 7 (3) 18-24.
Chapelle, C.A. (2013) The Encyclopedia of applied linguistics. Carol A. Chapelle (Ed.). United
States: Blackwell Publishing Ltd.
Colorado State University - Registrars Office (2008). Retrieved from
http://registrar.colostate.edu/Data/Sites/1/pdf/Transcript-Information.pdf
Flahive, D. (2014). Chapter II: Basic psychometric concepts: 2.1 Descriptive statistics: The
building-blocks of research [class handout]. English Department, Colorado State
University, Fort Collins, CO.
Huitt, W. (1996). Measurement and evaluation: Criterion- versus normreferenced testing. Educational Psychology Interactive. Valdosta, GA:
Valdosta State University. Retrieved 2/23/2105, from
http://www.edpsycinteractive.org/topics/measeval/crnmref.html
INTO CSU (2014). Syllabus for AEIN8203: Grammar. (Available from INTO Colorado State
University, Spruce Hall, 150 Old Main Drive, Fort Collins, CO 80523)
Jones, W. (2012). Assessing students grammatical ability. In Coombe, C., Davidson, P.,
21
OSullivan, B. & Stoynoff, S. (Eds.), The Cambridge guide to second language

assessment (247-256). New York, New York: Cambridge University Press.
Larsen-Freeman, D. & Anderson, M. (2011). Techniques and principles in language teaching.
Oxford, England: Oxford University Press.
Miller, D.M., Linn, R.L & Gronlund, N.E. (2009). Measurement and assessment in teaching.
Upper Saddle River, New Jersey: Pearson.
Popham, J. W. (1975). Educational evaluation. Englewood Cliffs, New Jersey:
Prentice-Hall, Inc.
Purpura, J.E. (2004). Assessing Grammar. Cambridge, United Kingdom: Cambridge University
Press.
Appendix A
Table of TLU Characteristics for Writing a Personal Statement
Characteristics of the setting
physical characteristics
participants
time of task
Characteristics of the test rubric
instructions
Home or library (possibly a food/drink place

with internet)
Individuals, heterogeneous group of students
with various L1 backgrounds, male and
female, ages late-teens-mid-thirties or older.
Also, college admittance judges who read the
personal statements (intended audience) will
be largely unfamiliar with the individuals
writing the statements
Any time of day a student sits down to write
this essay

language
channel
specification of procedures and tasks
structure
time allotment
scoring method
criteria for correctness
procedures for scoring the response
explicitness of criteria and procedures
Characteristics of the input
format
channel
form
language
length
type
degree of speededness
vehicle
language of the input
language characteristics
organizational characteristics
grammatical
textual
pragmatic characteristics
functional
sociolinguistic
topical characteristics
Characteristics of the expected response
format
channel
form
language
length
type
language of the expected response
grammatical
textual
functional
sociolinguistic
22
Visual for reading the requirements of the

essay and then writing it
Language-based
The language to be assessed (English)
Short list of requirements and points to
address in the essay, however processing
time may be long if the learner is unsure of
how to approach this type of essay
Items and prompts for both limited and
extended production
Normal speed
Reproduced
All accurate and complex syntactical abilities,

statements, ability to write accurately about
past, present and future, advanced level and
academic register of vocabulary
Cohesive, well-organized and coherent essay
Ideational, manipulative, creative
Academic and professional genre and
register, Standard English dialect, formal
Personal, some cultural (if applicable), some
technical (if specialized)
Visual, written and typed essay

Language-based
Target language (English)
Typically 1-2 pages
Extended-production
Dependent upon learner (could be a day to a
month)
Accurate and complex syntactical abilities,

statements, ability to write accurately about
past, present and future, advanced level and
academic register of vocabulary
Cohesive, well-organized and coherent essay
Ideational, manipulative, creative
Academic and professional genre and
23
register, Standard English dialect, formal
Personal, some cultural (if applicable), some
technical (if specialized)
Relationship between input and response
reactivity
scope of relationship
directness of the relationship
Non-reciprocal
Narrow scope, personal essays use personal
life experiential information and opinions
input is already within the learners personal
history
Indirect, learner must refer to information
relevant to their personal experiences
usually there is no information in the input
except instructions
Appendix B
203 Grammar Table of Specifications
Content Areas
Present
prefect/present
perfect
progressive (vs.
simple present)
Objectives
Recognizing/comprehending grammar
forms
Understanding
forms and
meaning in
production and
comprehension
Producing
grammar in
writing
contexts
Knowledge
of vocabulary
2
2.1, 2.2
2
3.1, 3.2
G: 0-10
M: 0-10
TC: 0-10
0-10 (1)
5
Total #
of
items
% of
items
13
37%
Past perfect/past
perfect
progressive (vs.
simple past)
1
2.3
7
3.1, 3.2
Expressions of
future
1
2.4
2
3.1, 3.2
3
16%

Modals
1
2.5
24
7
4.1-4.7
G: 0-10
M: 0-10
TC: 0-10
0-10 (1)
5
Vocabulary
10
1.1-1.10
10
Total # of items
18
10
35
% of items
14%
50%
16%
20%
27%
20%
100%
Appendix C
Test Task Characteristics
Characteristics of the setting
physical characteristics
participants
time of task
Characteristics of the test rubric
Classroom 203 in Spruce Hall, INTO CSU.

Florescent lighting, windows, 2nd floor, large
room with 20 desks.
Heterogeneous group of students with various
L1 backgrounds, male and female, ages lateteens-mid-thirties or older.
1:00 p.m.-2:20 p.m./ 80 minutes
instructions
language
channel
specification of procedures and tasks
structure
time allotment
scoring method
criteria for correctness

Written/Visual, aural if students need
clarification
Brief to lengthy, some with examples, some
without, provided one at a time
6 parts with 40 items, separate tasks are
clearly marked, sequence is provided as
structured to unstructured production, but
students need not follow that sequence
70-75 minutes in class
.5 point for each correct answer in part 1.1-
procedures for scoring the response
explicitness of criteria and procedures
25
1.3 and 1 point for each correct answer in
parts 2-4, 0-10 points for part 5
The instructor will score the test in the office,
each section of the test will be scored for
each student at the same time (i.e. a whole
test per student is NOT graded at once)
Answer key and rubric provided for teachers,
point values provided for students
Characteristics of the input

format
channel
form
language
length
type
vehicle
language of the input
Visual for reading the instructions of the tasks

and then writing them or answering them as
directed
Language-based
Short chunks, which should already be
familiar to students therefore short processing
time
Items to elicit grammatical forms and a
prompt to elicit an extended production
response using multiple tenses and aspects
Normal speed
Reproduced
grammatical
textual
functional
sociolinguistic
~92% of words in AWL and 1-2K, upperintermediate vocabulary and some AWL words
Short, cohesive and coherent, clear
Ideational, manipulative
Academic and formal register and genre,
Standard English dialect
Academic information based on classroom
tasks/syllabi
Characteristics of the expected response

format
channel
form
language
length
type
language of the expected response
Visual for writing prompt and answering cloze

questions as directed
Language-based
English
For the essay, eight or more sentences,
otherwise students should use the space
provided
Selected, limited production, extended
response
60 minutes
grammatical
Verb tenses and aspects, future tenses and

aspects, modals, the ability to read and write
using various tenses and aspects, focus on
form, usage and meaning of verbs and
26
grammatical forms, upper-intermediate
vocabulary and some AWL words
Cohesion, coherency, well-organized writing
(for the essay portion)
textual
functional
sociolinguistic
Ideational, creative
Academic and formal register and genre,
Standard English dialect, naturalness, some
cultural reference if students feel comfortable
including that
Academic information based on classroom
tasks/syllabi
Relationship between input and response
reactivity
scope of relationship
directness of the relationship
Non-reciprocal
Narrow scope
Direct for parts 1-5 and indirect for part 6
where students write about their own
experiences
Appendix D
AEIN8203 Grammar Final Exam
CSU Honor Pledge:
Name:___________________________
_________________________________________________________________________
_____________________________________________________________Initials:______
Write this statement above & initial: I have not given, received or used any unauthorized assistance.
Part I: Vocabulary and verb forms. (5 points)

1. For each verb 1, provide the following forms: verb 2 (past) and verb 3 (past
participle). (3 points) An example is provided in the table below.
V1
EX: hold
keep
forget
teach
V2
held
V3
held
2. Underline the correct word to complete the sentence. (1 point) An example is

provided below.
Ex: Would you please LEND / BORROW me a pencil?
2a. Can you LEND / BORROW me your car this weekend?
2b. May I LEND / BORROW that book from you?
3. Use Would you mind to write a question for each situation. (1 point) An example
is provided below.
Ex: Situation You are cold in grammar class and the window beside you and your
friend is open.
Answer: Would you mind if I closed the window?
3a. Situation The door beside your friend is open and you want her to close it.
__________________________________________________________________
3b. Situation Your friend didnt eat her chocolate cake and you want to eat it.
___________________________________________________________________
Part II: Error Correction: Each sentence has only one (1) error. Correct the error in each
sentence. Please do not rewrite the entire sentence. (5 points) An example is provided
below.
heard
Ex: I have hear this song many times before.

1.
Im late for class today. I should stay up all night talking to my friend last night.
2.
Rahafs clothes are dirty. She have playing soccer, recently.
3. Tony cook dinner by the time I got home.

4. Tomorrow, I go see my best friend in Denver.
5. At CSU, you dont have to smoke cigarettes in class.
27
28
Part III: Verb tenses: Read the timeline about Vicki, who is currently working at INTO
CSU. Then, use the information on the timeline to fill in the blanks below with the best
form of the verb provided. Consider the overall context when filling in the blanks.
Consider: There may be more than one form that would work well in the context. (5
points) An example has been provided in the first blank below.
Studied
in
Californi
a 2008
Moved
to Fort
Collins
2013
Traveled
to China
2010
Graduat
e
college
2009
Wanted
buy a
house
Fall
2012
Still
teaching
at INTO
2014
Got job
at INTO
Summer
2014
1 Vicki is a current teacher at INTO CSU and she __ Example: loves____ (love) to travel.
She is from Colorado, but she ______________ (study) in California in 2008. In 2010
she went to China. Before she traveled to China, she______________ (graduate) from
college. Vicki ______________ (work) in a restaurant in China for two years when she
moved back to the United States. After she moved back, she ______________ (look)
for a house. She__________________ (search) for a house for one year when she
finally found one in Fort Collins.
2. Fill in the blanks below about Vicki after she moved to Fort Collins. Use the best form
of the verb provided. Consider the overall context when filling in the blanks. There may be
more than one form that would work well in the context. Consider: you may not need to
use the timeline for all the answers. (7 points)
After she ______________ (move) to Fort Collins, Vicki was offered a job at INTO
CSU teaching English. Vicki worked in a restaurant in China and she ______________
29
(teach) English before, so she was very nervous. But now she is confident and likes
teaching English.
Now, Vicki ______________ (live) in Fort Collins. She ______________ (teach)
English at INTO CSU since the summer of 2014. She isnt nervous anymore. Next year,
Vicki ______________ (go) to Japan to teach English in Kobe. But she
______________ (not, got) her visa, yet. She ______________ (go) to Denver next
week to fill out the visa forms. She is very excited!
Part IV: Modals: Read the table below. Choose the best modal (there may more than one)
to fill in the blanks below. (7 points) An example is provided below.
can
could
had better
may
might
must
ought to
should
will
Ex: Im very busy tomorrow, so I ___should not/had better not__go to the party tonight.
1. Sarah: I forgot my book. ______________ I use your book today?
Ciao: Sorry, I forgot mine, too. Lets ask Pedro. ______________ you share your book
with us?
Pedro: No, I forgot mine, also.
2. Zhaowang has a test tomorrow morning. He failed the quiz last week because he
stayed up all night playing video games. He ______________ go to sleep early tonight.
3. When you get on an airplane, you are _______________ put your seatbelt on.
4. Sarahs stomach hurts. She ______________ have eaten so much candy.
5. Ayaka: Maho is crying. She ______________ be upset.

Yiping: Yes. She broke up with her boyfriend.
6. Olga: I saw Kevin this morning at the Mosque.
Moriah: You did? It ______________ been Kevin. He moved to Japan yesterday.
Part V: Short Answer: Read the sentences below. Then, write a paragraph (at least 8
sentences) in response to the question below. Include details and examples in your response.
Pay close attention to grammar in your answers.
You will be graded on your:
1. Grammar
2. Mechanics
3. Task completion (10 points)
1. What is the best thing youve accomplished in your life until now? Why did you want to
do this? How did you accomplish this thing? What advice can you give to someone who
wants to do the same thing?
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
30

_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
31
32
Extra Credit: What will you do after you finish studying at INTO CSU? (1 point)
_________________________________________________________________________
_________________________________________________________________________
Appendix E
ANSWER KEY: AEIN8203 Grammar Final Exam
(Note: CSU Honor Pledge not used for this test.)
Part I: Vocabulary and verb forms. (5 points)
Total =
/39 points
1. For each verb 1, provide the following forms: verb 2 (past) and verb 3 (past
participle). (3 points) An example is provided in the table below.
V1
EX: hold
keep
forget
teach
V2
held
kept
forgot
taught
V3
held
kept
forgotten
taught
2. Underline the correct word to complete the sentence. (1 point) An example is

provided below.
Ex: Would you please LEND / BORROW me a pencil?
2a. Can you LEND / BORROW me your car this weekend?
2b. May I LEND / BORROW that book from you?
3. Use Would you mind to write a question for each situation. (1 point) An example
is provided below.
Ex: Situation You are cold in grammar class and the window beside you and your
friend is open.
Answer: Would you mind if I closed the window?
3a. Situation The door beside your friend is open and you want her to close it.
Would you mind closing the door?
_________________________________________________________________________
3b. Situation Your friend didnt eat her chocolate cake and you want to eat it.
Would you mind if I ate your chocolate cake?
_________________________________________________________________________
Part II: Error Correction: Each sentence has only one (1) error. Correct the error in each
sentence. Please do not rewrite the entire sentence. (5 points) An example is provided
below.
heard
Ex: I have hear this song many times before.

1.
Im late for class today. I should not have stayed up all night talking to my friend
last night.
2.
Rahafs clothes are dirty. She have has been playing soccer, recently.
3. Tony had cooked cook dinner by the time I got home.

4. Tomorrow, I am going to/will go see my best friend in Denver.
5. At CSU, you dont have to must not smoke cigarettes in class.
Part III: Verb tenses: Read the timeline about Vicki, who is currently working at INTO
CSU. Then, use the information on the timeline to fill in the blanks below with the best
form of the verb provided. Consider the overall context when filling in the blanks.
Consider: There may be more than one form that would work well in the context. (5
points) An example has been provided in the first blank below.
33
Studied
in
Californi
a 2008
34
Moved
to Fort
Collins
2013
Traveled
to China
2010
Graduat
e
college
2009
Wanted
buy a
house
Fall
2012
Still
teaching
at INTO
2014
Got job
at INTO
Summer
2014
1.Vicki is a current teacher at INTO CSU and she __ Example: loves____ (love) to
travel. She is from Colorado, but she ____studied__________ (study) in California in
2008.
In
2010
she
she_______graduated/had
went
to
China.
graduated_______
Before
she
(graduate)
traveled
from
to
college.
China,
Vicki
______worked/had worked________ (worked) in a restaurant in China for two years

when she moved back to the United States. After she moved back, she
______looked________ (look) for a house. She_______(had) searched/had been
searching_____ (search) for a house for one year when she finally found one in Fort
Collins.
2. Fill in the blanks below about Vicki after she moved to Fort Collins. Use the best form
of the verb provided. Consider the overall context when filling in the blanks. There may be
more than one form that would work well in the context. Consider: you may not need to
use the timeline for all the answers. (7 points)
After she ________moved/had moved______ (move) to Fort Collins, Vicki was
offered a job at INTO CSU teaching English. Vicki worked in a restaurant in China and
she ______had not taught________ (not, teach) English before, so she was very
nervous. But now she is confident and likes teaching English.
35
Now, Vicki ________lives______ (live) in Fort Collins. She ________has

taught______ (teach) English at INTO CSU since the summer of 2014. She isnt
nervous anymore. Next year, Vicki ______will go/is going to go________ (go) to Japan
to teach English in Kobe. But she _______has not gotten_______ (not, got) her visa,
yet. She _______is going to go/will go_______ (go) to Denver next week to fill out the
visa forms. She is very excited!
Part IV: Modals: Read the table below. Choose the best modal (there may more than one)
to fill in the blanks below. (7 points) An example is provided below.
can
could
had better
may
might
must
ought to
should
will
Ex: Im very busy tomorrow, so I ___should not/had better not__go to the party tonight.
1. Sarah: I forgot my book. _____Could/May/Can_________ I use your book today?
Ciao: Sorry, I forgot mine, too. Lets ask Pedro. ________Could/Can/Would/
Will______ you share your book with us?
Pedro: No, I forgot mine, also.
2. Zhaowang has a test tomorrow morning. He failed the quiz last week because he
stayed up all night playing video games. He ______ought to/should________ go to
sleep early tonight.
3. When you get on an airplane, you are ________supposed to/must/had better______
put your seatbelt on.
4. Sarahs stomach hurts. She _____shouldnt/should not_________ have eaten so much
candy.

5. Ayaka: Maho is crying. She ________must______ be upset.
Yi: Yes. She broke up with her boyfriend.
6. Olga: I saw Kevin this morning at the Mosque.
Moriah: You did? It _________couldnt have/could not have_____ been Kevin. He
moved to Japan yesterday.
Part V: Short answer: Read the sentences below. Then, write a paragraph (at least 8
sentences) in response to the question below. Include details and examples in your
response. Pay close attention to grammar in your answers.
You will be graded on your:
1. Grammar
2. Mechanics
3. Task completion (10 points)
1. What is the best thing youve accomplished in your life until now? Why did you want to
do this? How did you accomplish this thing? What advice can you give to someone who
wants to do the same thing?
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
36

_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
Extra Credit: What will you do after you finish studying at INTO CSU? (1 point)
37
38
_________________________________________________________________________
_________________________________________________________________________
Appendix F
Writing Rubric
SCORE:
Grammar
9-10
The student
chose a variety
of grammatical
forms that fit
the overall
context and
used all of the
chosen forms
correctly and
appropriately
for the context
and meaning
they created in
response to the
question.
7-8
The student used
some variety of
grammatical
forms correctly
with a few errors
(such as had
instead of has)
and mostly used
grammar
appropriate to
the context and
meaning they
created in
response to the
question.
5-6
The student
used
grammatical
forms some
correctly and
some
incorrectly. Not
much variety in
forms. Meaning
was impacted.
Context and
meaning
created in
response to the
question were
sufficient.
3-4
The student
did not use a
majority of the
grammatical
forms
correctly nor
appropriately
for the context
and meaning
they created in
response to the
question.
Meaning was
impacted.
Mechanics
There are one

or two to no
mistakes with
spelling and
vocabulary.
Overall, the
writing is clear,
well organized
and cohesive
with minimal
errors that do
There are some

mistakes with
spelling or
vocabulary, but
overall the
writing is mostly
clear, organized
and cohesive
with only a few
errors that
slightly impact
There were
mistakes with
spelling,
vocabulary and
punctuation that
impacted
meaning.
Writing was not
very clear, well
organized or
cohesive.
The student
made many
mechanical
errors and the
meaning is
greatly
impacted.
Writing is not
clear, well
organized or
cohesive.
1-2
The student
attempted
the question,
but the
grammar is
incorrect or
does not
include any
grammatical
structures
from this
class. The
meaning is
not clear and
/or not
obviously
related.
There are
many
mechanical
errors that
impede
meaning for
a majority or
all of the
response.
0
The
student
did not
attempt
the
question
.
The
student
did not
attempt
the
question
.
Task
Completion
not impact
meaning.
Punctuation is
mostly correct
with little or no
errors.
The student
answered all
parts of the
question with
an answer that
was
creative/origina
l. Student
provided rich
details and
examples.
Student met or
exceeded the
sentence
minimum.
39
meaning in some
parts.
The student
answered all
parts of the
question with an
answer that was
somewhat
creative/original.
Student provided
good details and
examples.
Student met the
sentence
minimum or was
lacking 1-2
sentences.
The student
answered parts
of the question,
but not all parts.
Student
provided some
details and
examples.
Student did not
meet sentence
minimum.
The student
did not answer
all parts of the
question.
Student
provided little
to no details
and examples.
Student did
not meet
sentence
minimum.
The student
did not
answer any
parts of the
question
and/or was
off-topic.
Student did
not meet
sentence
minimum.
The
student
did not
attempt
the
question
.
Appendix G
Score Reporting Sheet
Name: ___________________________________________________
Part 1:
5 points
score___________/10
percentage ______________
Part 2:
5 points
score___________/10
Part 3.1: 5points
score___________/10
3.2: 7 points
score___________/10
score___________/10
Part 4:
7 points
Part 5: 10 points
score___________/10
percentage _______________
SCORE:
Grammar
9-10
The student chose a
variety of
grammatical forms
that fit the overall
context and used all
of the chosen forms
correctly and
appropriately for the
context and meaning
they created in
response to the
question.
7-8
The student used
some variety of
grammatical forms
correctly with a few
errors (such as had
instead of has)
and mostly used
grammar
appropriate to the
context and
meaning they
created in response
to the question.
5-6
The student used
grammatical forms
some correctly and
some incorrectly.
Not much variety
in forms. Meaning
was impacted.
Context and
meaning created in
response to the
question were
sufficient.
3-4
The student did
not use a majority
of the
grammatical
forms correctly
nor appropriately
for the context
and meaning they
created in
response to the
question.
Meaning was
impacted.
Mechanics
There are one or two

to no mistakes with
There are some

mistakes with
There were
mistakes with
The student made

many mechanical
1-2
The student
attempted the
question, but
the grammar is
incorrect or
does not
include any
grammatical
structures from
this class. The
meaning is not
clear and /or
not obviously
related.
There are many
mechanical
0
The
student
did not
attempt
question.
The
student
Task
Completion
spelling and
vocabulary. Overall,
the writing is clear,
well organized and
cohesive with
minimal errors that
do not impact
meaning. Punctuation
is mostly correct with
little or no errors.
The student answered
all parts of the
question with an
answer that was
creative/original.
Student provided rich
details and examples.
Student met or
exceeded the
sentence minimum.
40
spelling or
vocabulary, but
overall the writing
is mostly clear,
organized and
cohesive with only a
few errors that
slightly impact
meaning in some
parts.
The student
answered all parts
of the question with
an answer that was
somewhat
creative/original.
Student provided
good details and
examples. Student
met the sentence
minimum or was
lacking 1-2
sentences.
spelling,
vocabulary and
punctuation that
impacted meaning.
Writing was not
very clear, well
organized or
cohesive.
errors and the

meaning is
greatly impacted.
Writing is not
clear, well
organized or
cohesive.
errors that
impede
meaning for a
majority or all
of the response.
did not
attempt
question.
The student
answered parts of
the question, but
not all parts.
Student provided
some details and
examples. Student
did not meet
sentence
minimum.
The student did

not answer all
parts of the
question. Student
provided little to
no details and
examples.
Student did not
meet sentence
minimum.
The student did

not answer any
parts of the
question and/or
was off-topic.
Student did not
meet sentence
minimum.
The
student
did not
attempt
question.
Appendix H
B-index Item Analysis
41
42
43

Final Test Project

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Final Test Project

Uploaded by

Copyright:

Available Formats

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

modals of polite request, necessity, advisability, probability/certainty (INTO CSU,

203 Grammar Final Test Project

considered when distinguishing a criterion-referenced and a norm-referenced test. They

203 Grammar Final Test Project

Cohesion and organization

Present perfect and present perfect progressive (vs.

Ability to understand and

Understanding connections between grammar forms and

203 Grammar Final Test Project

% of instruction time spent on each

203 Grammar Final Test Project

on each content area in the 203 Grammar class.

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

Recommended time spent for test-takers (minutes)

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

assessment are below in Table 1:

203 Grammar Final Test Project

203 Grammar Final Test Project

of the B-index values for each item.

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

203 Grammar Final Test Project

OSullivan, B. & Stoynoff, S. (Eds.), The Cambridge guide to second language

Home or library (possibly a food/drink place

203 Grammar Final Test Project

Visual for reading the requirements of the

All accurate and complex syntactical abilities,

Visual, written and typed essay

Accurate and complex syntactical abilities,

203 Grammar Final Test Project

directness of the relationship

203 Grammar Final Test Project

Classroom 203 in Spruce Hall, INTO CSU.

The language to be assessed (English)

203 Grammar Final Test Project

procedures for scoring the response

explicitness of criteria and procedures

Characteristics of the input

Visual for reading the instructions of the tasks

Characteristics of the expected response

Visual for writing prompt and answering cloze

Verb tenses and aspects, future tenses and

203 Grammar Final Test Project

Part I: Vocabulary and verb forms. (5 points)

203 Grammar Final Test Project

2. Underline the correct word to complete the sentence. (1 point) An example is

Ex: I have hear this song many times before.

Rahafs clothes are dirty. She have playing soccer, recently.

3. Tony cook dinner by the time I got home.

203 Grammar Final Test Project

203 Grammar Final Test Project

worked/had worked__ (worked) in a restaurant in China for two years

Now, Vicki lives (live) in Fort Collins. She __has