Reading Study Report 2014 Final1

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/280572668
Progress Report on the Reading Skills of Grade 2 students in Kosovo Primary

Schools 2014
Technical Report · September 2014
CITATIONS READS
0 1,687
1 author:
Besnik A Krasniqi
University of Prishtina
154 PUBLICATIONS 1,329 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Formal and informal Institutional activities View project
WBC-INCO View project
All content following this page was uploaded by Besnik A Krasniqi on 31 July 2015.
The user has requested enhancement of the downloaded file.

Progress Report on the Reading Skills of Grade 2 students in
Kosovo Primary Schools
September 2014
By Besnik Krazniqi and Keith Prenton

CONTENTS
Section Title Page
1 Executive Summary 3
2 Background 5
3 Design Of Research Study 7
4 Design Of EGRA Instrument 8
5 Data Gathering 12
6 Data Preparation 13
7 Analysis & Results 15
8 Performance By Gender 19
9 Conclusions And Recommendations 20
Appendix A Basic Education Program Strategy for Improving 21

Reading Skills
Appendix B Detailed Analysis & Tables 35
Besnik Krasniqi.and Keith Prenton, September 2014

USAID Basic Education Program: Progress Report on the Reading Skills of Grade 2 students in Kosovo Primary Schools,
1. EXECUTIVE SUMMARY
In 2012, USAID’s Basic Education Program developed an Albanian-language early grades
reading assessment (A-EGRA). The A-EGRA was intended:
 to enable teachers to diagnose the strengths and weaknesses of individual students in
order to inform their teaching;
 to enable Program staff to identify strengths and weaknesses in Kosovo students’
reading skills in order to plan appropriate interventions;
 to enable the Program to measure improved reading skills as part of its Performance
Monitoring Plan.
This report is based on two research studies using the A-EGRA.

1. A baseline study: undertaken in June 2012, in order to provide baseline data on the
reading skills of students at the end of Grade 2
2. A second study: undertaken in May 2014, to provide progress data and compare with
2012 data.
It is also planned to undertake a third study in May 2015, to provide data on progress and
compare with the two previous studies.
The baseline study tested a sample of 800 grade 2 students from 40 schools in different parts of
Kosovo. It produced a rich set of data that was used by the Program to develop a literacy
strategy aimed at a change in teaching practices and in turn an improvement in student’s
reading skills.
The baseline study identified reading comprehension as a key area for improvement. This
finding was supported by other evidence. Observation in classrooms coupled with input by
teachers and others during the development of the A-EGRA forms revealed the tendency to ask
questions based on explicitly-stated information whereas in order to develop higher level and
critical thinking skills, students need to use inference and or prior knowledge to inform
interpretation among other comprehension skills.
In 2013 the Program began a training program for teachers and developed resource materials
including a set of graded reading books designed to encourage reading for meaning and
comprehension skills. However, because of the time required for development, only reading
materials aimed at Grade 1 students were available in the academic year 2013-14.
Consequently, significant improvement in the reading skills of grade 2 students was not
expected until 2015, when they would have had 2 years of improved teaching and access to
appropriate reading materials.
Nevertheless, with regard to Reading Comprehension skills, the 2014 EGRA study shows a
decrease of 13% of the test-takers at ‘risk category’ in 2014 compared to 2012. It is likely that
the introduction of measures to improve reading comprehension skills, within the training
provided to teachers, influenced the improvement among children.
However the study shows that there are still some areas of concern. Oral fluency still requires
attention as results reveal that 50.8% of the test-takers are at risk in 2012. This situation has not
improved in 2014 as 59.3% of test–takers are at risk. The appropriate use of the new graded
reading books for Grade 2 would be a useful resource in improving oral fluency. This can be
Besnik Krasniqi.and Keith Prenton, September 2014 Page 3

supported by online versions of the books in which the text read by an actor, providing a good
model for students.
Listening comprehension and dictation results show some need for concern as there is an
increase by 7% of students in the ‘at risk’ category. However, listening comprehension is an
indicator of language development which still appears to be well established in these grade two
students despite the increase of the above-mentioned category.
Likewise, dictation scores were high for both 2012 and 2014 (small increase in 2014) and
probably can be explained by the amount of time teachers in Kosovo spend on this area,
possibly at the expense of opportunities for all students to practice oral reading. This is an area
for further exploration and attention.

2. BACKGROUND
2.1 Program Interventions in reading
The motto of USAID’s Basic Education Program is: “Developing students’ 21st Century skills
with schools and communities”. One of the key skills that the Program wishes to develop is
functional literacy.
The improvement of Reading Skills was not, originally, a specifically stated aim of the Program.
However in 2011 USAID published its Education Strategy 2011-151 which adopted as its first
goal, “Improved reading skills for 100 million children in primary grades by 2015”. In its 2011-
2012 yearly work plan, the Basic Education Program aligned its work with this goal and revised
its Program Monitoring Plan (PMP) accordingly. This alignment involved the development of a
literacy strategy for the Program (see Annex 1), which included:
 the development and implementation of a reading course for teachers aimed at the early
grades;
 the development and provision of a set of teaching and learning aids to Grade ! & 2
classrooms;
 the development of a set of graded reading books for emerging readers; and
 the development of an Early Grades Reading Assessment (EGRA) in the Albanian
language.
2.2 Monitoring and evaluating the interventions
In order to measure the impact of these interventions, relevant indicators were added to the
Program’s Performance Monitoring Plan (PMP). Program staff developed the following indicator
with which to measure progress:
Number and proportion of primary school students, who after 2 years of schooling, show
improved reading skills.
USAID also prescribed the following indicator, upon which the Program is required to report:
The proportion of students who, by the end of two grades of primary schooling, demonstrate
that they can read and understand the meaning of grade level text”.
The Program defined “improved reading skills” as the equivalent of “can read and understand
the meaning of grade level text”.
In order to provide data for these indicators, it was agreed to undertake a baseline study and
further annual research studies after Program interventions had taken place.
This report is based on two research studies:

 The first study was done during June 2012, to provide baseline data for these two
indicators.
1 Downloadable from: http://transition.usaid.gov/our_work/education_and_universities/documents/

USAID_ED_Strategy_feb2011.pdf

 During May 2014, the BEP undertook a second study to provide progress data for these
two indicators and compare with 2012 data.
It is also planned to undertake a third study in May 2015, to provide data on progress and
compare with the two previous studies.
2.3 Rationale for an Albanian language early grade reading assessment.
At the beginning of the Program, there was very little hard data about standards of achievement
in Albanian-language reading in Kosovo. There was no national reading policy or strategy; and
no high-quality reading tests were available for the early grades. Available tests of
comprehension assessed a very limited range of skills. Primarily, they tested retrieval of
explicitly-stated information from individual sentences. Consequently, it was necessary to
develop a suitable test in order to collect data that appropriately measured grade two reading
skills. Over the course of several months, the Assessment Component (led by an International
Assessment Expert and involving the MEST and educators from Kosovo and Albania) adapted
and tested the USAID-supported Early Grades Reading Assessment (EGRA) to the Albanian
language. The three different tests developed can be used for differing purposes including the
research studies. One test has been made available to teachers for use as a diagnostic tool in
schools and the Program is training specialist teachers in its use.
In addition to the use of the A-EGRA in evaluating progress the studies were designed to have a
positive impact on the Program in two ways.
 A-EGRA can provide an exceptionally rich data set of information about overall national
standards, diversity of performance between schools, and areas of strength and
weakness. The fact that the test used is similar in form and structure to tests used in
many other countries also allows achievement to be compared internationally, at least in
general terms. Therefore A-EGRA could help to create awareness within the MEST of
the need for action, and mobilize support for program interventions in the area of
reading.
 The A-EGRA studies could provide information to guide and inform Program
interventions in reading. The underlying multi-level model of reading literacy is well
accepted throughout the OECD. It therefore has automatic credibility, and the Program
will be able to link to it to increase practitioners’ awareness of the need to teach and
assess a range of comprehension skills, and also as a framework for developing
teaching and testing materials.

3. DESIGN OF RESEARCH STUDY

3.1 Sample Size: 800 grade 2 learners in 2012. 796 grade 2 learners now in 2014.
3.2 Instrumentation: Albanian EGRA forms were adapted and provided by the Program
They were made available using “Tangerine” software using a tablet PC. The same A-
EGRA research instrument was used in both 2012 and 2014.
3.3 Population: The population sampled is defined as all grade 2 learners in (40) schools
that participated in the baseline study in year 2012.
3.4 Sampling Frame: A list of 40 schools which participated in the baseline study in 2012
and in BEP interventions relating to teaching reading in the academic year beginning in
September 2013, and the number of grade 2 learners in each. From 40 schools which
participated in baseline study in year 2012, 31 schools continued participation in BEP
activity relating to teaching reading. As one school already finished their school year due
to school construction (“Migjeni” Mitrovica) another ten schools from the list of sixty eight
schools, currently involved in reading related activities will be randomly selected to
achieve a total of 40.
3.5 Sample selection: In the 2012 baseline study the schools were selected, and then
learners within schools. The forty schools were selected from the frame with probability
proportional to size (PPS). The ten extra schools selected in year 2014 were also
selected from the frame with probability proportional to size (PPS). In each of the forty
schools, twenty grade 2 learners were sampled randomly from the register received
beforehand from the school directors.

4. DESIGN OF EGRA INSTRUMENT

4.1 Background
The Early Grades Reading Assessment (EGRA) is a simple individual test of reading ability,
designed to be administered in grades 1-3. EGRA was designed originally to help USAID
partner countries to measure how well children in the early grades of primary school are
acquiring reading skills. Development of the instrument was supported by USAID, and
undertaken by RTI International, beginning in 2006.
EGRA was based on a review of research, and existing instruments such as the DIBELS
(Dynamic Indicators of Basic Early Literacy Skills), which is a set of procedures and measures
for assessing the acquisition of early literacy skills from kindergarten through sixth grade.
USAID, the World Bank and RTI convened a workshop of cognitive scientists, early-grade
reading instruction experts, research methodologists, and assessment experts to review the
proposed key components of the instrument. Following this workshop, a draft instrument was
developed, which was piloted in several countries and languages during 2007. Since that time,
EGRA has been used in many countries, mainly across Africa, Latin America and South Asia.
As a result, there is a rapidly developing evidence base, documented by the EGRA tracker2, and
also a growing range of support materials and tools, such as the EGRA Toolkit3, as well as a
recent e-book, The Early Grade Reading Assessment: Applications and Interventions to
Improve Basic Literacy4, to support development and application of EGRA worldwide.
4.2 Albanian Adaptation of EGRA
As noted above, the USAID Education Strategy 2011-155 has as its first goal “Improved reading
skills for 100 million children in primary grades by 2015”. The rationale for this is that universal
primary education is of limited value if acceptable learning outcomes are not achieved – and
that “children who do not attain reading skills at the primary level are on a lifetime trajectory of
limited educational progress and therefore limited economic and developmental opportunity” (p.
9). With this in mind, the Strategy document states that “No matter how primary grades are
defined, USAID will measure the performance of our programs primarily though the
improvement of reading skills for primary grade students after two years of schooling, consistent
with international measures adopted by the Education for All Fast Track Initiative.”
Reading performance in Kosovo is not believed to be anywhere near as low as in many low-
income countries. Since written Albanian is highly regular phonetically, there is anecdotal
evidence that children tend to learn decoding skills fairly quickly and easily; but their
comprehension skills may be relatively weak. However, hard evidence available about
standards of reading in the early grades was available only from EGRA 2012 study. The
Assessment Division of the Ministry of Education, Science and Technology (MEST) carries out
regular sample-based assessments at grade 5 level; but there were no tools available to
teachers or administrators for assessing reading skills below that level; nor were there any hard
data about standards across the nation.
2 Downloadable from: https://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&id=188

3 Downloadable from: https://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&id=149
4 Downloadable from: http://www.rti.org/pubs/bk-0007-1109-wetterberg.pdf
5 Downloadable from: http://transition.usaid.gov/our_work/education_and_universities/documents/
USAID_ED_Strategy_feb2011.pdf

In response to the new strategic context established by USAID, the Basic Education Program
(BEP) is supporting teaching of reading in Kosovo schools; and given the lack of available data,
an essential element in this support was agreed to be development of early grades reading
assessment. The Program team therefore embarked on adapting the USAID EGRA instrument
into Albanian.
The initial development process followed closely the procedure laid down in the EGRA Toolkit
(Chapters IV and V). In February 2012, a three-day workshop was held, led by a BEP
consultant, and attended by two professors of Albanian language, three teachers, and four field-
worker candidates. During this workshop, the sub-tests of the EGRA were reviewed for
relevance to Albanian. It was agreed that the first draft Albanian EGRA would follow closely the
structure proposed in the EGRA Toolkit, but with the “Letter Name” and “Letter Sound” subtests
being combined (because in Albanian, letter names and sounds are identical). Three parallel
draft forms of the EGRA were then produced in Albanian language; and the participants
(teachers and field workers) practiced administering the test to one another
Following the initial development workshop, the draft test forms were reviewed. One of the
problems faced by developers is the lack of suitable word lists and other support materials in
Albanian, so it was difficult to judge objectively the linguistic level of the materials. The Program
team will seek to remedy this lack as part of the wider development work in reading. However, it
did seem that the comprehension materials produced needed editing. The texts tended to be
very explicit in content and quite complex linguistically – especially grammatically – while the
questions mainly focused on retrieval of explicitly-stated information. The texts were made less
complex linguistically, with shorter sentences and simpler vocabulary; but also adapted to make
them less explicit, so as to leave some room for inference; and the questions were accordingly
adapted so that they included more scope for retrieval of information across two sentences,
forming an interpretation and responding to the text using knowledge of the world.
Following the initial development process, the three forms were piloted in three schools, one in
Pristina, one in Lipjan, and one in Ferizaj. In each school, a minimum of twenty students were
selected randomly. Most selected students were second-grade; but in Lipjan (where
achievement was generally very high) a few first-graders were included; while in Ferizaj (where
achievement was relatively low) a few third-graders were also included. Two forms were used
in each school, and each form was used with at least twenty students.
As a result of the pilot, it was clear that the instrument was discriminating between high- and
low-achieving readers. In one school, very few of the second-graders were much challenged by
it; but in the other two schools, the range of performance was much wider. The viability of the
instrument itself was therefore confirmed. Limited revisions were however made. One sub-test
(initial letter/sound identification) was eliminated as non-discriminating, and the order of the
“Phonemic awareness” and “Letter name/sound knowledge” sub-tests was reversed, reflecting
their relative difficulty. Specific comprehension questions were modified or changed where they
were non-functional (e.g., questions such as “What is a bee?” were replaced). In the highest-
achieving school, test takers could answer most or all of the comprehension questions, so it was
concluded that even though readers in the lower achieving schools often struggled to answer
them, they were a legitimate target to set. The overall hierarchy of comprehension questions
was therefore maintained.
4.3 Description of Instrument

The A-EGRA is an individual test – the supervisor tests children one at a time. It has seven sub-
tests, as follows:
i) Phonemic awareness
This sub-test consists of 10 items (words). The test taker has to segment each word into
phonemes. This sub-test is untimed; the test taker’s score will be the number of items answered
correctly (i.e., the number attempted, minus the number of errors).
ii) Letter name/sound knowledge
This sub-test consists of one hundred randomly selected letters. The test taker has to read
these letters aloud. This sub-test is timed, with a time limit of 60 seconds. The test taker’s
score will be the number of letters read correctly per minute (clpm)).
iii) Familiar word reading
This sub-test consists of fifty common words. The test taker has to read these words aloud.
This sub-test is timed, with a time limit of 60 seconds. The test taker’s score will be the number
of words read correctly per minute (cwpm).
iv) Unfamiliar word reading
This sub-test consists of fifty non-words. The test taker has to read these non-words aloud.
This sub-test is timed, with a time limit of 60 seconds. The test taker’s score will be the number
of words read correctly per minute (cwpm).
v) Oral reading fluency (5a) and Reading comprehension (5b)
The test taker first has to read a text of 50-60 words aloud. This sub-test is timed, with a time
limit of 60 seconds. The number of seconds taken to complete the task is recorded, or if the
task is not completed within the minute, the number of words actually read. The number of
errors made is also recorded. The test taker’s score will be the number of words read correctly
per minute (cwpm).
The test taker then has to answer comprehension questions about the text previously read. The
questions cover a range of levels, from retrieval of explicitly-stated information through forming
an interpretation up to making a judgment about the text using knowledge of the world. The
total number of questions is five; but a test taker who did not finish reading the test was not
asked questions about the part they had not read. The test taker’s score will be the number of
questions answered correctly, regardless of the actual number of questions asked.
vi) Listening comprehension
The test taker listens to the supervisor reading a short text aloud; then answers five questions
about it. The questions cover a range of levels similar to the reading comprehension questions.
However, they are generally less difficult than those on the reading text, and while they include
questions requiring extraction of distributed information and forming a simple interpretation, they

do not include questions relating to making a judgment using knowledge of the world. The test
taker’s score will be the number of questions answered correctly.
vii) Dictation
The supervisor reads a short sentence, which the test taker writes down. The test taker’s
writing is scored for correctness of five key words, plus word spacing, word order, punctuation
and initial capitalization. All items are scored “correct” (2 marks), “partially correct” (1 mark –
except capitalization and punctuation) and “incorrect” (0 marks)).
Testing one child takes about 10-15 minutes.
The physical test package consists of three parts:
 the administrator’s manual, which gives the instructions for administration and
scoring;
 a set of laminated sheets for the test taker to read from; and
 an individual scoring sheet for each test-taker, to be completed by the person
administering the test, available in Tangerine using a tablet.

5. DATA GATHERING
5.1 Enumerators
The Program in cooperation with the Pristina Faculty of Education selected eight graduate
students from the FE as enumerators responsible for data gathering. The eight enumerators
were trained at the Professional Development Center in Dardania School, Pristina, by two
Program facilitators. The facilitators were certified in EGRA and participated in developing the A-
EGRA. The two-day training on May 7 and 8 included practice in applying the tests to grade 2
students in selected schools. This round of practice was recorded and critiqued the following
day for further capacity building of enumerators.
5.2 Data Gathering
Enumerators worked in teams of two. Each individual enumerator tested ten students in a given
school during a single shift. Therefore, each team of two enumerators tested twenty students in
a school in a single visit. Four teams of enumerator were needed. Each team visited ten
schools, one per day for two weeks.
5.3 Quality Assurance – by BEP staff
Quality assurance was provided by the Program. It involved observing each team in the field
while working with students. Quality assurance personnel visited each team at least once, and
monitored them against the test administration procedures using a standard monitoring tool
developed BEP. Four of the Program’s staff monitored the field-work to ensure proper
administration.
5.4 Outcome of Data Gathering
A total of 801 Grade two learners from forty schools were tested in 2012 and 796 Grade two
learners in 2014. Enumerators completed an individual performance recording sheet for each
learner tested directly in the tablet PC using “Tangerine” software. This sheet included all the
required information about individual performance.
Data analysis and report writing was done by an independent consultant, Besnik Krasniqi, who
co-authored this report.

6. DATA PREPARATION
6.1 Data Entry and Cleaning
Each team of enumerators uploaded their data gathered with Tangerine to the cloud before the
end of the day. There was no need for data entry because Tangerine (properly used) has clean
CSV data.
Initial data transference and quality checks were completed by the BEP team. These checks
showed that the overall quality of the data transference was good.
6.2 Sampling Weights
The sample was chosen by a two-stage process. First, schools were selected with probability
proportional to size; then a fixed number of students (20) were selected randomly within each
school. Selecting schools with probability proportional to size improves the representativeness
of the sample, because it means that larger primary units (i.e., schools) have a higher chance of
selection than smaller units.
Selecting a fixed number of students within a school is less desirable, because students
selected from within a single school are generally more similar to one another than students
selected randomly from different schools; but it is necessary for reasons of administrative
convenience.
However, both of these factors affect the probability of selection. Schools were not selected
with equal probability; and although within a school students were in fact selected with equal
probability, the probability of a student in a large school being selected was lower than the
probability of a student in a small school being selected. To compensate for these unequal
probabilities of selection, sampling weights were applied, which are in each case the reciprocal
of the probability of selection.
Sampling weights were computed separately for the school and the student within the school.
Since we sampled with PPS, the school weight for the ith sampled school can thus be defined
as:
Where n is the number of sampled schools, mi is the measure of size for the ith school and
Where N is the total number of schools in the stratum.
Assuming that the sampling procedure within the school was followed, the student weight within
a school is constant. For the ith sampled school the student weight can be defined as:

Where mi is the measure of size for the ith school and ti is the number of students tested in the
ith school.
The final sampling weight is the product of these two weights. However, in order not to confuse
general readers, a linear transformation of the total sampling weights was used, such that the
sum of the weights is equal to the sample size. This means that the numbers in the tables
reflect the total size of the sample, rather than the population. The sampling weights were
worked out in MS Excel; then merged with the student data file in SPSS.
6.3 Computation of Intermediate Variables
The data set as originally entered consisted of the basic information on the student recording
sheet. For example, in the “letter name/sound knowledge” subtest, the raw data consisted of:
the number of letters read; the number of errors made: and the time taken. For purposes of
analysis, these variables had to be transformed into the final measure. For example, in the
“letter name/sound knowledge” subtest, the final measure is “correct letters per minute”. The
following were used
sub-test 1: “Correct words” was computed as: (words attempted – total-errors).

sub-test 2: “Correct letters per minute” was computed as: ((total-errors)/seconds)*60. Since the
resulting figure could be a decimal fraction, a grouped frequency was also computed (in bands
of 5).
sub-test 3: Correct words per minute were computed as: ((total-errors)/seconds)*60. Since the
resulting figure could be a decimal fraction, a grouped frequency was also computed (in bands
of 5).
sub-test 4: Correct non-words per minute were computed as: ((total-errors)/seconds)*60. Since
the resulting figure could be a decimal fraction, a grouped frequency was also computed (in
bands of 5).
sub-test 5a: Correct words per minute were computed as: ((words-errors)/seconds)*60. Since
the resulting figure could be a decimal fraction, a grouped frequency was also computed (in
bands of 5).
sub-test 5b: For each test taker, the following variables were derived:
 number of questions asked; and
 number of questions answered correctly (i.e. only the sum of the “E saktë”
column; score 1 point for each correct answer, zero for anything else)
sub-test 6: the number of questions answered correctly was derived (i.e. only the sum of the “E
saktë” column; score 1 point for each correct answer, zero for anything else)
sub-test 7: the total score was computed (i.e., sum of the “pikët” column)

7 ANALYSIS & RESULTS

7.1 Defining Adequacy of Performance
The A-EGRA is designed to be criterion-referenced (i.e., assessing performance against defined

goals or outcomes) rather than norm-referenced (i.e., ranking performance based on total
score). One of the major issues to be addressed is therefore how to define adequacy of
performance.
Standards of comparison were sought from outside Kosovo; but relatively few were found.
Percentile oral fluency norms6 were found for the US. These are based on a large number of
cases, but do not attempt to define adequacy. Norms for the DIBELS test7 do attempt to define
adequacy, either in terms of level of “risk” or “deficit”. These however are only available for
letter naming, non-word reading, and oral fluency. Where relevant, these norms have been
given in the tables below. The basic principle enshrined in the DIBELS approach has also been
applied to reading and listening comprehension; but this is not as yet empirically based.
According to Seymour et al.8, learners acquire reading in alphabetic languages in three phases.
In the first phase, the learner acquires knowledge of the smallest building blocks of reading, by
recognizing letter sounds (grapheme-phoneme correspondence) and their combination into
simple words. This involves basic skills such as fluent decoding of letters into sounds, creation
of a bank of words they can recognize on sight, and the ability to decipher new or unfamiliar
words through an understanding of letter-sound correspondence.
In the second and third phases, learners acquire a full understanding of the spelling system and
fluency with rimes (larger units of letter-sound correspondence). They then progress to fluency
with morphemes (association of phonemes with semantic meaning), and build a framework that
involves not only decoding but also attributing meaning to written text.
For purposes of analysis, the subtests were divided into three main sections which loosely
reflect these three levels:
 Basic skills (phonemic awareness and letter name/sound knowledge)

 Sight vocabulary and decoding (familiar word reading and non-word reading)
 Oral fluency and comprehension (listening/reading)
7.2 Basic Skills
First two sub-tests – phonemic awareness and letter name/sound knowledge – are essentially
aimed at first graders; but they were included in case a significant proportion of learners were,
at the end of grade 2, still reading at a grade 1 level.
a) Phonemic Awareness
6 Excerpted from: Hasbrouck, Jan and Gerald A. Tindal. “Oral Reading Fluency Norms: A valuable
assessment tool for reading teachers.” The Reading Teacher, Vol. 59 No. 7, April 2006, pp. 636-644
7 Downloaded from: https://dibels.uoregon.edu/docs/benchmarkgoals.pdf
8 Seymour, P. H. K., Aro, M., & Erskine, J. M. (2003). Foundation literacy acquisition in European
orthographies. British Journal of Psychology, 94, 143–174.

The phonemic awareness test, which involved segmenting words into phonemes, proved
generally rather easy. In this case, the individual phonemes were not scored; test takers had
to segment the whole word correctly to get a mark.
The maximum score for the 25 words was therefore 25. The mean score on this sub-test
was 21.06 out of 25. Half of the test takers scored 23 or more out of 25. Three quarters of
test takers scored 20. Of the remainder, most scored 14 or more out of 25.
Concerning the category of test takers classified as the “discontinued” category (which
represents those who failed to answer any items), the data show an increase in performance
as this category has decreased from 4.3% to 1.9 %. This improved performance of second
grade children is despite the fact that this test over-states the extent of real failure, because
“discontinued” includes a number of children who did well enough on the remainder of the
test to suggest that they in fact failed not because they could not do the task, but because
they were not ready, or did not understand what to do.
b) Letter Name/Sound Knowledge
Letter name/sound knowledge is a fundamental and very basic measure of reading

readiness. The mean number of letters read correctly in 2012 was 78.03, and the standard
deviation 19.29 while in 2014 was 97.5 and the standard deviation 16.23. This finding shows
that there is a significant improvement in 2014 compared to 2012. There is also DIBELS
norms for letter name; but the DIBELS Letter Naming Fluency (LNF) norms are only for
grade 1 level. Where LNF is below 25 letters correct per minute (LCPM), the test taker is
deemed to be “at risk”. Where LNF is 25 LCPM or more, but less than 37 LCPM, the test
taker is deemed to be at “some risk”. Where LNF is 37 LCPM or more, the test taker is
deemed to be at “low risk”.
By these standards, none of the test takers in this sample were defined as being “at risk”.
7.3 Sight Vocabulary and Decoding
a) Familiar Word Reading (CWPM)
This is a measure of sight vocabulary – recognition of high-frequency words selected from

early-grade reading materials, independent of context. Using a list of unrelated words
allows for a purer measure of word recognition and decoding skills than does reading
comprehension paragraphs, as test takers are unable to guess the next word from the
context. This is in fact reflected in the higher scores test takers achieved on the oral fluency
test. There is an increase in mean of correct words reading per minute in 2014 compared to
2012 indicating an improvement in among the second grade children.
b) Non-word Reading (CNPM)
Successful readers must combine both decoding and sight recognition skills. Reading
unfamiliar words involves decoding, which tends to be slower than sight recognition of
vocabulary. Even in the early grades, many children have a substantial sight vocabulary. If
they do not also acquire decoding skills, they will not be able to add new vocabulary very
efficiently. Non-word reading avoids the problem of sight recognition of words, and offers a
good measure of decoding skill. The survey shows a significant improvement on overall
score of CNPM in 2014 compared to 2012.

Virtually no children lack decoding skills in 2014. In particular, there is a substantial

improvement in terms of a decrease of children in the ‘emerging’ category with fewer in the
.deficit category
7.4 Oral Fluency and Comprehension
a. Oral Fluency (CWPM)
The EGRA results presented in table below show that overall oral fluency among second grade
children has shown a small increase in 2014 (73.39) compared to 2012 (72.23). However, a
rather high proportion of test takers fell into the “at risk” category. In 2014 the analysis shows a
higher increase of children “at risk” and a decrease of “low risk” category. Therefore, levels of
oral fluency amongst test takers are low enough to be a cause for concern.
b. Reading Comprehension
The reading comprehension test was designed to test a range of comprehension skills, starting
from retrieval of explicitly-stated information, but covering also inference and using prior
knowledge to inform interpretation. The intention was that an established reader at this grade
level should get all the items right, or make at most one mistake. On this basis, a three-level
classification of scores was developed:
 An established reader is one who gets 4 or 5 marks (i.e., can answer all levels of
grade-appropriate questions, including interpretation and using knowledge of the world);
 An emerging reader may get 2 or 3 marks (can answer at least some questions requiring
retrieval of distributed information, or formation of an interpretation);
 A reader who gets only 1 mark or none at all is defined as a deficit reader (can only
retrieve explicitly-stated information, if at all).
In 2012, a rather large number of test takers – about 40% - fell into the “deficit” category while in
2014 this figure was much lower at 27%. This finding suggests that there is a decrease of
‘deficit’ category and increase of ‘emerging’ category.
c. Listening Comprehension
Listening comprehension is not generally taught. It is therefore important primarily as an

indicator of language development. The same categories were applied to the listening
comprehension test as to the reading comprehension test. In 2014 the percentage of children in
the ‘established category’ has decreased. At the same time there is an increase in the
‘emerging’ category.
7.5 Dictation
Dictation is basically a test of learners’ ability to write, spell and use grammar properly.
Students’ ability to hear sounds correctly and then write the letters and words corresponding to
the sounds they hear demonstrates their success with the alphabetic principle. Their ability to
write the words with correct inflections also demonstrates both their comprehension of the
sentence and their mastery of grammar.

The sentence that was read contained five major lexical items (four nouns and one verb); each
of these was scored, along with word spacing, word order, initial capital and final full stop. Each
item was scored two points for fully correct, one for partially correct (with criteria defining what
constitutes “partially correct” or zero for incorrect. The dictation test was therefore scored out of
18. Comparative analysis of 2014 and 2012 suggests that there are no differences found
relating to dictation between the two periods. The achieved mean score for both 2014 and 2012
strongly indicates that few test takers have difficulty with the dictation test. Even the bottom
decile of test takers show students achieving something.

8. PERFORMANCE BY GENDER
In this section we report on EGRA performance broken-down by gender. This gender analysis
will enable to draw more specific conclusions on how male and female children perform in
different EGRA sub-sections. Tables showing detailed analysis are shown in Appendix 2.
The share of male in the total sample was higher compared to female possibly reflecting their
respective shares in total population of second grade children.
We did not find statistically significant gender differences regarding phonemic awareness.
We did not find gender differences regarding Letter Name/Sound Knowledge.
With regard to familiar word recognition, we did find statistically significant gender differences.
On average female performed better than male.
We did not found gender differences regarding non –word reading.
Concerning oral fluency we found significant statistical differences on EGRA performance

between male and female. Female children have comparatively higher mean score per minute
(75.4) compared to male (71.6) suggesting the need to investigate the lower performance of
males. In particular males perform very poorly within the at-risk group and the gender
differences are driven by this group. Therefore future measure should be targeted to reduce the
number of males at risk.
Our analysis suggests that there are no statistically significant differences regarding reading
comprehension or listening comprehension by gender.
Finally we did not find statistically significant differences between male and female regarding
dictation. Both groups display similar performance.

9. CONCLUSIONS AND RECOMMENDATIONS

The aim of this study to provide progress data for BEP expected outcome indicator, “Number
and proportion of primary school students, who after 2 years of schooling, show improved
reading skills”. However in fact, this study represents by far the most comprehensive
comparative assessment of the second grade children in Kosovo during two span periods, 2014
and 2012. It produced a rich set of data that can be used to further improved reading skills
including focusing on a change in teaching practice.
In 2012, the baseline study identified reading comprehension as a key area for improvement.
Probably the measures taken to improve reading comprehension skills influenced the
improvement among children. Therefore, 2014 EGRA study shows a decrease of 13% of the
test-takers at ‘risk category’ in 2014 compared to 2012.
Oral fluency also requires attention as results reveal that 50.8% of the test-takers are at risk in
2012. This situation has not improved in 2014 as 59.3% of test–takers are at risk. A future focus
of the EGRA program should be improvement of oral fluency.
In contrary to reading comprehension, the study identified listening comprehension as a key

area for improvement, in particular, supporting students to develop listening skills. Listening
comprehension and dictation results show a need for concern as there is an increase by 7% of
students in the ‘at risk’ category. However, as indicated, listening comprehension is an indicator
of language development which still appears to be well established in these grade two students
despite the increase of the above-mentioned category.
Likewise, dictation scores were high for both 2012 and 2014 (small increase in 2014) and
probably can be explained by the amount of time teachers in Kosovo spend on this area,
possibly at the expense of opportunities for all students to practice oral reading. This is an area
for further exploration and attention.
With regard to Gender differences, in only one area did the study find statistically significant
differences. Concerning oral fluency we found significant statistical differences between male
and female. It is recommended that the Program investigates the lower performance of males
and identifies measures to reduce the number of males at risk.
The provision of supplementary grade reading books for Grade 2 in the academic year 2014-15
is an opportunity to address the need for further improvement, particularly in oral fluency. The
availability of an online version, read by actors, means that support for improving reading skills
is also possible at home as well as in school. It is recommended that the program finds ways to
publicize these resources among the wider public and provide additional guidance to teachers
where possible.

APPENDIX A: BASIC EDUCATION PROGRAM STRATEGY FOR

IMPROVING READING SKILLS
1) WHY IMPROVE READING?
“USAID believes the most strategic impact it can make in basic education is to address
early grade reading as an outcome that is critical to sustain and ensure learning for
children.” (USAID Education Strategy 2011-15),
USAID’s strategy document cites two studies – one that has shown that a 10% increase in the
share of students reaching basic literacy translates into a 0.3 percentage point higher annual
growth rate for that country9, and another that has shown that early grade reading competency
is critical for continued retention and success in future grades10
Primary grade reading competency is essential to successful learning in future grades, and the
development of the skills necessary for productive livelihoods. Children in primary grades who
are not reading at grade level are therefore at risk of a lifetime trajectory of limited educational
opportunity and limited economic opportunities. What applies at the level of the individual child
has consequences at the national level. A nation with a high level of illiteracy is less able to take
advantage of development opportunities. An improvement in national standards of literacy
translates into increased annual economic growth. The 2006 UNESCO Education for All
Monitoring Report11 focused on literacy. Chapter 5 identifies benefits – human, political,
cultural, social and economic – summarizes (pages 143-145) the evidence for economic
benefits. The U.S. National Institute for Literacy carried out a study12 of adult literacy which
“found strong relationships between literacy proficiency and earnings among high school
dropouts” (p. 19).
2) THE SITUATION IN KOSOVO
Reading is taught in Kosovo schools, with a measure of success. A baseline survey carried out
by the BEP13 using an Albanian adaptation of the USAID-supported Early Grades Reading
Assessment (A-EGRA) showed that by the end of grade 2 there are in fact very few non-readers
in Kosovo public schools. Almost all students have basic phonemic awareness and knowledge
of letter-sound correspondence. Most students also have a reasonable level of decoding skills,
and some sight vocabulary.
However, the survey also showed areas of weakness which will need to be addressed if Kosovo
is to compete effectively in the greater European economy. About half of all students tested did
not achieve an acceptable level of oral fluency for their grade level; and about four out of ten
could only answer reading comprehension questions at the very lowest level, retrieval of
explicitly-stated information. However, it should not be assumed that all students are weak in
9 Hanushek, E. and L.Woessmann (2009). Do Better Schools Lead To More Growth? Cognitive Skills,
Economic Outcomes, and Causation. NBER Working Paper 14633. National Bureau of Economic Research.
10 Patrinos,H.A.andVelez,E.(2009). “CostsandbenefitsofbilingualeducationinGuatemala:Apartialanalysis.”
International Journal of Educational Development 29(6): 594598.
11 UNESCO (2005). Literacy for Life: EFA Global Monitoring Report. Paris, 2005
12 National Institute for Literacy, Adult Literacy Development and Economic Growth, Washington, DC 20006
13 USAID Basic Education Program, Establishing M&E Reading Baseline using Albanian-language Early Grade
Reading Assessment (A-EGRA), Pristina, 2012

reading. In fact, about a third read at a satisfactory level, and a quarter had a sufficient level of
sight vocabulary, decoding skill, oral fluency and comprehension to suggest that they could be
on their way to achieving the desired level. But about four in ten were at clear risk of failing to
achieve the required basic level of proficiency.
It seems likely that the teaching must bear significant responsibility for this situation. One
indicator of this is that test-takers’ dictation scores were unexpectedly high. The overall pattern
of result suggested that some teachers at least were putting a great deal of emphasis on basic
decoding skills and on dictation-like tasks, but much less on developing fluency and
comprehension.
3) SPECIFIC CHALLENGES IN IMPROVING READING
Many factors which can contribute to low reading levels are not found in Kosovo. However,
some critical challenges do remain, such as limited instructional time, lack of relevant and
accessible reading material, ineffective teaching practices and limited use or availability of
reading assessment instruments.
a) Instructional time
In Kosovo, most basic education schools are two-shift. The primary grades are normally
taught in the afternoon shift, and have relatively limited contact hours.
Within the new Kosovo curriculum framework, the amount of time allocated to languages in
grades 1 and 2 is 38%, which will help to resolve this problem, when it is implemented by
schools.
b) Availability of resources
Many schools have very limited library facilities; the library generally has very limited
resources; and in some cases, it is rarely open. As a result, relevant and accessible reading
material is hard to find; a focus group study on reading among students and teachers
conducted by the BEP early in 201214 found that most students had difficulty in getting the
books they want to read, and teacher agreed that libraries were inadequate. They said that
students frequently had to read age-inappropriate and uninteresting material, simply
because nothing else was available.
The BEP focus group study also showed that technology is largely absent from schools.
Teachers are skeptical of the internet, and want students to continue to focus on paper
books; but students depend on it increasingly (in the absence of alternatives), although they
are frustrated that they cannot find all that they want.
c) Traditional teaching organization and practices
In Kosovo, a child remains with the same teacher throughout the first five grades. If a child
has a capable teacher, and interacts well with that teacher, this is a very effective strategy. It
allows the teacher to build up a strong relationship with each child in her class, to know their
14 See: Report from focus groups with teachers of primary and lower secondary schools; and Report from
focus groups with students of primary and lower secondary schools, both by Shpëtim Elezi, and available
from the BEP.

strengths and weaknesses, and therefore support their learning as effectively as possible.
However, if the teacher is less capable, or does not relate well with the child, the resulting
problems will be magnified, because the child will not change teacher at the end of one
grade.
The BEP focus group study on reading showed that many teachers are aware of modern
methods of teaching reading; but observational evidence suggests that some primary-level
children are at a distinct disadvantage because their teachers do not practice effective
strategies for teaching reading.
In teaching and testing comprehension, too little attention is given to developing oral fluency,
although there is an awareness of the importance of reading aloud; and in comprehension,
there is an excessive emphasis on questions requiring retrieval of explicitly-stated
information, and too little attention given to higher-order questions.
d) Limited use or availability of reading assessment instruments
Currently there are no external assessments before the grade 5 assessment, which is
sample-based. There is currently no formal measure of early grades reading in Albanian,
although one is developed by the BEP, and will most probably be adopted by the MEST
when it is fully ready. Therefore, teachers have very little opportunity to become aware of
the actual level of reading achievement. The BEP focus group study showed that even
those teachers who are aware of the importance of assessing reading do not have clear
criteria or good access to diagnostic assessment tools, so there is no setting of targets for
improvement.
e) Limited public awareness
There is not a strong tradition of reading for pleasure in Kosovo; and little public awareness
of the importance of parental support in promoting children’s reading. In the BEP focus
group study cited above, teachers generally expressed dissatisfaction with the low level of
interest parents showed in their children’s reading. However, parents that participated in a
survey organized by BEP reported that they were interested in increasing the reading skills
of their children. Most of the parents surveyed were not happy about their access to
teachers. They felt that schools are not very open and welcoming to parents.
4) READING STRATEGY
There is a growing body of research that identifies key interventions that improve reading,
including structured reading instruction, providing more and better reading materials, training
teachers to teach reading, parental and community participation, and frequent assessment.
There is also widespread agreement that improving learning outcomes on a national scale,
particularly in reading, requires participation at all levels, including teachers, school
management, national policy makers, communities and the public at large.
The BEP Reading Strategy aims to be comprehensive, and to deploy all the relevant resources
of the Program. It includes: professional development for teachers of reading; additional
reading materials for schools, including by the introduction of relevant ICT resources; early
grades reading assessment instruments, and training for teachers and others in their use;
strengthening school management through training for school directors and school boards; and

mobilization of parents and the wider community in support of the efforts of schools and
students.
BEP will also encourage teachers to use a comprehensive approach to literacy as reading and
writing skills are so closely related. (See Section 5c)
5) KEY PILLARS IN READING STRATEGY
Within this framework, the interventions that BEP will support can be divided into three broad
areas, as follows:
a) Improved Reading Instruction
i) Support or provide teacher professional development;

ii) Encourage reading at home;
iii) Support supply of reading materials;
iv) Increase awareness of performance standards in reading;
v) Promote regular use of reading diagnostic tools;
vi) Address gender dimensions that impact reading levels of girls and boys
b) Improved Reading Delivery Systems
i) Assist the MEST to set performance standards and benchmarks for reading;
ii) Support school-based implementation of reading improvement programs;
iii) Strengthen central and local systems for testing reading outcomes and monitoring
performance;
iv) Promote parent and community engagement in children’s reading;
v) Encourage better resourced and accessible school and community libraries;
vi) Establish professional standards and codes of conduct (e.g. assessment)
vii) Support professional development for school directors and governing bodies;
viii) Support appropriate, cost-effective and scalable technologies (e.g., audio, video,
computers, internet, mobile devices) that have the potential to improve reading and
the teaching of reading;
ix) Identify gender dimensions that could impact reading levels of girls and boys
c) Improved Writing Instruction

i) Support or provide teacher professional development;
ii) Provide templates, videos and other support materials for schools;
iii) Organize competitions and other activities to encourage students to be active and
creative writers;
iv) Provide advice for parents how to help students with writing through Parent Councils.
d) Increased Engagement, Accountability, and Transparency
i) Encourage school management committees to include reading in school

development plans;
ii) Mobilize communities to address the reading issues of the school;
iii) Promote community and education stakeholder access to and utilization of education
data for local decision making;

iv) Implement a media campaign to increase public awareness about the importance of
reading;
v) Mobilize and engage the not for profit and private sectors to provide a local market in
learning materials or to advocate for improved reading environment.
6) THE CONTRIBUTION OF BEP PROJECT COMPONENTS TO IMPROVING READING
The Basic Education Program has three main components, plus a set of cross-cutting goals in
areas such as gender integration.
a) Component 1: Training for School Directors, School Governing Bodies and Parent
Councils
In collaboration with the Ministry of Education, the Program will:
i) Support professional development for school directors and governing bodies

ii) Encourage schools to include reading in school development plans
iii) Mobilize communities to support the school in addressing reading
iv) Promote community and stakeholder access to data for local decision making
v) In order to promote parent and community engagement in children’s reading, and
especially to encourage reading at home, the Program will develop a workshop for
Parent Councils which will give guidance to parents how to help their children learn
to read.
b) Component 2: Setting Standards and Early Grades Reading Assessment;
In collaboration with the Assessment Unit of the Ministry of Education, the Program will:
i) develop an early grade reading assessment in Albanian based on international best

practice.
ii) train teachers to administer the early grade reading assessment as a screening and
diagnostic tool;
iii) define and disseminate performance standards and benchmarks for reading at grade
5 level; and
iv) support central and local testing and monitoring of reading outcomes
c) Component 3: Teacher Professional Development
In collaboration with the Ministry of Education, the Program will:
i) Develop a three-day, accredited teacher professional development program on

teaching reading;
ii) Train two teachers in each partner school as reading facilitators;
iii) Deliver the reading course through a school-based approach that supports the
creation of a community of learning in regards to reading;
iv) Support supply of reading materials;
v) Support school-based implementation of reading improvement programs; and
vi) Support appropriate, cost-effective and scalable technologies (e.g., audio, video,
computers, internet, mobile devices) that have the potential to improve reading

d) Cross-Cutting
The Program has appointed a Literacy Coordinator, whose responsibility is to ensure a

coordinated approach to implementing the Program’s Reading Strategy across
components. Cross-cutting activities will include:
i) Implement a media campaign to increase public awareness of the importance of

reading;
ii) Mobilize and engage the private sector to provide learning materials or advocate for
improved reading environment;
iii) Organize literacy-focused activities (e.g. Children’s Book Fair, Bookmaking
competitions, best library competition);
iv) Propose relevant professional standards and codes of conduct ;
v) Identify and address gender dimensions that could impact reading levels of girls and
boys
ANNEX 1 – CLASSROOM TECHNIQUES AND ACTIVITIES TO IMPROVE READING SKILLS
These will be included in an in-service teacher training course and a guidebook for teachers.
Videos will also be made to demonstrate some of the activities.
The literacy The literacy center is the part of a classroom where students practice reading,
center writing and other communication skills. It enables an appreciation of literature
to be addressed in the classroom every day. The Center can include a quiet
library and reading area with books and other reading materials. This quiet
space enables children to browse books, read one another, and enables
adults to read children. Covering the floor with carpet and soft cushions helps
to reduce noise and is more comfortable for reading. Materials for the literacy
center can include:
 Books appropriate for the age group,
 books created by children,
 encyclopedias,
 a thesaurus
 alphabets
 a word wall
 wall dictionary15
 a washing line16
 letter or word cards17 depending on the grade
 magazines
 posters with poems, stories and songs
 Quotations
 Paper, pencils, pens, crayons, eraser, glue,
 Bookmaking materials
 computers
 video camera
15 Provided to trained teachers as part of a “Reading Kit” (see Appendix 2)


 dolls,
 puppets and stuffed animals,
 masks and costumes for drama
Key words Teachers make lists and flash cards of most used words. In Kosovo we began
with the 20 most used words in Albanian.
Word Walls Students write words on card and these are displayed on the wall so that all
children can see them (particularly from the classroom writing center). In the
early grades teachers can write the words (e.g. the 20 most common
Albanian words) but children will soon want to copy words themselves.
At lower-grades, word walls can display words that students meet in their
reading and other frequently used words. New words can be added to the
word wall each week. As the “word wall” grows in size, students can see how
many words they have learned. Seeing words on the wall helps children
become interested in words and understand that words are important and can
be used over and over again. The word wall helps them learn the names of
letters, alphabetical order, and letter-sound relationships. It provides extra
exposure and challenge for students who are at many different skill and
interest levels. Students can reference their personal word wall pages during
writing projects or journal writing. The word walls are constant learning cues
for the students, who can use the word wall as a huge dictionary and look at
the word wall when they are unsure of how to spell a word.
4 & 5 grade teachers can use word walls for displaying more sophisticated
words that can be substituted for more frequently used words, therefore
developing a wider vocabulary in their students.
Paired Older students are a “buddy” to younger students and hear them read on a
Reading regular basis.
Shared Multiple copies of texts are provided to small groups. Individual students read
Reading in turn aided by their colleagues.
Self and Peer Teachers make materials available in the literacy corner so that students can
Assessment assess and correct their own work.
Materials can include :
 word lists and dictionaries so they can check spellings,
 a thesaurus or word wall so they can identify alternative vocabulary,
 posters with grammar and punctuation advice so they can correct any
errors they make,
 lists of grading criteria for the task so they know what is important.
Cloze Teachers provide texts with missing words to develop students’ ability to
Exercises predict and use grammatical and contextual cues to help them read more
quickly.
Missing Teachers provide texts with missing endings to encourage students’

Endings imagination, creative writing and prediction skills.
Big Books Teachers make or use large format books which can be read by a whole

group. This facilitates discussion and enables the teacher to focus on words
to make grammatical, vocabulary or other points.
Use of ICT Students can access existing games, such as the cloze exercises discussed
above, and use other software in Albanian to practice reading. Later they can
undertake internet research using “webquests”. They can produce and edit
text on word processing software and share these with colleagues. They can
create stories and place them on a portal such as “LibraTek18” where they can
read stories by other students. They can write e-mails and blogs and
communicate with students in other schools and countries.
Students can send texts on mobile phones and make up their own
abbreviated vocabulary. This can lead to realistic discussions on grammar
and sentence construction.
Story
Sequence Helping students learn to recall the facts of a story in the proper order is a
skill that aids comprehension. Sequencing is an important part of problem
solving across subjects. Sequence structures help students of varying
abilities organize information and ideas efficiently. Below are some examples
in different subjects:
Language Arts
Story maps provide one way to help students organize the events from a
story. Helping students learn transition or signal words that indicate a
sequence (first, second, last) will also help them learn about sequence.
Sequence sticks, story chains, and a story sequence craft all help students
practice ordering events within a story.
Mathematics
The math curriculum includes ordinal numbers (first, second, third, etc).
Patterns are also a form of sequencing my encouraging the use of vocabulary
words such as "What bead goes first? Then which bead? Which bead is
third?" Encouraging students to write out the steps for solving problems that
include regrouping is an excellent way to have them think through the steps
in order. Teachers can use a simple sheet of paper folded into four squares
and ask students to write the steps in order in the squares. Cut or tear out the
pages from an old calendar. Mix up the months and hand out the stack of
pages. Ask students to order the months from January to December by laying
the pages out on the floor. Which month goes first? Then which one? Which
month is last?
Science
Helping students sequence also develops their scientific inquiry skills. In
order to study or observe changes in something, students must follow along
and record changes. The changes happen in a particular order, which
students can document by writing or drawing pictures.
Social Studies
Students can make a timeline of their own life, and include important
18 See page 10

milestones such as when they learned to walk, talk, ride a bike and go to
school. Once students understand the process of charting important
milestones on a timeline, topics from the social studies curricula can be used.
RAFT
RAFT is a writing strategy that helps students understand their role as a
writer, the audience they will address, the varied formats for writing, and the
topic they'll be writing about. By using this strategy, teachers encourage
students to write creatively, to consider a topic from a different perspective,
and to gain practice writing for different audiences.
What does RAFT stand for?
Role of the Writer:

Who or what are you as the writer? A pilgrim? A soldier? The President?
Audience:
To whom are you writing? A friend? Your teacher? Readers of a newspaper?
Format:
In what format are you writing? A letter? A poem? A speech?
Topic:
What are you writing about? Why? What's the subject or the point?
Why use RAFT?
It includes writing from different viewpoints.

It helps students learn important writing skills such as audience, main idea,
and organization.
It teaches students to think creatively about writing
It can be used across various content areas
Writing Research on the writing process suggests that writers learn the most about
Conferences writing when they share and reflect on their writing. In classrooms, this is
most commonly done through writing conferences as part of the revision
stage. Whether they occur with pairs, with small groups, or with the teacher,
the social benefits of sharing writing improves writing.
Why conduct writing conferences?
 They involve a non-threatening audience of peers

 Peers or teacher can provide immediate feedback or can immediately
ask for clarifications
 They can promote positive attitudes about writing and increased
motivation to revise
 Students experience a wide range of writing abilities and topics
There are many, many ways to conduct writing conferences. Teachers will
have the most success if they model conference behaviors with the whole
class. Class discussions can focus on what sorts of comments and actions
are useful and constructive, and what sorts of behaviors are less helpful to
the author sharing his or her work.

Donald Graves (1982) identified six characteristics of successful writing

conferences. Conferences should:
(1) have a predictable structure;
(2) focus on a few points;
(3) demonstrate solutions to students' problems;
(4) permit role reversals;
(5) encourage use of a vocabulary appropriate for writing; and
(6) stimulate pleasure in writing.
The authors Students who write their own stories, poems and books can read them from
chair the “authors chair” before they are added to the library collection. It is good
to find a chair with an unusual or different shape. When children sit on this
special chair to read their best work for other children, they feel important and
this activity encourages them to write more carefully and thoughtfully. The
teacher or a visitor can also use the chair when they read to students.
Bookmaking Making books is much more interesting than writing in a copy book. Through
by students “Book Making” children think about themselves as “authors”. They can create
books, news papers, magazines, letters and other written materials. On the
computer they can write e-mails and blogs. This helps them understand how
written language is used in real world situations and motivates them to write
well. The topics of the books can be connected with the actual curriculum but
also related to children’s everyday life or their fantasy. Children can make
their own, individual book but they can also produce book in pairs or as a
group.
Teachers provide bookmaking materials in the literacy center. The literacy
center can include blank papers in different sizes, patterns and colors, color
pencils, markers, card, staplers, hole punchers, glue, needles and thread and
other items useful for self made books.
Once they complete their own book, Students can present the book from the
“authors chair” (see above) before it is added to the library collection. The
book making activity not only helps children to acquire the literacy skills, but
increases their confidence and independence.
“LibraTek” BEP will develop a portal where students can share their stories and other
Online Library writing. This provides an audience for young authors.
Book Reports
ZIP Time (No Walk, No Talk!)
ANNEX 2 – LITERACY KIT
The following kit of equipment will be provided to schools of teachers who attend the Reading
Course
“Mini-books” 25 “mini-books” for Grade 1 students. Each of the “mini-books” consists of 8

pages which are printed on both sides of one sheet of A4 paper. When the
paper is cut and folded it can be stapled or folded to make a small book. Each
page contains basic vocabulary appropriate for Class 1 students and a black

and white illustration which can be colored by the students. By the end of
February, 2013, the Program plans to make these “mini-books” available for
download from the internet but will also print sets of the books for teachers
who attend the Program’s reading course.
Graded A series of 20 graded full-color reading books for students in Grades 1 & 2.
Reading These will use graded vocabulary based on a word bank created after
Books research by the Program’s Assessment unit. It is planned that the whole set
will be ready for distribution to schools for the 2013-14 academic year, after
piloting in selected schools during this academic year.
Letter Cards Cards with the letters of the Albanian alphabet
Word Cards Cards of the 100 most used words in Albanian.
Wall The wall dictionary is similar to the word wall but has pockets to keep the
Dictionary word cards. This is useful in a shift school where different classes may need
to learn different words. The wall dictionary hangs on the wall and can easily
be taken down and replaced with a different one. This is useful in a school
with multiple shifts. Students can use the wall dictionary like a real dictionary.
They think of the initial letter of the word and look in the appropriate pocket to
find the word they need. Students can use this wall dictionary at any time to
help to learn spellings and recognize new words. It is also good for practicing
alphabetical order.
To make the wall dictionary sew enough pockets on it and label each pocket
with a letter of the alphabet. You can make wall dictionaries for all languages
– just make sure to make the correct number of pockets. Make sure you
leave space at the top and bottom of the wall dictionary for pieces of round
wood or cane. These keep the wall dictionary in shape and string is tied to the
top piece of wood for hanging. The teacher or the children can prepare the
letter labels to pin on the pockets. The teacher can put some basic word
cards in the wall dictionary but the children should make cards for each new
word they learn and put it in the correct pocket.
Washing line The washing line can be used for mathematics but also for active literacy
work.
 Letter cards can be hung on the washing line so the children can
make words.
 Word cards can be hung on a washing line to form sentences.
 Students can then copy their words and sentences on paper or into
their copy books.
ANNEX 3 – PROGRAM ACTIVITIES TO PROMOTE INTEREST IN READING AND LITERACY

Literacy Day Poetry “Unplugged” in Support of Literacy: On September 8, 2011, three

well-known Kosovo poets donated their time and talent in support of
International literacy Day. Arif Demolli, Mevlyde Saraçi and Xhevat Syla read
from their work to students, and then discussed the importance of literacy and
what inspires them to write. Forty students from five Pristina primary schools
were invited to the event, which was called “Poetry Unplugged.” After the
readings and discussion, the students participated in a poetry composition
workshop. The Program organized the event, which took place in the
comfortable and colorful library of Pristina’s innovative Green School, which
was officially opened by the U.S. Ambassador earlier in the year. The school,
which features recycled material and an eco-friendly design, was built with
assistance from USAID. To encourage reading among the students, the
Program procured child-friendly furnishings and books for the school library.
Authors In 2013, the Program plans a similar activity to the one above but this will
Caravan entail authors travelling to a number of schools throughout Kosovo to
introduce their work to students.
Parents In 2012 the Program developed a workshop for Parent Councils in schools.
Workshops This workshop focuses on how parents can help their children get the best
out of school. In particular it provides guidance to parents how to help their
children learn to read.
Classroom In June 2011 the Program worked with the parents and staff of one primary
Makeover school to transform a bare room to a child-friendly learning environment with
a literacy corner and other features to help young students learn better. The
classroom renovation activity was planned to
 raise the awareness of the need to improve the learning environment in
Kosovo’ primary school classrooms
 demonstrate how schools, parents and local communities can do this;
and
 offer practical advice on cost-effective ways to make classrooms more
effective learning environments.
The Program has developed a manual and a training program for school
boards to enable them to organize their own classroom makeover. Each year
the Program will support 50 such renovations.
The national broadcaster, RTK, made six TV programs showing the
classroom makeover activities in six different schools. These were shown
weekly from 17th December 2011.

School Library The program will organize a competition in 2012-13 to recognize the library
of the Year which does the most to promote reading.
A good school library is much more than the one that simply provides
students with books to read. It should help students develop a love of
literature and develop research skills that will enable them to become life-long
learners. There are many opportunities for schools to create and maintain a
school library that helps students upgrade their reading skills and become
“functionally literate”. This competition seeks to reveal the extent to which
these opportunities are used by our schools, but also to learn about original
ideas made real by schools in respect of school library usage. The program
will give prizes of education equipment to the winning library.
Criteria used by the evaluation committee to select the winners include:
 Provision of a friendly environment to all students (librarian’s approach
to students, accessibility to the library, displays, etc.);
 A diversity of reading materials (books, magazines, graphic books-
comics);
 Percentage of students who are active users;
 Number and types of events organized to promote literacy such as
competitions, challenges, story writing and book binding (especially
among younger children);
 Strategies to break-down gender and all other kinds of stereotypes;
 Effectiveness in maintaining library’s database and records of
individual student use;
 Use of electronic and online resources;
 Any other significant result that leads to promotion of reading (e.g.
peer support and older students helping younger student with their
reading).

Role Play and In its reading course the Program will encourage teachers to enable students
Movie Making to act out what they read to reinforce vocabulary and raise interest. A ‘home
made’ puppet theatre was featured in the classroom makeover TV program to
show how puppets can be used to encourage shy students to participate in
role play. The Program has also trained teachers to use small video cameras
and distributed these to 30 schools. The training courses included sessions
on: how to write a story board; how to prepare for filming and use the
cameras effectively; as well as the use of different software to be used in
editing filmed materials. In 2011 and 2012, the Program held a “Junior Oscar”
film competition in which schools were given the opportunity to enter short
movies, of up to 10 minute duration. The winning films were presented to a
gathering of students, teachers and guests in the ABC Cinema, Pristina, as
part of the “Skena Up” International Students Film & Theatre Festival.
Children’s In 2012, the Program organized a Children’s Book Fair where publishers
Book Fair could showcase local books for children. The Fair also featured a model
classroom equipped to promote reading and literacy in which students
undertook a number of literacy-based activities including listening to authors
read their books, writing their own stories and binding books.

Graded In 2013, the Program will publish a set of 20 (twenty) graded reading books,
Readers each 16 pages long, based on stories and characters that motivate students
and enable them to read with understanding from an early age.
The target audience for the twenty story books will be children in Grades 1
and 2. The stories will use limited vocabulary appropriate to the age group.
Relevant word lists will be provided by the Program’s experts. A list of
vocabulary to be used in the proposal is given in Attachment The books will
develop the sight vocabulary of early readers through word recognition and
improve their verbal fluency and comprehension skills. The books will be in
four levels, each with five books.
Level 1 books will contain between 50 and 80 words
In order to help students improve their reading skills, the vocabulary in the
four levels will become progressively wider and sentences will become more
complex.
The criteria to be used by the Basic Education Program in selecting stories
for publication will include the use of limited graded vocabulary and key
words; phonetics; illustration closely linked to the text; characters that
students can relate to; gender equity and inclusiveness; humor; rhyme; and
engaging stories appealing to students from Grades 1 – 3.
“Mini-Books” In 2013, as part of its efforts to improve the reading skills of Kosovo’s
for Emergent students, the Program will publish a set of 25 (twenty five) stories in the form
Readers of photo-copiable “Mini-Books”. Each “mini-book” will be composed of 8
pages printed on two sides of an A4 sheet. The A4 sheets can be cut into 2
pieces and folded to make an 8 page “mini-book” which can be read by the
student. The black and white illustrations accompanying the stories can be
colored by the students. In total the 25 stories will contain 200 illustrations.
The target audience for the twenty five stories will be children in Grades 1
and 2. The stories will use limited vocabulary appropriate to the age group.
Instructions for each illustration will be provided by the Program.
Bookmaking This activity, which will be organized in 2013, aims to promote creativity
competition among primary school children by asking them to write and create short
for students books. This includes binding books, filling them with stories and illustrating
them. Teachers will be trained to facilitate this process with their students.
The Program will invite interested teachers to attend regional workshops for
book making with some of their students. Books created by children in their
schools will be exhibited at the regional level, and the best books will be
selected for a central exhibition to be held in May 2013. A jury will select the
best books in each category and the Program will give prizes for the best
books.

APPENDIX B: DETAILED ANALYSIS & TABLES
Defining adequacy of performance
The A-EGRA is designed to be criterion-referenced (i.e., assessing performance against defined goals or outcomes) rather than
norm-referenced (i.e., ranking performance based on total score). One of the major issues to be addressed is therefore how to
define adequacy of performance.
Standards of comparison were sought from outside Kosovo; but relatively few were found. Percentile oral fluency norms19 were
found for the US. These are based on a large number of cases, but do not attempt to define adequacy. Norms for the DIBELS
test20 do attempt to define adequacy, either in terms of level of “risk” or “deficit”. These however are only available for letter naming,
non-word reading, and oral fluency. Where relevant, these norms have been given in the tables below. The basic principle
enshrined in the DIBELS approach has also been applied to reading and listening comprehension; but this is not as yet empirically
based.
According to Seymour et al.21, learners acquire reading in alphabetic languages in three phases. In the first phase, the learner
acquires knowledge of the smallest building blocks of reading, by recognizing letter sounds (grapheme-phoneme correspondence)
and their combination into simple words. This involves basic skills such as fluent decoding of letters into sounds, creation of a bank
of words they can recognize on sight, and the ability to decipher new or unfamiliar words through an understanding of letter-sound
correspondence.
In the second and third phases, learners acquire a full understanding of the spelling system and fluency with rimes (larger units of
letter-sound correspondence). They then progress to fluency with morphemes (association of phonemes with semantic meaning),
and build a framework that involves not only decoding but also attributing meaning to written text.
For purposes of analysis, the subtests were divided into three main sections which loosely reflect these three levels:
 Basic skills (phonemic awareness and letter name/sound knowledge)

 Sight vocabulary and decoding (familiar word reading and non-word reading)
 Oral fluency and comprehension (listening/reading)
19 Excerpted from: Hasbrouck, Jan and Gerald A. Tindal. “Oral Reading Fluency Norms: A valuable assessment tool for reading teachers.” The Reading
Teacher, Vol. 59 No. 7, April 2006, pp. 636-644
20 Downloaded from: https://dibels.uoregon.edu/docs/benchmarkgoals.pdf
21 Seymour, P. H. K., Aro, M., & Erskine, J. M. (2003). Foundation literacy acquisition in European orthographies. British Journal of Psychology, 94, 143–
174.
Besnik Krasniqi.and Keith Prenton, September 2014

Basic Skills
First two sub-tests – phonemic awareness and letter name/sound knowledge – are essentially aimed at first graders; but they were
included in case a significant proportion of learners were, at the end of grade 2, still reading at a grade 1 level.
a) Phonemic Awareness
The phonemic awareness test, which involved segmenting words into phonemes, proved generally rather easy.
In this case, the individual phonemes were not scored; test takers had to segment the whole word correctly to get a mark. The
maximum score for the 25 words was therefore 25. The mean score on this sub-test was 21.06 out of 25. Half of the test takers
scored 23 or more out of 25. Three quarters of test takers scored 20. Of the remainder, most scored 14 or more out of 25.
Concerning the category of test takers classified as the “discontinued” category (which represents those who failed to answer
any items), the data show an increase in performance as this category has decreased from 4.3% to 1.9 %. This improved
performance of second grade children is despite the fact that this test over-states the extent of real failure, because
“discontinued” includes a number of children who did well enough on the remainder of the test to suggest that they in fact failed
not because they could not do the task, but because they were not ready, or did not understand what to do.
VTTotal
N Valid 796
Missing 0
Mean 21.06
Std. Deviation 5.706
Percentiles 10 14.30
25 20.00
50 23.00
75 25.00
90 25.00
VTTotal

Valid Cumulative
Frequency Percent Percent Percent
Valid 0 15 1.9 1.9 1.9
1 2 .2 .2 2.1
2 10 1.2 1.2 3.3
3 3 .3 .3 3.6
4 4 .5 .5 4.1
6 6 .7 .7 4.8
7 7 .9 .9 5.7
8 5 .6 .6 6.4
9 5 .6 .6 7.0
10 4 .5 .5 7.5
11 4 .5 .5 8.0
12 6 .8 .8 8.8
13 2 .2 .2 9.0
14 8 .9 .9 10.0
15 13 1.6 1.6 11.6
16 13 1.7 1.7 13.3
17 26 3.2 3.2 16.5
18 23 2.9 2.9 19.3
19 22 2.7 2.7 22.1
20 41 5.1 5.1 27.2
21 53 6.6 6.6 33.8
22 62 7.7 7.7 41.5
23 119 15.0 15.0 56.5
24 80 10.1 10.1 66.6

25 266 33.4 33.4 100.0

Total 796 100.0 100.0
b) Letter Name/Sound Knowledge
Letter name/sound knowledge is a fundamental and very basic measure of reading readiness. The mean number of letters read
correctly in 2012 was 78.03, and the standard deviation 19.29 while in 2014 was 97.5 and the standard deviation 16.23. This
finding shows that there is a significant improvement in 2014 compared to 2012. Tthere is also DIBELS norms for letter name;
but the DIBELS Letter Naming Fluency (LNF) norms are only for grade 1 level. Where LNF is below 25 letters correct per
minute (LCPM), the test taker is deemed to be “at risk”. Where LNF is 25 LCPM or more, but less than 37 LCPM, the test taker
is deemed to be at “some risk”. Where LNF is 37 LCPM or more, the test taker is deemed to be at “low risk”.
By these standards, virtually none of the test takers in this sample were defined as being “at risk” at all.
Letter Knowledge (name/sound) scores divided into DIBELS Categories
2012 2014
DIBELS Category Freq. Percent Valid % Freq. Percent Valid %
Vali
At Risk (LNF < 25) 3 .4 .4 9 1.2 1.2
d
Some Risk (25 <= LNF <
9 1.1 1.1 14 1.8 1.8
37)
Low Risk (LNF >= 37) 787 98.4 98.5 773 97.1 97.1
Total 799 99.9 100.0 796 100.0 100.0
Mis
System Missing 1 .1 0 0 0
sing
Sight Tota Vocabulary
800 100.0 796 100
and l Decoding
a) Familiar Word Reading (CWPM)
This is a measure of sight vocabulary – recognition of high-frequency words selected from early-grade reading materials,
independent of context. Using a list of unrelated words allows for a purer measure of word recognition and decoding skills than
does reading comprehension paragraphs, as test takers are unable to guess the next word from the context. This is in fact

reflected in the higher scores test takers achieved on the oral fluency test. There is an increase in mean of correct words reading
per minute in 2014 compared to 2012 indicating an improvement in among the second grade children.
CWPM 2012 CWPM 2014

N Valid 799 796
Missing 1 0
Mean 49.90 54.97
Std. Dev. 19.69 15.43
CWPM 2012 CWPM 2014

Percentiles
10 23.66 40.00
25 36.00 45.00
50 50.00 49.27
75 62.73 36.03
90 75.00 77.13
c) Non-word Reading (CNPM)
Successful readers must combine both decoding and sight recognition skills. Reading unfamiliar words involves decoding,
which tends to be slower than sight recognition of vocabulary. Even in the early grades, many children have a substantial sight
vocabulary. If they do not also acquire decoding skills, they will not be able to add new vocabulary very efficiently. Non-word
reading avoids the problem of sight recognition of words, and offers a good measure of decoding skill.
The tables below give the mean number of correct non-words read per minute, and the performance at the 10th and 90th
percentiles, and the quartiles. Accordingly, the EGRA survey shows a significant improvement on overall score of CNPM in 2014
compared to 2012.
CNPM 2012 CNPM 2014

N Valid 798 796
Missing 2 0

Mean 40.50 47.65

Std. Dev. 15.88 10.57
Percentiles CNPM 2012 CNPM 2014

10 21.00 37.00
25 29.00 42.00
50 39.59 47.00
75 51.72 50.00
90 60.00 62.64
The performance of this group was overlaid on the DIBELS performance levels for this skill, and the result is given in the table
below.
Non-word reading - DIBELS Categories (NWF)

2012 2014
Categories Frequency Percent Frequency Percent
Deficit (NWF < 30) 216 27.01 18 2.2
Emerging (30<=NWF<50) 355 44.35 545 68.4
Established (NWF >= 50) 227 28.33 224 28.1
Total Valid 798 99.70 786 98.7
System Missing 2 0.30 10 1.3
Grand Total 800 100.00 796 100.0
This measure suggests that although virtually no children lack decoding skills in 2014. In particular, there is a substantial
improvement in terms of decrease of children in emerging category. Data basically show that large proportion of children moved
from deficit to emerging category in 2014 compared to 2012. The established category has remained almost the same.
Oral Fluency and Comprehension
a. Oral Fluency (CWPM)

The EGRA results presented in table below show that overall oral fluency among second grade children has shown a small increase
in 2014 (73.39) compared to 2012 (72.23).
CWPM 2012 CWPM 2012
N Valid 794 796

Missing 6 0
Mean 72.23 73.39
Std. Deviation 30.74 23.70
Kosovo Kosovo US Norms22

Percentiles CWPM 2014
CWPM 2012
10 35.00 54.00 31
25 50.00 58.00 61
50 68.73 62.00 89
75 93.28 86.12 117
90 114.55 106.28 142
This table shows that while performance amongst the lowest achievers in Kosovo exceeds U.S. standards for their percentile group,
standards of oral reading in the U.S. are generally higher than in Kosovo – and that the difference increases amongst the higher
achievers. These differences remain stable in 2014 as well as 2012.
The results were also allocated to the DIBELS categories, as given in the table below. This analysis shows a rather high proportion
of test takers falling in the “at risk” category. In 2014 the analysis shows a higher increase of children “at risk” and a decrease of
“low risk” category.
Oral Fluency in DIBELS Categories

2012 2014
Frequency Percent Frequency Percent
22 From: Hasbrouck, Jan and Gerald A. Tindal. “Oral Reading Fluency Norms: A valuable assessment tool for reading teachers.” The Reading Teacher,
Vol. 59 No. 7, April 2006, pp. 636-644

At Risk (ORF < 70) 406 50.80 472 59.3

Some Risk (70<=ORF<90) 159 19.86 144 18.1
Low Risk (ORF >= 90) 229 28.59 170 21.4
Total Valid 794 99.25 787 98.8
System Missing 6 0.75 10 1.2
Grand Total 800 100.00 796 100.0
Therefore, if the DIBELS categories are accepted, levels of oral fluency amongst test takers are in fact low enough to be a cause for
concern. This is even more pronounced in 2014.
b. Reading Comprehension
The reading comprehension test was designed to test a range of comprehension skills, starting from retrieval of explicitly-stated
information, but covering also inference and using prior knowledge to inform interpretation. The intention was that an established
reader at this grade level should get all the items right, or make at most one mistake. On this basis, a three-level classification of
scores was developed:
 An established reader is one who gets 4 or 5 marks (i.e., can answer all levels of grade-appropriate questions, including
interpretation and using knowledge of the world);
 An emerging reader may get 2 or 3 marks (can answer at least some questions requiring retrieval of distributed information,
or formation of an interpretation);
 A reader who gets only 1 mark or none at all is defined as a deficit reader (can only retrieve explicitly-stated information, if at
all).
The table below shows the distribution of scores according to these categories.
Reading comprehension divided into DIBELS-type categories

2012 2014
Valid Frequency Percent Frequency Percent
0 or 1 – Deficit 322 40.30 218 27.3
2 or 3 – Emerging 225 28.17 371 46.6
4 or 5 – Established 252 31.54 207 26.0
Total 800 100.00 796 100.0

The above table shows that the test takers fall very much into three groups. In 2012, a rather large number of test takers – about
40% - fall into the “deficit” category while in 2014 this figure was much lower 27%. This finding suggests that there is a decrease of
‘deficit’ category and increase of ‘emerging’ category.
Oral Fluency - Correct Words per Oral Fluency - Correct Words per
Reading Minute 2012 Minute 2014
Comprehension N N
Score Std. Std.
Mean Min. Max. Mean Min. Max.
Dev. Dev.
0 83 42.66 29.21 4.00 171.82 92 66.22 32.68 38.00 286.15
1 233 62.42 26.68 12.00 130.34 268 67.25 18.51 29.00 155.00
2 114 70.80 28.04 25.00 157.50 160 70.93 20.77 46.00 148.80
3 111 74.70 28.14 34.00 164.35 122 77.08 21.40 48.00 143.07
4 113 84.89 24.21 39.00 164.35 72 83.74 19.96 55.86 143.07
5 139 95.20 24.68 47.00 164.35 82 91.68 25.53 55.00 148.80
This analysis shows quite a lot of overlap in the middle range in 2012 – those who score 2 or 3 out of 5 have a very similar rate of
oral fluency, around 70+ words per minute. This overlap holds particularly for those who score 4 out of 5, and those who score 5 in
2014. In 2012 this group shows a relatively clear separation. The finding suggests that an oral fluency rate of at least 80 words per
minute is desirable (those who score 4 and 5 out of 5).
c. Listening Comprehension
Listening comprehension is not generally taught. It is therefore important primarily as an indicator of language development. The
same categories were applied to the listening comprehension test as to the reading comprehension test. The results are given
below.
Listening comprehension divided into DIBELS-type categories

2012 2014
Valid Frequency Percent Frequency Percent
0 or 1 – Deficit 158 19.77 218 27.3
2 or 3 – Emerging 320 39.99 371 46.6
4 or 5 – Established 322 40.24 207 26.0
Total 800 100.00 796 100.0

This table shows rather higher proportions of test takers in the upper two categories in 2012, while in 2014 a percentage of children
in ‘established category’ has decreased. At the same time there is an increase of ‘emerging’ category. This is a cause of concern for
EGRA performance.
A correlational analysis was done aimed at analysis of the relationship between reading comprehension and listening
comprehension. The correlation analysis shows that the relationship is significant at 99.9% level of confidence.
Correlations
LK_Tot DK_Tot
Kendall's tau_b LK_Tot Correlation Coefficient 1.000 .386**
Sig. (2-tailed) . .000
N 816 816
**
DK_Tot Correlation Coefficient .386 1.000
Sig. (2-tailed) .000 .
N 816 816
Spearman's rho LK_Tot Correlation Coefficient 1.000 .471**
Sig. (2-tailed) . .000
N 816 816
DK_Tot Correlation Coefficient .471** 1.000
Sig. (2-tailed) .000 .
N 816 816
**. Correlation is significant at the 0.01 level (2-tailed).
Dictation
Dictation is basically a test of learners’ ability to write, spell and use grammar properly. Students’ ability to hear sounds correctly and
then write the letters and words corresponding to the sounds they hear demonstrates their success with the alphabetic principle.

Their ability to write the words with correct inflections also demonstrates both their comprehension of the sentence and their mastery
of grammar.
The sentence that was read contained five major lexical items (four nouns and one verb); each of these was scored, along with word
spacing, word order, initial capital and final full stop. Each item was scored two points for fully correct, one for partially correct (with
criteria defining what constitutes “partially correct” or zero for incorrect. The dictation test was therefore scored out of 18.
Comparative analysis of 2014 and 2012 show a slight increase of mean of dictation score. This suggests that there are no
differences found in terms of dictation in two periods.
Dictation - Total Score
2012 2014
N (Valid) 797 796
Missing 3 0
Mean 13.74 13.96
Std. Dev. 4.45 3.76
Minimum 0 0
Maximum 18 18
Percentiles 2012 Score 2014 Score
10 8 10
25 12 13
50 15 15
75 17 16
90 18 18
The achieved mean score for both, 2014 and 2012 strongly indicates that relatively few test takers have much difficulty with the
dictation test. Even the bottom decile of test takers show students mostly achieving something.
EGRA PERFORMANCE BY GENDER

In this section we report EGRA performance broken-down by gender. This gender analysis will enable to draw more specific
conclusions on how male and female children perform in different EGRA sub-sections. Table below reports that share of male in the
total sample was higher compared to female probably reflecting their respective shares in total population of second grade children.
EGRA children by gender

Valid Cumulative
Frequency Percent Percent Percent
Valid Female 364 45.7 45.7 45.7
Male 433 54.3 54.3 100.0
Total 796 100.0 100.0
In order to investigate gender differences regarding the EGRA performance of children we conduct hypothesis testing comparing
means of male and female and also using Chi-square hypothesis testing where were have categorical score data.
Phonemic awareness
According to independent sample t-test we did not find statistically significant gender differences regarding phonemic awareness.
The null hypothesis that there are no gender differences is accepted as p-value for t-test is 0.729 which far greater than accepted
level of 0.05 or 0.10.
Group Statistics
dumm
y 1=f Std. Std. Error
2=m N Mean Deviation Mean
VTTotal Femal
364 21.13 5.718 .300
e
Male 433 20.99 5.701 .274
Independent Samples Test for Phonemic Awareness Mean

Levene's Test for Equality of
Variances t-test for Equality of Means

95% C
Mean Std. Error Differen
F Sig. t df Sig. (2-tailed) Difference Difference Lower
VTTotal Equal variances assumed .009 .924 .346 794 .729 .141 .406 -.6
Equal variances not
.346 770.157 .729 .141 .406 -.6
assumed
Letter Name/Sound Knowledge
The same procedure was applied using t-test we did not find gender differences regarding Letter Name/Sound Knowledge.
Group Statistics
dumm
y 1-f
and 2- Std. Error
m N Mean Std. Deviation Mean
Shkronjat_correct_per_mi 1 9.732480
nute 364 7.2979359 .3827156
E1
2 9.767869
433 20.9942090 1.0091271
E1
Independent Samples Test

Levene's Test for
Equality of Variances t-test for Equality of Means
95% Confidenc
Mean Std. Error Difference
Shkronja Equal variances assumed 3.475 .063 -.306 794 .760 -.3538903 1.1555966 -2.6222738
t_correct Equal variances not
_per_mi assumed -.328 551.390 .743 -.3538903 1.0792630 -2.4738604
nute

Sight Vocabulary and Decoding
a) Familiar Word Reading (CWPM
As regard for familiar word regarding we did find statistically significant gender differences. On average female perform
better than male. T-test as t-test p value of 0.079 I lower than 0.10 suggest that there are gender differences.
Group Statistics
dummy
1=f
2=m N Mean Std. Deviation Std. Error Mean
FjalNjoh_correct_per_minu Female 5.603712E
te 364 16.2623020 .8528215
1
Male 5.408872E
433 14.6623589 .7047745
1

95% Co
Mean Std. Error Differenc
FjalNjoh_correct_ Equal variances assumed 4.110 .043 1.777 794 .076 1.9483982 1.0964621 -.203907
per_minute Equal variances not
1.761 737.997 .079 1.9483982 1.1063506 -.223571
assumed
b) Non-word Reading (CNPM)
The t-test performed and reported below did not found gender differences regarding non –word reading. This is confirmed by
hi t-test p value of 0.56 which is far larger than accepted level of 0.10 or 0.05.

Group Statistics
dumm
y 1-f
and 2- Std. Std. Error
m N Mean Deviation Mean
FjaPanjo_correct_per_ 1 4.741410
364 10.8240326 .5676298
minute E1
2 4.785331
433 10.3754282 .4987149
E1

95% C
Mean Std. Error Differe
FjaPanjo_correct_per Equal variances assumed .864 .353 -.583 794 .560 -.4392100 .7528170 -1.9169
_minute Equal variances not
-.581 758.824 .561 -.4392100 .7555926 -1.9225
assumed
Oral Fluency and comprehension
Concerning the oral fluency and comprehension we find significant statistical differences on EGRA performance between male and
female. Female children have comparatively higher mean score per minute (75.4) compared to male (71.6) suggesting the cause of
concern to investigate more low performance of males.
Group Statistics
dumm
y 1-f

OrFlu_correct_per_min 1 7.541404
364 23.0999752 1.2114002
ute E1
2 7.169793
433 24.1047187 1.1586397
E1

95% C
Mean Std. Error Differe
OrFlu_correct_per_mi Equal variances assumed 2.194 .139 2.209 794 .027 3.7161046 1.6825031 .41342
nute Equal variances not
2.217 780.813 .027 3.7161046 1.6762865 .42554
assumed
In addition to the t-test for controlling for mean sample differences between male and female we conducted chi-square test to
investigate differences between Dibels categories the chi-square tests presented below rejects null hypothesis at 0.02 level
of significance suggesting that there are statistically significant differences between male and female. In particular as Table
below shows that male perform very poorly within the risk group and these gender differences are driven by this group.
Therefore we, suggest that future measure should be targeted to improve the category at risk for male.
OrFLu_DEIBELS * dummy 1-f and 2-m Crosstabulation

dummy 1-f and 2-m
1 2 Total
OrFLu_DEIBEL 1 Count 197 275 472
S % within
41.7% 58.3% 100.0%
OrFLu_DEIBELS
2 Count 73 72 145
% within
50.3% 49.7% 100.0%
OrFLu_DEIBELS

3 Count 90 80 170
% within
52.9% 47.1% 100.0%
OrFLu_DEIBELS
Total Count 360 427 787
% within
45.7% 54.3% 100.0%
OrFLu_DEIBELS
Chi-Square Tests
Asymp. Sig.
Value df (2-sided)
Pearson Chi-Square 7.838a 2 .020
Likelihood Ratio 7.834 2 .020
Linear-by-Linear
7.419 1 .006
Association
N of Valid Cases 787
a. 0 cells (.0%) have expected count less than 5. The
minimum expected count is 66.33.
c) Reading comprehension by gender
Statistical hypothesis testing using chi-square suggest that there are no statistically significant differences regarding reading
comprehension by gender. The Pearson Chi-Square p-value of 0.13 exceeds conventional 0.10 level and we cannot reject null
hypothesis about homogeneity of means for male and female.
RCOMP_dibels * dummy 1-f and 2-m Crosstabulation

dummy 1-f and 2-m
1 2 Total
RCOMP_dibel 1 Count 108 110 218
s % within
49.5% 50.5% 100.0%
RCOMP_dibels
2 Count 173 199 372

% within
46.5% 53.5% 100.0%
RCOMP_dibels
3 Count 83 124 207
% within
40.1% 59.9% 100.0%
RCOMP_dibels
% within
45.7% 54.3% 100.0%
RCOMP_dibels
Chi-Square Tests
Asymp. Sig.
Value df (2-sided)
Linear-by-Linear
3.781 1 .052
Association
d) Listening comprehension by gender
Similar to reading, for listening as well the statistical hypothesis testing using chi-square suggest that there are no statistically
significant differences regarding reading comprehension by gender. The Pearson Chi-Square p-value of 0.13 exceeds conventional
0.10 level and we cannot reject null hypothesis about homogeneity of means for male and female.
Listening_DiBELS * dummy 1-f and 2-m Crosstabulation

dummy 1-f and 2-m
1 2 Total
Listening_DiBEL 1 Count 108 110 218

S % within
49.5% 50.5% 100.0%
Listening_DiBELS
2 Count 173 199 372
% within
46.5% 53.5% 100.0%
Listening_DiBELS
3 Count 83 124 207
% within
40.1% 59.9% 100.0%
Listening_DiBELS
% within
45.7% 54.3% 100.0%
Listening_DiBELS
Chi-Square Tests
Asymp. Sig.
Value df (2-sided)
Linear-by-Linear
3.781 1 .052
Association
Dictation by gender
Finally we performed sample mean test for dictation score. Accordingly we did not find statistically significant differences between
male and female regarding dictation. Both groups display similar performance regarding dictation.
Group Statistics

dumm
y 1-f
DicSC_total 1 364 13.8687 3.80664 .19963
2 433 14.0300 3.72643 .17912

95% Confidence Interva
Mean Std. Error Difference
F Sig. t df Sig. (2-tailed) Difference Difference Lower Upper
DicSC_total Equal
variances .100 .751 -.602 794 .547 -.16126 .26771 -.68676 .36424
assumed
Equal
variances not -.601 765.099 .548 -.16126 .26821 -.68776 .36525
assumed
View publication stats

Reading Study Report 2014 Final1

Uploaded by

Copyright:

Available Formats

You might also like

Reading Study Report 2014 Final1

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Reading Study Report 2014 Final1

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Progress Report on the Reading Skills of Grade 2 students in Kosovo Primary

Technical Report · September 2014

Formal and informal Institutional activities View project

WBC-INCO View project

The user has requested enhancement of the downloaded file.

By Besnik Krazniqi and Keith Prenton

Section Title Page

3 Design Of Research Study 7

4 Design Of EGRA Instrument 8

7 Analysis & Results 15

9 Conclusions And Recommendations 20

Appendix A Basic Education Program Strategy for Improving 21

Besnik Krasniqi.and Keith Prenton, September 2014

This report is based on two research studies using the A-EGRA.

Besnik Krasniqi.and Keith Prenton, September 2014 Page 3

Besnik Krasniqi.and Keith Prenton, September 2014 Page 4

2.2 Monitoring and evaluating the interventions

This report is based on two research studies:

1 Downloadable from: http://transition.usaid.gov/our_work/education_and_universities/documents/

Besnik Krasniqi.and Keith Prenton, September 2014 Page 5

2.3 Rationale for an Albanian language early grade reading assessment.

Besnik Krasniqi.and Keith Prenton, September 2014 Page 6

3. DESIGN OF RESEARCH STUDY

Besnik Krasniqi.and Keith Prenton, September 2014 Page 7

4. DESIGN OF EGRA INSTRUMENT

4.2 Albanian Adaptation of EGRA

2 Downloadable from: https://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&id=188

Besnik Krasniqi.and Keith Prenton, September 2014 Page 8

4.3 Description of Instrument

Besnik Krasniqi.and Keith Prenton, September 2014 Page 9

ii) Letter name/sound knowledge

iii) Familiar word reading

iv) Unfamiliar word reading

v) Oral reading fluency (5a) and Reading comprehension (5b)

vi) Listening comprehension

Besnik Krasniqi.and Keith Prenton, September 2014 Page 10

Testing one child takes about 10-15 minutes.

The physical test package consists of three parts:

Besnik Krasniqi.and Keith Prenton, September 2014 Page 11

5.2 Data Gathering

5.3 Quality Assurance – by BEP staff

5.4 Outcome of Data Gathering

Besnik Krasniqi.and Keith Prenton, September 2014 Page 12

6.2 Sampling Weights

Where N is the total number of schools in the stratum.

Besnik Krasniqi.and Keith Prenton, September 2014 Page 13

6.3 Computation of Intermediate Variables

sub-test 1: “Correct words” was computed as: (words attempted – total-errors).

Besnik Krasniqi.and Keith Prenton, September 2014 Page 14

7 ANALYSIS & RESULTS

The A-EGRA is designed to be criterion-referenced (i.e., assessing performance against defined

 Basic skills (phonemic awareness and letter name/sound knowledge)

7.2 Basic Skills

Besnik Krasniqi.and Keith Prenton, September 2014 Page 15

b) Letter Name/Sound Knowledge

Letter name/sound knowledge is a fundamental and very basic measure of reading

7.3 Sight Vocabulary and Decoding