Digital School Libraries and Student Performance
Digital School Libraries and
Student Performance: An
Evaluation of the Digital
Collections at High Schools in
Pennsylvania and New Jersey
June 7, 2010
©Gina Cacace , Graduate Student at Drexel
University’s iSchool, School Library Media program
gdc27@drexel.edu
Paper produced for course INFO 653, Digital
Libraries with Professor Xia Lin .
Keywords: digital libraries, education, school library
media, school library, digital collection, student
performance, digital literacy
Abstract
This quantitative study extends the breadth of
research that shows a strong relationship
between traditional school library quality and
student performance (Michie & Chaney, 2005;
Lance et al., 2000). Since the way that students
learn is changing, the ways that libraries can
improve that learning is also subject to change
(Roes, 2001). I will try to determine whether in
an age of “digital learners,” schools with higher
quality digital collections will have better
performing students.
Thirty digital collections of high schools in
Pennsylvania and New Jersey were evaluated
for their collection size, quality, currency, scope,
content organization, navigation, browsing and
searching capabilities. A Spearman rank
correlation of -.6273 was calculated which
shows within a .01 probability (n=30, p=.01,
critical value .478) that schools with better
digital collections have better performing
students.
2010
Introduction
The new generation of students are “digital
learners” (Brown, 2002), with more than 78% of
children between the ages of 12 and 17 going
online everyday (Arafeh et al., 2002). It is well
established that most high school students
utilize the Internet to do school work (Lorenzen,
2001; Arafeh et al., 2002).
The Pew Internet & American Life Project Poll
found that 71% of high school students
reported to have used only Internet resources
for their last school report. In contrast, only
24% said that they used books or magazines
from the library. It is clear that school libraries
need to adapt in order to remain relevant in a
digital world.
What is most interesting about the work
coming from the Pew Internet & American Life
Project is that through interviews with high
school students they found that the main
metaphor used by students to explain their use
of the Internet for school work is that the
Internet is a reference library:
“ Much like a school-issued textbook or a
traditional library, students think of the Internet
as the place to find primary and secondary
source materials for their reports, presentations
and projects” (Arafeh et al., 2002:5).
In fact, John Seely Brown predicts:
“the real literacy of tomorrow entails the ability
to be your own personal reference librarian-to
know how to navigate through confusing,
complex information spaces and feel
comfortable doing so” (Brown, 2002: 14).
1|Page
 Digital School Libraries and Student Performance 2010
In the coming era where the Internet is the new and evaluated all school districts that spent +/-
library and students are the new reference $1,000 per student. There were 26 school
librarians, where do traditional school libraries districts that fell into this category, with overall
and librarians fit? Many predict that school rankings ranging from 1 - 103 with a mean rank
libraries and librarians will provide access to of 41. Because some school districts have more
digital collections that organize information in than one high school, I evaluated a total of 30
ways that will help students find credible high school library digital collections.
sources (Dresang, 2005).
Philadelphia’s school district (with its 45
Prior research has only evaluated the circulation different public high schools and numerous
statistics and print collection quality of school charter schools) was excluded from the sample.
libraries to assess their value. This study posits The district ranked 104/105 and would have
that school libraries with high quality digital skewed the results since it contained more high
collections will provide their students with the schools than the rest of the districts combined.
resources necessary to navigate the digital
world. By providing access and credibility to The school districts and corresponding high
resources, the digital collections of school schools evaluated are listed in Table 1.
libraries can help improve student learning as School District High School
measured by test performance and overall Tredyffrin-Easttown Conestoga High School
School District
school ranking. Unionville-Chadds Ford Unionville High School
Methacton School District Methacton High School
Methods Hatboro-Horsham School Hatboro-Horsham High School
School ranking data was obtained from the District
Council Rock School Council Rock High School North
2008 report by Philadelphia Magazine District
(Sweeney, 2008). The rankings of 105 school Council Rock School Council Rock High School South
District
districts in Pennsylvania and New Jersey (8 total Wallingford-Swarthmore Strath Haven High School
counties) were based on the following criteria: Upper Dublin School Upper Dublin High School
District
Lower Moreland Township Lower Moreland High School
Standardized test scores (grade level
Garnet Valley School Garnet Valley High School
testing under the No Child Left Behind District
Act and SAT scores) Downington Area School Dowington High School East
District
Graduation rates Downington Area School Dowington High School West
District
Student-faculty ratio North Penn School District North Penn High School
Percentage of graduates intending to West Chester Area School West Chester Henderson High
enroll at two- and four-year colleges District School
West Chester Area School West Chester East High School
District
The results were weighted by the number of West Chester Area School West Chester Rustin High School
students enrolled (Sweeney, 2008). District
Abington School District Abington Senior High School

In order to control for the effect of budget on Kennett Consolidated Kennett High School
School District
library quality, I took the cost per student of the Quakertown Community Quakertown High School
School District
top high school ($14,222 annual cost per Pennsbury School District Pennsbury High School
student for Tredyffrin-Easttown School District)

2|Page
 Digital School Libraries and Student Performance 2010
Springfield School District Springfield High School or links to sites with questionable credibility.
Spring-Ford Area School Senior High School Currency was determined a few ways – dead
District
Pennridge School District Pennridge High School links hinted that the digital collection was not
Octorara Area School Octorara Area High School maintained. The copyright at the bottom of the
District
Upper Moreland School Upper Moreland High School
digital collection’s site also indicated whether it
District was updated recently. News feeds of new
Pottsgrove School District Pottsgrove High School
resources (Twitterfeeds, scrolling
Chichester School District Chichester High School
announcements, new book lists, etc.) received
Bristol Borough School Bristol High School
District high currency scores. Scope was scored on
Gloucester City Gloucester City High School whether or not the collections expanded
William Penn School Penn Wood High School outside of the general history/social studies
District
Table 1. Study Sample realm. Some schools only had resources for
particular classes while others had resources on
The official high school websites were found by a wide range of topics. Dresang’s (2005) meta-
accessing the Wikipedia page for each school analysis of the information seeking behavior of
district. Library website URLs were located by students found that students prefer browsing
browsing the main high school website. Twenty over planned searches. It is also well known
nine of the thirty high schools had dedicated that children (K-12) read in a non-linear
library websites with digital collections manner. Because of this, digital collections had
(characterized by electronic resources such as to be well organized in order to allow for
subscription databases, external links with discovery through clicking. The use of images
annotations organized by subject, and and other visual organizational methods were
pathfinders). Any school without a digital considered good. Lists of links were not
collection was given a rating of 0. considered user friendly and received lower
Digital collections were evaluated using a scores. Pathfinders organized by subject that
modified version of the digital repository review contained annotated links, on the other hand,
developed by Professor Xia Lin at Drexel received high scores. Many collections did not
University. The collections were evaluated for have formal search engines. If they did, a simple
their size, quality, currency, scope, content keyword test was performed to evaluate its
organization, navigation, browsing and effectiveness.
searching capabilities and ranked on scales of 0 The total number of points possible from the
– 3 and 0 – 4. Where features were unavailable evaluation was 29. See Table 2 for the
or inaccessible because of a secure log in a evaluation criteria and scoring system used.
score of 0 was given because studies show that
“hitting a wall” (or requiring a log in for access) All efforts were made to ensure that the
is a deterrent against using a service, especially evaluations of the digital collection were done
for students (Dresang, 2005). independently of the school’s rank. The rank
data provided by Philadelphia Magazine was
Collection size was determined by the number kept separate from the evaluation and URL
of resources, the quality was determined by information.
whether they were well known academic sites

3|Page
 Digital School Libraries and Student Performance 2010
Category Possible values for the rating determine whether the place each coordinate
Collection Size 0 Unknown
1 Small holds in its respective data set is related.
2 Medium
3 Large
4 Very large Digital
Quality 0 Unknown School Collection Difference
1 Poor Ranking Rank Evaluation Rank (d) d2
2 Mixed Score
3 Good
4 Excellent 1 1 24 26 -25 625
Currency 0 Unknown
1 Not-up-to-date 3 3 22 23.5 -20.5 420.25
2 Up-to-date 20 9 29 30 -21 441
3 Update-frequently 34 17 18 19 -2 4
Scope 0 Unknown
1 Various coverage 8 5.5 18 19 -13.5 182.25
2 Focused coverage
8 5.5 15 15 -9.5 90.25
3 Comprehensive coverage
Content Organization 0 Unknown 7 4 23 25 -21 441
1 Lack of organization
2 2 26 27 -25 625
2 Good organization
3 Excellent organization 13 8 27 28.5 -20.5 420.25
Navigation 0 Unable to access
1 Difficult-to-use 12 7 18 19 -12 144
2 Ok 21 10.5 9 7 3.5 12.25
3 Easy-to-use
4 Excellent 21 10.5 10 8.5 2 4
Browsing 0 Unable to access 36 18 19 21.5 -3.5 12.25
1 Difficult-to-use
2 Ok 27 14 22 23.5 -9.5 90.25
3 Easy-to-use 27 14 19 21.5 -7.5 56.25
4 Excellent
Searching 0 Unable to access 27 14 0 1 13 169
1 Difficult-to-use 24 12 27 28.5 -16.5 272.25
2 Ok
3 Easy-to-use 49 21 17 17 4 16
4 Excellent
66 24 7 4 20 400
Table 2. Evaluation Criteria for Digital Collections
28 16 14 13 3 9

Results 46 19 7 4 15 225

Figure 1 graphs the two sets of data: the 47 20 12 12 8 64

schools ranking on a scale of 1-103 (1 being 54 22 7 4 18 324

good, 103 being poor) and the evaluation (0 77 27 10 8.5 18.5 342.25
59 23 11 10.5 12.5 156.25
being bad and 29 being excellent). Figure 1
74 26 15 15 11 121
shows that the data followed a general negative
99 29 5 2 27 729
trend, where schools with better student
102 30 15 15 15 225
performance (lower ranks) tended to have
67 25 11 10.5 14.5 210.25
higher quality digital collections.
91 28 8 6 22 484

In order to better test the relationship between ∑d2 7315

the data points, a Spearman’s rank correlation Table 3. Spearman’s Rank Correlation Data
was calculated (see Table 3). This compares the
relative rank of each data set (x = school rank,
y= digital collection evaluation score) to

4|Page
 Digital School Libraries and Student Performance 2010

School Rank and School Library Digital

Collection Quality for 30 High Schools in
Pennsylvania and New Jersey
35
Digital Collection Quality (out of 29)

30

25

20

15

10

0
0 20 40 60 80 100 120
School Rank as measured by Philadelphia Magazine (out of 103)

Figure 1. School Rank versus School Library Digital Collection Quality

The calculation for rho (ρ) shown in Table 4

yields a Spearman rank correlation of -.6273. A
negative correlation means that the data acts in
opposition. Such that y (the digital collection
quality) decreases when x (the school’s rank)
increases. This occurs because the school rank Table 4. Spearman’s Rank Calculations
data is set so that a higher number is a lower
rank, or lower student performance.

Based on an adjusted critical value of .478 collection and student performance. The results
(n=30, p=.01)1, the Spearman rank correlation show that as the quality of the school library’s
found in this study is within the .01 level of digital collection increased, so did the student’s
significance. performance and the school’s overall ranking.
Since we controlled for the expenditures per
Discussion student, we know these results are not due to
This study found a significant correlation budgetary discrepancies. Instead, it seems that
between the quality of a school library’s digital the schools that invest in digital resources for
1
their students are rewarded with better
This website was used to determine the critical
performing students and a higher overall school
values for a small sample:
http://www.sussex.ac.uk/Users/grahamh/RM1web/ rank. This confirms our suspicions that students
Rhotable.htm

5|Page
 Digital School Libraries and Student Performance
growing up in the digital age require different
resources than earlier cohorts of students.
The results of this study speak volumes for the
Digital Library community. We now know that
student performance is linked to the quality of
digital libraries. Educators should take this very
seriously and ensure that adequate funding is
provided to its school library media centers to
develop a digital counterpart to their traditional
brick and mortar collections.
Study Limitations
The design of this study is not without its
limitations. First, the use of the Philadelphia
Magazine rankings to capture student
performance was not the best data set, as the
raw data used to compile the rankings was not
made available. According to Sweeney (2008),
the rankings of schools were accurate within +/-
10 places.
In addition, the evaluation system used to score
the digital collections has not been standardized
or tested for its accuracy. It would have been
desirable to have the collections independently
scored by different evaluators in order to
compare the ways in which the tool was used.
Further Research
A study this small that yields a statistically
significant correlation should be duplicated in
other areas with larger sample sizes to ensure
that the relationships found here translate
outside of the Pennsylvania and New Jersey
area. Further inquiry should not just evaluate
the quality of a digital collection but its actual
use to determine whether that affects student
performance. Knowing what digital resources
students are using and why will help us better
design digital libraries.
2010
Since Philadelphia’s school district contained
more high schools (45) than the entire sample
size (30), it was not included in this research.
Further investigation into the relationship
between digital collections and student
performance in Philadelphia’s high schools
would be a valuable set of data to compare to
the work done in this study.
Conclusion
This study was based upon prior research that
shows that the quality of a school library’s
collection is related to student performance
(Michie & Chaney, 2005; Lance et al., 2000).
Since students are now digital learners, it was
hypothesized that the relationship between
students and the school library would change.
This investigation has shown that in
Pennsylvania and New Jersey, schools with
higher quality digital collections made available
through their school libraries had higher
performing students.
References
Arafeh, S., Levin, D. & Rainie, L. (2002). The digital
disconnect: the widening gap between Internet-savvy
students and their schools. Pew Internet & American Life
Project. Available:
http://pewinternet.org/Reports/2002/The-Digital-
Disconnect-The-widening-gap-between-Internetsavvy-
students-and-their-schools.aspx
Aslanidou, S. & Menexes, G. (2008). Youth and the
Internet: Uses and practices in the home. Computers and
Education, 51 (2008), pp. 1375-1391.
Ben-David Kolikant, Y. (2009). Digital students in a book
oriented school: Student’s perceptions of school and the
usability of digital technology in schools. Educational
Technology & Society, 12 (2), pp. 131 – 143.
Brown, J.S. (2002). Growing up digital: How the web
changes work, education and the ways people learn.
United States Distance Learning Association Journal, 16
(2). Available:
6|Page
 Digital School Libraries and Student Performance 2010
http://www.usdla.org/html/journal/FEB02_Issue/article01
.html

Dresang, E.T. (2005). The information seeking behavior of

youth in the digital environment. Library Trends, 54 (2),
pp. 178-196.

Dresang, E.T. & Koh, K. (2009). Radical change theory,

youth information behavior and school libraries. Library
Trends, 58 (1), pp. 26-50.

Fullerton, K., Greenberg, J., McClure M., Rasmussen, E., &

Stewart, D. (1999). A digital library for education: the PEN-
DOR project. The Electronic Library, 17 (2), pp. 75-82.

Lance, K.C, Rodney, M. J., & Hamilton-Pennell, C. (2000).

Measuring Up to Standards: The Impact of School Library
Programs & Information Literacy in Pennsylvania Schools.
Greensburg, Pa.: Pennsylvania Citizens for Better Libraries.

Lorenzen, M. (2002). The land of confusion? High school

students and their use of the World Wide Web for
research. Research Strategies, 18 (2001), pp. 151 – 163.

Masullo, M. & Mack, R. (1996). Roles for Digital Libraries in

K-12 Education. D-Lib Magazine,
hdl://cnri.dlib/september96-masullo

Michie, J.S. & Chaney, B.W. (2005). Evaluation of the

Improving Literacy Through School Libraries Program.
Rockville, MD : US Department of Education.

Roes, H. (2001). Digital libraries and education: Trends and

opportunities. D-Lib Magazine, 7 (7/8), doi
10.1045/july2001-roes

Strom, P., Strom, R. , Wing, C., & Beckert, T. (2009).

Adolescent learning and the Internet: Implications for
school leadership and student engagement in learning.
NASSP Bulletin, 93 (2), pp. 111 – 121.

Sweeney, B. (August 8, 2008). The top 50 school districts of

2008. Philadelphia Magazine. Available:
http://www.phillymag.com/articles/the_top_50_school_di
stricts_2008/

7|Page