Professional Documents
Culture Documents
Report 1
Report 1
Teaching 21st century learners requires teachers to regularly evaluate their teaching
methods and testing modalities to ensure that we are nurturing the skills kids will need to be
successful in this quickly advancing world. In my own teaching practice, I have contemplated the
best mode of administering assessments in order to meet the needs of all my learners. The
specific question that I have aimed to address in my study is: Is student achievement greater
when assessments are administered digitally or paper copies? Historically, assessments have
been administered to students on paper and pencil. More often lately you see assessments, both
high-stakes and low stakes, administered digitally. I wanted to find out whether students were
more successful using a digital testing platform versus traditional paper and pencil tests. To
experiment with this question, I tested 12 of my fifth-grade students over the course of a grade-
level reading unit. I assigned assessments to a treatment group and a control group in two
opposite formats- digital and paper copies. The assessments ranged from low stakes to high
stakes in order to collect a range of assessment type data. I used this data in addition to
observations, interviews and numerous scholarly articles relating to this topic to formulate my
Review of Literature
Before conducting my own research, I spent some time reviewing literature that explored
the findings of several other studies surrounding the topic of testing modalities. These studies
helped me to make more informed decisions about the way I would execute my own experiment
Performance in Paper-Based versus Computer-Based Testing that among the many studies in
which researchers compare student performance outcomes in online courses versus traditional
courses, there was largely found to be no significant difference in student performance. This
article investigates how student performance differs on an accounting exam between paper-based
testing and computer-based testing. Anakwe points out that her study differs from previous
studies in that her focus is solely addressing the testing methods of traditionally taught courses
rather than the course as a whole. The study revealed that across three cohorts of accounting
students, there was no notable difference in test scores between online and traditional testing
methods (Anakwe, 2008). When I first began thinking and planning my own research, I was
confidently anticipating finding more student success in paper testing. Upon reviewing this first
piece of literature, my thinking began to shift towards a more open-minded expectation for what
The next article I read was from a study that was presented at the Annual International
three different types of computer-based tests. The computer-based tests varied in their navigation
and answer selection features. The study was conducted using a sample of 9,698 fourth through
twelfth grade students, all of whom were enrolled in a science course. Cari F. Herrmann-Abell,
Paper-Based Science Assessments that paper-based testing and computer-based testing produce
equivalent results. In addition to these findings, the researchers also found that students are more
successful when the computer-based test allows test-takers to directly click on an answer choice
and that there is no impact on free versus restricted navigation. When surveyed, elementary and
middle school age students largely prefer computer-based tests over paper-based tests
(Herrmann-Abell, et al., 2018). I found it interesting that again, there seemed to be so little
disparity between paper-based testing and computer-based testing results. I also never considered
the different factors of an online test that could impact student success. This article inspired me
to incorporate a more quantitative piece into my research in which I surveyed kids on their
Another study that inspired me to consider additional factors and implement more
qualitative data into my research was one conducted at the University of Florence. This study
consisted of 606 students who were analyzed while using their preferred method of testing (paper
or digital). Those who selected computer-based testing used their own personal devices and were
asked to participate in a survey where they rated their perceived self-efficacy and satisfaction
when testing digitally. In their study, Andrea Nardi and Maria Ranieri explored the potential for
relationship between computer-based testing and student self-efficacy and satisfaction. The
results of this study found that students performed better on computer-based tests and that a
positive relationship exists between students and their recognition of self-efficacy and
satisfaction when taking digital tests (Nardi and Ranieri, 2019). Before reading this article, I’d
never considered the idea of self-efficacy or positive and negative relationships among students
and testing modes. This theory led me to survey my students beyond just their preference of one
mode over the other and to consider the factors that led them to their selection. In order to gather
this qualitative data, I added a section to my student survey in which students had the
opportunity to elaborate on their preferred testing method and explain their decision-making
process. These raw thoughts and opinions from students proved to be the most insightful
paper testing versus digital testing was being researched, there were always additional factors or
conditions being considered. The research is no longer simply “apples versus oranges”, but
rather an analysis of which conditions contribute to the best development of said “apples” or
“oranges”. Roy Clariana and Patricia Wallace conducted a study comparing different factors
associated with paper-based testing and computer-based testing. The factors considered were
content familiarity, computer familiarity, competitiveness, and gender. This study was conducted
students were post-tested digitally, and 51 students were selected as the paper-tested group. This
study found that computer-based tests positively impacted student scores. When identifying
found between testing modes. Alternatively, there was a relationship found among content
familiarity and testing mode (Clariana and Wallace, 2002). It was concluded that higher attaining
students performed better on computer-based tests. At this point in my review of the literature
surrounding this topic, I realized that it may be more beneficial for my research purposes that I
rely more on the qualitative data than the number crunching and score analyzing.
Muhammet Yasar Özden, Ismail Ertürk, and Refik Sanli conducted a study on students’
perceptions of online assessments. The purpose of this study was to gather data on what students
based testing. Data was collected using a paper-based survey and conducting of student
interviews. The study found that a majority of the students who were surveyed noted key features
such as immediate feedback, randomized question order, item analysis of the questions, and
obtaining the scores immediately after the exam (Ozden, et al., 2004). The researchers agreed
that these features contributed to positive achievement on exams. After reading this article, I was
curious to hear perceptions from my elementary age students of online assessments and the
factors that contribute to positive achievement and success on exams for their age group. I chose
to leave my survey question open ended rather offering them selections in order to allow for
authentic responses of the factors that impact their experience with online testing.
After reading each of these articles and reviewing the findings of many researchers, I was
surprised to see how little difference was recorded when comparing paper-based testing to
computer based testing. It seems that the comparison is no longer one mode versus the other, but
rather the conditions of each respective mode that contribute to more positive achievement. After
some reflection, my perception of my own research question has changed drastically. I was
almost certain that I was going to find more clear success in one mode versus the other. I now
understand the importance of the qualitative data that will come from my study as well as the
different factors I should consider when grouping and surveying students such as preference, test
Methodology
Once I reviewed the literature surrounding my topic, I felt more prepared to formulate my
own assessment instrument in order to collect both quantitative and qualitative data. I planned to
In order to collect quantitative data, I planned to focus my study on the district-wide unit
collection assessments that are administered at the beginning and end of each English Language
Arts unit. The test is typically 10-15 multiple choice questions based on lengthy reading
passages. The digital version is implemented as a google form where the passages are viewed,
and answers are selected on their screen. The paper version is a traditional document where
students circle their multiple-choice answer. On the day of the pre-test I randomly selected six
students for a heterogeneous treatment group and six students for a heterogeneous control group.
For the treatment group, I administered the fully digital assessment and for the control group I
administered the traditional paper and pencil version. On the day of the post-test, at the end of
the unit, I again administered the digital version to the treatment group and the paper version to
the control group. I chose these assessment instruments and stuck with the same mode of
administering each time in order to see which group would perform better, and also to compare
the results in both low-stakes (pre-test) and high-stakes (post-test) testing environments.
In order to collect qualitative data, I administered a student survey in which they shared
their preferences and perceptions of online tests versus traditional paper tests in both high-stakes
and low-stakes environments. The survey was a simple Google form made up of four questions
where students select either “paper” or “digital” preference, one question gauging student effort
with a “pre-test”, “post-test” or “equal” selection, and one open ended question that allowed for
students to explain their digital or paper testing preferences and perceptions. The questions were
as follows:
2. Is it easier to go back and check your answers when you test digitally or on paper?
4. Do you find it easier to read long texts when they are given digitally or on paper?
6. Explain why you prefer digital or paper Collection Pre/Posttests better. Give 3
wrong answers, there is no grade associated with their responses, but rather that I just wanted to
know their thoughts and feelings about the way they display their learning on tests. I made sure
they understood that nobody would see their answers except me, so they had the freedom to be
Analysis
Once the course of my research was complete, I began to analyze the results. I began first
with my qualitative data. I reviewed the results of the student survey to gain an understanding of
my students’ perceptions of online versus virtual testing as well as their narratives of why they
prefer one mode over the other. Summaries of the results are below:
I organized my quantitative data into two charts. The first chart represents the highest,
lowest and average scores from the Pre-Test for both digital and paper responses. The second
chart represents the highest, lowest and average scores from the Post-Test for paper and digital
responses as well. I thought that these three data points would give the best overview of the
results from each respective testing modality. The charts are presented below:
Once my data was compiled, I took a moment to reflect on the validity of my data. I
contemplated whether the effects of the nature of this school year had any impact on the
outcomes of my research. Due to Covid-19 protocols, this school year has been unlike any other.
Did a year of digital learning positively or negatively impact my students’ views of digital
testing? Would my results have been different had this school year been more traditional? I
concluded that there was no true way for me to weigh the impact of the pandemic on my
Findings
After much reflection and analysis of the data produced from my research, I was
surprised to see such little disparity among the quantitative data. Though my own expectations
were not met, the quantitative results did align well with the literature I reviewed previously. As
seen in the charts above, class averages between digital and paper testing only revealed a 10-11-
point difference. As a whole, students only achieved slightly more success when testing digitally
compared to on paper- not vastly different. When comparing the Pre-Test averages to the Post-
Test averages, there is a much clearer disparity between high-stakes testing results and low-
In addition to the quantitative data revealing little disparity, the surveys of student
perceptions of digital testing versus paper testing revealed a fairly moderate and balanced
response as well. When directly asked their preference, approximately half of the students chose
digital and half chose paper. It was interesting though, that almost 70% of students selected that
they feel paper tests are more important than digital tests. I attribute this response to the fact that
students reported that they put equal effort into collection pre-tests (low stakes) as they do post-
The most interesting aspect of this research study was the responses I received when
asking students to explain and elaborate on why they prefer one testing mode over the other.
Many students cited the ability to focus, internet connectivity issues, ease of reading and ability
to make notes as reasons which would support paper-testing. In support of digital testing, kids
explained that they preferred typing rather than writing, digital navigation rather than flipping
pages and immediate feedback at the completion of a digital test. I found the qualitative data to
be the most insightful to me as a teacher. Moving forward, I plan to allow my students to choose
their preferred method of testing when taking Collection Pre and Posttests. I am curious to find
out whether this will impact student success in the future. I look forward to continuing this
Clariana, R., & Wallace, P. (2002). Paper-Based versus Computer-Based Assessment: Key
Factors Associated with the Test Mode Effect. British Journal of Educational
Nardi, A., & Ranieri, M. (2019). Comparing Paper-Based and Electronic Multiple-Choice
Ozden, M. Y., Erturk, I., & Sanli, R. (2004). Students’ Perceptions of Online Assessment: A