Updated Research Paper

ANALYZING THE SCORE DIFFERENCE BETWEEN ON-PAPER AND ONLINE SAT EXAMS 1
CAUSED BY IN-PERSON SAT CLASSES IN A LARGE, PUBLIC, NEW YORK HIGH SCHOOL
Analyzing the Score Difference Between On-Paper and Online SAT Exams Caused by
In-Person SAT Classes In a Large, Public, New York High School
AP Research
Word Count: 5496

Abstract
Prior research has examined the link between in-person extracurricular classes and
performance on paper SATs. As the SAT shifts to an online format, questions about its
standardized nature and the relationship between in-person, extracurricular test preparation and
score increases on online exams. As a result, this research sought to study how in-person
extracurricular classes impact student score increases on online SATs and the difference between
scores on online and paper SATs.
To analyze this, students took an initial benchmark, were allowed to attend 3, 45 minute
Math and English classes, and then took an online and paper culminating exams. Statistical tests
were drawn between attendance and improvement, the online culminating exam and the initial
benchmark exam, and the online and paper culminating exams. Additionally, qualitative
interviews were conducted to collect data.
It was concluded that there was no relationship between attendance and score
improvement. Additionally, it was noticed that students performed better on online exams than
on paper exams. It also can be concluded that the connections made in prior research (i.e. greater
HSGPA indicates success on the SAT, higher math class averages relates to more improvement
on the math section of the SAT, etc.) can be extended to online exams.
Introduction
For millions of American students, entrance into colleges and universities across the
nation rests on a variety of factors, one of which is their performance on a standardized test.
Here, a standardized test refers to a private corporation’s college entrance exam which has been
nationally adopted. For most students seeking higher education, colleges limit them to two
testing options: the SAT and the ACT. Of these two exams, the SAT has proven to be the most
popular. In fact, according to the College Board, “Over 2.2 million students in the class of 2019
took the SAT…Over 8 million students took a test from the SAT suite during the 2018-2019
school year…,” (College Board, 2019). Comparatively, according to the ACT, Inc., about 1.78
million students took the ACT in 2019, 52% of the 2019 graduating class.
The standardized testing industry has been an incredibly stable portion of the college
admissions process, cementing itself as the first indicator of college readiness - if a student does
not meet the minimum score requirement of a higher education institution, their chances of
entering that school decreases significantly (Robinson & Monks, 2005, p. 4). The industry has
also been incredibly profitable; in the fiscal year 2019, the College Board made over $1.1 billion
in profit (Internal Revenue Service, n.d., Part 1, Line 12). The emergence of these standardized
entrance exams, and their genuine importance to college acceptance, resulted in the formation of
an entire test preparation sub industry. This test preparation industry is also exceptionally
valuable, earning about another $1 billion, according to Moss et al., who discusses the rise of an
expensive test preparation industry centered around wealthy families (Moss et al., 2012, p. 17).
Despite the stability of this industry, students that want to take the SAT can expect
enormous changes in the coming years. On January 25th, the College Board confirmed that the
SAT would be going digital, would be shortened to two hours, would allow calculators on all of
the math, and would only consider one question per reading passage (College Board, 2022).
Additionally, there will be more unique tests and scores will be returned in days rather than
weeks (College Board, 2022). This new SAT could have large implications for students,
including accessibility to tests and new aspects of or approaches to college admissions.
Gap in the Research
Pre-existing research indicates that there is a link between in-person college test
preparation and some sort of exam score change, including score increases or no change
(Donaldson, 2013; Moss et al., 2012; Ringer, 2014). However, existing research does not contain
a significant amount of information regarding the connection between in-person SAT classes and
score improvements on an online examination. Additionally, previous research similar to this
study has been conducted on the ACT, but similar research on the SAT has not been previously
conducted (Moss et al., 2012). Thus, a second gap of this research is to address the SAT
specifically. Therefore, the purpose of this research is to measure the effectiveness of in-person
SAT classes on an online SAT examination, in relation to a paper exam. Research regarding this
information is especially important for the test preparation industry and for the admission of
millions of students that will take exams in the following years. In an effort to analyze the score
difference between a paper SAT and on online SATs, the guiding research question is: How do
in-person SAT classes impact the outcome of students on online SATs when compared to
on-paper SATs in a large, public New York highschool?

Literature Review
Search Strategies
All sources were pulled from a variety of databases, many of which related in some form
to the field of education and statistical research. The most prevalent databases used include
ProQuest Multiple Databases, ERIC, ScienceDirect, and Google Scholar. To limit the scope of
the sources used, keywords included SAT, Scholastic Aptitude Test 1, test preparation, college
entrance exams, financial aid, and test optional.
College Entrance Exams As Measures of Student Academic Merit
There is a significant and ongoing debate regarding the use of college entrance exams as
a measurement of student academic ability. Furthermore, that debate expands to include racial,
gender, and socioeconomic inequalities when student academic ability is used to provide
merit-based financial student aid. To limit the scope of this study’s collected data and
conclusions, only the economic lens will be fully developed in this portion of the literature
review.
For universities, comparing exam scores between students and carefully controlling the
average exam score of the institution provides them with a higher national ranking, impacting the
quality and number of applicants that the college or university may receive (Appelrouth &
Zabrucky, 2017). By managing the average college entrance exam score that their institution
offers, schools may seek to intake students of a high academic ability. By accepting students
based on their academic ability, as measured by their SAT or ACT scores, schools have
1
As of 1990, SAT no longer stands for the Scholastic Aptitude Test. This keyword was only used because prior
research often refers to the SAT by its previous name. Today, SAT does not refer to an acronym, and is merely the
name of the examination.
essentially equated academic merit to entrance exam performance. In fact, according to
Appelrouth and Zabrucky, 88% of four year higher education institutions place a moderate or
considerable importance upon these scores; the SAT and ACT take important roles as
measurements of academic merit. To institutions, academic ability represents a desirable trait that
should be recruited, specifically through merit-based financial aid (Monks, 2009, p. 3). Monks
also reaffirm that merit awards are only offered to those students that are identified as most
desirable by the institution, further underpinning the importance of standardized entrance exams
to individual students. The size of these merit-based scholarships are not insignificant. More
importantly, however, they are often tied to performance on the SAT or ACT. For example, a
1340 on the SAT at the University of Alabama would qualify a student for $4,000 per year while
a SAT score of 1450 would give students $26,950 per year - free tuition (Appelrouth &
Zabrucky, 2017, p. 3). An increase of 110 points means that students would save roughly
$90,000 more, illustrating the significance of standardized testing performance to scholarships.
Immediately, the connection between student merit and college entrance exams, as well as the
fiscal consequences becomes clear.
When combined, Monks (2009) and Applerouth and Zabrucky (2007) indicate a clear
connection between the usage of the SAT and ACT as measurements of student academic ability.
Furthermore, these sources connect the use of these tests as measurements of student ability to
justify large sums of merit-based financial aid. Together, these sources provide a deeper
contextualization of the use of the SAT and its importance in the field of college admissions.
Additionally, these two sources provide the precedent for the analysis of student academic
tracking and score improvement.

Data Collected in Prior Research
Data (factors) collected in prior areas of research have significant importance to this
study. Because the study is similar to other studies previously conducted in this field of expertise
(e.g., Moss et al., 2012), there is certain data collected in prior research that should also be
collected in this research.
One variable that must be collected because of its crucial role as an estimator of SAT
score and an important determining factor in the realm of college admissions is high school GPA
(HSGPA). According to Mattern et al. (2011), there are significant discrepancies between SAT
score and HSGPA. This discrepancy takes many forms, but can be simplified to a single
conclusion: “the discrepancy...increasingly overpredicts…[first year college GPA]...as the
discrepancy between a student’s SAT and HSGPA performance becomes larger in the direction
of higher HSGPA,” (Mattern et al., 2011, p. 653). Furthermore, there is evidence of a statistically
significant correlation between HSGPAs and SAT/ACT scores at the .01 level (Donaldson, 2013,
p. 59). In an effort to analyze both the discrepancy and the correlation between HSGPA and SAT
scores in the context of a digitally administered SAT, it would be appropriate to collect HSGPA.
The collection of this data is further explained in Methods (pg. 13). Important correlations have
also been drawn among other factors in prior research. The most common of these factors is
socioeconomic status, but also includes performance in school math courses and involvement in
extracurricular activity (Briggs, 2001, p. 11). For students with strong socioeconomic
backgrounds who perform well in their high school math classes and are involved in
extracurricular activities, coaching in the math section proves to be the most effective.
In summary, HSGPA, socioeconomic status, performance in math classes specifically,
and extracurricular involvement are staple factors of this field of research, and should be
collected and discussed in their appropriate sections (refer to Method [pg. 13] and Appendix G).
The Impact of Test Preparation on Student SAT Scores
Research on the impact of test preparation on student SAT scores is abundant, but
generally has no clear conclusion. Researchers in this field have slightly differed their variables
from study to study, or have attempted to replicate other studies in different regions; these
research attempts often have significantly different outcomes. As a result, there is not a single
clear conclusion of the impact of test preparation classes on SAT scores. Despite the lack of a
decisive conclusion among past research, prior conclusions can be divided into two branches:
test preparation has no impact, and test preparation does have an impact.
In 2013, educational researchers at a private Virginia university conducted a survey of
696 college freshmen to establish what form, if any, of standardized test preparation they had
taken and the impact it had on standardized entrance exam scores. After collecting information
regarding the types of test preparation used, including but not limited to “online resources,
self-help resources, school workshops, and/or private tutoring…,”, three different mean
regression samples were used (Donaldson, 2013). The initial mean regression sample involved
taking the overall mean SAT scores of the students and comparing them against the mean score
of no preparation. Second, the mean score of each testing method was compared to the score of
no preparation. Finally, in an independent part of the study, correlations were calculated between
high school GPAs and individual SAT scores. The study draws two conclusions. First,
preparation methods did not yield significant increases in SAT or ACT scores. The second
conclusion is that entrance exam scores are correlated positively with high school GPA. To
explain this second conclusion, Donaldson (2013) claims that a student with a high GPA should
have the skillset and the knowledge to outperform their peers on college entrance exams.
On the other hand, there is evidence to indicate that test preparation methods can provide
students with score increases. The 52 students that participated in this study were provided with a
private tutoring course for the ACT, spanning 20 hours, multiple classes, various practice ACTs,
and culminated in the October 2011 ACT. After participating in the 20 hour ACT preparation
course, the study determined that the course resulted in a 0.65 point improvement. This study
states that although the increase may seem miniscule, “... raising the scores of a group by this
amount using a short-term preparation class represents a significant gain. These results indicate
that preparation can increase ACT scores,” (Moss et al., 2012, p. 23).
Alongside Moss et al. (2012), Ringer (2014) inspected the link between a different test
preparation program, the Cambridge Test Preparation Program, and the SAT scores of students in
a local school district. 1,233 juniors and seniors were volunteers in this program, which offered
them “... one section of SAT…classes per semester… two official retired SAT tests, skills review,
powerful test-taking strategies, and practice tests,” in addition to well trained teachers and
specific subscores (Ringer, 2014, p. 5). By measuring the growth of SAT scores during the
Cambridge implementation from 2005-2010, Ringer was able to conclude that the Cambridge
Program showed a significant increase in verbal and math SAT scores.
Here, the preexisting research shows a variety of different opinions regarding the
connecting between exam preparation classes and SAT scores. The difference in opinions
indicates that it would be appropriate to conduct further research regarding the effectiveness of
in-person, extracurricular preparatory classes on SAT outcomes.
Summary
Crucial research in the field of college entrance exam research has established that the
SAT measures student academic ability and academic merit, established the variables crucial to
the development of a thorough and well-constructed research method, and established the open
interpretation of test preparation and its correlation to SAT score (e.g. Appelrouth & Zabrucky,
2017; Mattern et al., 2011; Ringer, 2014).
Remaining in line with the traditions of prior research and filling the current gap in
research regarding online SATs, this study will attempt to quantify the impact in-person SAT
classes have on the outcomes of online SATs, when compared to on-paper SATs using the
different factors necessary to draw valid conclusions.
Research Design and Method
Study Design
The goal of this research study is to determine the effectiveness of in-person SAT classes
on online SATs when compared to on-paper SATs in a large, public New York highschool. The
goal of this research is to examine the effectiveness of current test preparation methods,
specifically in-person, extracurricular classes, on the newly transformed SAT. This research is
vitally important because it provides new research on in-person, extracurricular classes on the
new SAT specifically.

A three part, quantitative and qualitative mixed content analysis was conducted. This
approach allowed for the quantitative and qualitative analysis of the effectiveness of in-person
SAT classes on the online SAT. In order to determine the effectiveness of in-person SAT
preparation and its outcome on online exams when compared to on-paper exams, three different
experimental techniques were employed: statistical regression (t-test) between all examinations,
analysis of variance (ANOVA) test between attendance and score improvement, and qualitative
interviews with subjects (Figure 1). The addition of the qualitative analysis provides a new spin
on a previously limited and underrepresented way to analyze effectiveness, firmly positioning
this research within a gap. The addition of these qualitative interviews also provides a personal
narrative component to the research, therefore providing new access to details regarding the
effectiveness of individual SAT preparation (Powers, 1988, p. 15). Additionally, although
researchers have calculated statistical regression examples between benchmark and cumulative
on-paper exams, there have not been any regressions calculated between the benchmark and
culminating online exams. Thus, the addition of a new quantitative measurement and the
expansions of previous qualitative and quantitative tools mentioned in the literature review
emphasizes the scope and importance of this experimental design.

Figure 1: Illustrative alignment of study design with Research Question and Gap Positioning
Subjects
The subjects of this research are all high school students that have been previously
exposed to the SAT sphere of products. These individuals included all sophomores and juniors in
this large, public New York highschool. This demographic was chosen because not only have
they been previously exposed to the SAT or the PSAT/NMSQT, but also because they are the
primary stakeholders in the greater realm of SAT usage. Juniors and sophomores are the most
likely to take the SAT and have taken 1 or 2 PSATs/NMSQTs and are also the most likely
students to take an exam preparation class. As the primary stakeholders, they are the most
appropriate subjects. Potential subjects were collected through the distribution of research
summarizing informational sheets through this public school’s History/Social Studies
department.
Data Summation
Data collection was split into two different portions: organizational/correlative data
collection, and statistically/testing data collection. Organizational (also referred to as correlative)
data are the data collected in prior research necessary to draw valid comparisons and conclusions
in keeping with scientific tradition, as established in the literature review. Statistical (also
referred to as testing) data refers to the quantitative data collected through the administration of
practice SATs, as provided through Khan Academy 2.
Organizational Data Instruments
The collection of organizational data was collected through a Data Collection Form
administered (Appendix A). All portions of this questionnaire that are deemed crucial to the
determination of correlations are shown below (Table 1). All questions were approved by the
Institutional Review Board (IRB).
2
All Khan Academy content is available for free at www.khanacademy.org. Citation consistent as requested by
Khan Academy, in accordance with their terms of service.
Table 1: Questions asked in the Data Collection Form containing information pertaining to
correlative statistics3
Questions
Q1 Please provide your Report Card/Schedule

(Quarter 1)
Q2 Which gender do you identify with?
Q3 What ethnicity do you identify with?
Q4 What would you consider your unweighted

GPA?
Q5 Do you qualify for Free or Reduced Lunch?
Q6 How many extracurriculars are you actively

involved in?
Q7 Please provide one of your previous PSAT

and/or SAT scores, including the
Math/English subscores.
Q8 Have you ever taken a SAT preparation

course (ex. Private companies, local library,
NOT self studying)
The collection of this data will be used to organize results following the conclusion of the
research experiment. The provision of personal class schedules by the subjects provides a basis
for the classification of academic track, an important variable that would help researchers in this
field determine whether there is a significant relation between academic track and score. In this
large, public, New York high school, core classes consist of one of each English, Social Studies,
Mathematics, Foreign Language, and Science course. Furthermore, the school provides three
3
Questions may not be represented in the table exactly how they were asked in the Data Collection Form, primarily
for formatting and concision. For a 100% accurate representation of the Data Collection Form, please refer to
Appendix A.
different class classifications: regents/regulars, honors, or Advanced Placement (AP). In an effort
to subdivide students, an individual must have three classes under one classification. For
example, if Student A is enrolled in AP Physics, Honors US History, AP Calculus, Honors
Spanish, and AP English Language and Composition, they are considered an “AP” student. This
classification is important to the study because it will help analyze the new online SAT as
measurements of academic merit.
Statistical Data Instruments
The collection of statistical data was conducted through the administration of an initial
benchmark exam and two different culminating exams: one on-paper and one online. Exams
were used in accordance with the Khan Academy Terms of Service, as indicated above.
Data collected included Evidence Based Reading and Writing Scores (reading
comprehension scores) and Math: No Calculator section scores. To conduct the statistical t-tests
themselves, a TI-84 Plus CE graphing calculator was used. Additionally, student attendance was
collected in order to connect attendance to student improvement.
Qualitative Interview Instruments
Interviews were also conducted to collect specific qualitative data and measure student
perceptions regarding the test preparation classes. Although the conversations themselves
differed between students, the questions below were consistently asked in every interview (Table
2).
Table 2: Coordinate Questions asked during qualitative Interviews
Q1 Q2 Q3 Q4
Did you feel the Did you take the Were you more Did you think the
content of the SAT 2022 March 12th comfortable with the SAT preparation class
preparation class SAT? If so, did you on-paper or online was more helpful for
helped you improve find that this course exam? math or english?
your math/english was helpful on the Which specific topic
skills (unrelated to SATs (in class and in (Alg. 1, Geometry,
the SAT)? March)? Grammar, etc.)?
All of these questions were also approved by the IRB. Transcripts of all interviews can be found
in Appendix E.
Procedures
After the distribution of the information sheets and Informed Consent Forms (Appendix
B), subjects were asked to join a Google Classroom and a Remind class. Remind is a mobile
messaging app that allows teachers, and in this case researchers, private communication to their
students or subjects. Google Classroom was chosen because it offered easy centralization for all
research related materials, including Informed Consent Forms, Data Collection Forms, Lesson
Plans, and other information. Remind was only used to send reminders to subjects to fill out
important forms or to remind subjects of class dates. Upon submission of the Informed Consent
Form and Data Collection Form, students were inducted subjects.
Forty-two individuals joined the Google Classroom, but only 36 students submitted both
forms. Of these 36, only 6 students were interviewed. All 36 students took the initial benchmark
examination and the two culminating examinations. Initially, all participating subjects took a
benchmark examination consisting of two reading passages and a math no-calculator section.
Following the completion of the benchmark examination, all students attended three, 45 minute
math preparation classes and three, 45 minute English preparation classes. Math classes
consisted of test-taking strategies and ways to “beat the test”, while English classes consisted of
reading comprehension skills and grammar rules. Following the conclusion of all test preparation
classes, subjects took two concluding benchmark exams: the first on paper, the second online.
To ensure privacy, no names will be mentioned in this study. Subjects that volunteered for
interviews will only be referred to through study codes (ex. Student A). Additionally, students
were informed in the Informed Consent Form that all responses were voluntary. Furthermore, the
words “subjects”, “volunteers”, “students”, and “participants” all refer to the same thing in the
course of this study. The procedure and method of this study was approved by the IRB.
Delimitations
Delimitations were used to narrow the possible subject pool. All ninth-graders were
excluded from the study because they have not taken an SAT or PSAT, and because it is not
possible for them to be classified along the AP track. Also, all twelfth-graders were excluded
because most of them have already taken their college entrance exams and therefore don’t need
preparation.
Results
Quantitative Results
The quantitative results of this research paper was conducted using two different
statistical tests. The first statistical test was an ANOVA test, which stands for “Analysis of
Variance”. ANOVA tests are used to compare more than two groups and determine whether a
relationship exists between them. The second statistical test used was a t-test, which determines
whether there is a significant difference between the means of two groups.
ANOVA Test Results
In this study, the ANOVA test sought to establish a relationship between subject
attendance and improvement in three different regions: overall score, math subsection, and the
english subsection. Additionally, this ANOVA sought to draw connections between the initial
examination as well as the on-paper and online culminating exams.
The ANOVA test takes multiple test groups and determines whether there is a statistically
significant connection between them. For the purposes of this study, groups were divided by
absences (Table 3).
Table 3: Six Groups, Shortened Summary Statistics, Paper Culminating Exam, Overall
Group Name n (sample size) Median Score Change
1) No Absences 3 -3
2) 1 Absence 6 0.5
3) 2 Absences 9 -1
4) 3 Absences 7 0
5) 4 Absences 6 -0.5
6) 5+ Absences 2 1.5
The median score changes indicated in Table 3 show that students that attended the most classes
(Group 1) experienced a score decrease of 3 points, while students that attended the least classes
(Group 6) increased their score by 1.5 points. Furthermore, the median score changes show no
pattern of increase or decrease. The absence of a pattern (i.e score decreases with more absences)
indicates that there is no relationship between greater attendance and positive score
improvements. Additionally, because the p-value 4 is greater than 0.05, we can conclude that
attendance does not lead to a positive score improvement on the SAT (Table 4).
Table 4: ANOVA Test Conclusions, summarized
Question Conclusion p-value Df
Is there a relationship between Attendance DOES NOT p = 0.526 32

attendance and positive score lead to a positive score
improvement? improvement on the SAT 0.526 > 0.05
For a more in-depth representation of the ANOVA Test results, please refer to Appendix C.
T-Test Results
The t-tests conducted for this research were divided into three different categories. T-test
1 (TA) was between the Paper Culminating Exam and the Initial Benchmark Examination. T-Test
2 (TB) was between the Online Culminating Exam and the Initial Benchmark Examination.
T-Test 3 (TC) was between the Online Culminating Exam and the Paper Culminating Exam. The
results of TB and TC will be discussed in this subsection. For the results of TA, English results
(TA, TB, and TC), and an in-depth explanation of TB and TC, please refer to Appendix D. For T B
and TC, it is important to point out the difference between collective and individual score
changes. Collective score changes are discussed first, and individual score changes will follow.
Collective scores use Table 5 and can be generalized to large populations. However, these
calculations result in over and underapproximations. To remedy this, individual score changes
are also analyzed. This analysis provides statistics that can be generalized to individual students.
4
In statistical processes, the p-value represents the probability of a certain event occurring. In this example, the
p-value indicated that the probability of a student not attending classes but still improving (or vice-versa) is 52.6%.
TB Results
The goal of TB was to determine whether students improved significantly on the Online
Culminating Exam. Table 5 illustrates the mean scores for the Math and English subsections on
each exam.
Table 5: Average Scores for Each Exam (out of 20)
Benchmark Benchmark On-Paper On-Paper Online Online

Math English Math English Math English
12.000 14.909 11.818 14.242 13.061 15.273
Here, Table 5 exhibits that students improved by 1.061 points on the Math portion of the online
exam when compared to the benchmark exam. Additionally, when comparing total score
improvement on the Online exam to the Paper exam, subjects improved by 1.425 points overall.
Tables 6 and 7 indicate the summarized test conclusions for TB. Tables 6 and 7 also demonstrate
significant improvement, as both p-values5 are below 0.05. Finally, according to Tables 6 and 7,
individual students improved by an average of 1.424 points overall and 1.895 points on the math
subsection.
5
In this scenario, the p-value represents the probability that Online scores are greater than Initial Benchmark scores
in the context of this study. In the context of this study, the results were expected to occur less than 5% of the time.
However, because these results did occur, the outcome is statistically significant.
Table 6: TB T-Test Conclusions, overall scores, summarized
Question Conclusion Avg. Change p-value Df
Is there a Students DO perform +1.424 points p = 0.019 32

statistically significantly better on
significant the online exam, when 0.019 > 0.05
improvement compared to the Initial
on the Online Benchmark.
exam?
Table 7: TB T-Test Conclusions, Math subsection scores, summarized
Is there a Students DO perform +1.895 points p = 0.011 32

statistically significantly better on
significant the online math, when 0.011 > 0.05
improvement compared to the Initial
on the Online Benchmark’s math.
exam (math
subsection)?
To summarize, test TB marked significant overall and math subsection improvement on the
online culminating exam.
TC Results
The goal of TC was to determine whether students performed noticeably better on online
exams than on paper exams. By taking a t-test of overall scores, math subsection, and english
subsection scores, test TC can analyze student score changes through different academic lenses.
Table 5: Average Scores for Each Exam (out of 20)
Benchmark Benchmark On-Paper On-Paper Online Online

Math English Math English Math English
12.000 14.909 11.818 14.242 13.061 15.273
According to Table 5, Online Math scores exceeded Paper Math scores by 1.243 points. Subjects
also performed better on the Online exam overall, collectively scoring 2.274 points higher.
Tables 7 and 8 portray the results of test TC from an individual lens, opposed to the collective
lens described above. Table 7 dictates that individuals score 2.273 points higher overall on online
exams when compared to on-paper exams. Table 8 indicates that students score 1.061 points
higher on online math sections than they would have on paper math sections.
Table 8: TC T-Test Conclusions, overall scores, summarized
Is there a There IS a statistically +2.273 points p = 0.001 32

statistically significant difference
significant between online and 0.001 > 0.05
difference paper exams. Students
between the TEND TO DO
Online and Paper BETTER on online
exams? exams overall.
Table 9: TC T-Test Conclusions, math subsection scores, summarized
Is there a There IS a statistically +1.061 points p = 0.01 32

statistically significant difference
significant between online and 0.01 > 0.05
difference paper exams. Students
between the TEND TO DO
Online and Paper BETTER on online
exams (math math subsections
subsection)?
Thus, TC notes that Online exam scores are, collectively, 2.274 points higher than Paper exams
overall and 1.243 points higher than Paper math subsection scores. Individually, the Online
scores are 2.273 points higher overall and 1.061 points higher in math.
Qualitative Results
All qualitative results were gathered from the interview process. Interview responses
(Appendix E) were codified by theme, per question (Appendix F) and analyzed. In this section,
each question will have different paragraphs, each relating to a different theme. 6 subjects were
asked all 4 questions, with a total of 24 responses.
Table 2: Coordinate Questions asked during qualitative Interviews
Q1 Q2 Q3 Q4
Did you feel the Did you take the Were you more Did you think the
content of the SAT 2022 March 12th comfortable with the SAT preparation class
preparation class SAT? If so, did you on-paper or online was more helpful for
helped you improve find that this course exam? math or english?
your math/english was helpful on the Which specific topic
skills (unrelated to SATs (in class and in (Alg. 1, Geometry,
the SAT)? March)? Grammar, etc.)?
Question 1
When asked whether or not SAT preparation classes helped students improve their
English or Math skills unrelated to the SAT, 4 out of 6 students believed there was some
significant improvement. Of these 4 students, 3 agreed on one common area of improvement.
Subjects 2, 4, and 6 believed that English classes improved their written grammar skills. Subject
2 stated, “I feel as though it really helped my grammar…,”. Subject 6 also stated that “I
definitely felt thorough improvement within my grammar…,”. Subject 4 gave valuable anecdotal
evidence regarding the value of grammar in essay writing, saying, “Specifically, probably essay
writing…something that was actually really useful…was the dangling participles. I didn’t realize
that the subject had to be directly after the comma, and so I started utilizing that a lot more once I
saw it.”
Question 2
All 6 volunteers took the March 12th, 2022 SAT. This question was asked before scores
were released. The goal of question two was to determine whether subjects felt the classes were
helpful, regardless of whether their actual SAT scores reflected an improvement. All 6 subjects
answered yes. Subjects 5 and 6 both responded with a concise “Yes. Yes.” Subject 2 responded
with “Yes. 100%,” while Subject 3 replied “Yes. Definitely.” The other two subjects responded
with similar answers. As a result, it is safe to conclude that all subjects believed the classes
themselves were helpful, regardless of their numerical reflection on either culminating exams or
the March 12th SAT.

Question 3
Although the majority of the students (4/6) preferred the Paper exam, the most interesting
responses stem from the minority (2/6) students. These 2 students (Subjects 1 and 5) introduced a
mixed preference theme to the results. Subject 1, interviewed first, felt as if they were, “focused
more when I took the [online] exam, especially on the English.” This response, however, gives
no reason as to why Subject 1 felt increased focus. Subject 5 provides us with a potential
reasoning, stating, “...in terms of the reading and grammar, the online exam is better because you
can see the questions and have the article next to you.” Here, the formatting difference provides a
potential reasoning for their conclusions. On the other hand, Subject 5 believed math was better
on the paper exam “...because it's just a lot easier to write on it.” Subject 1 reinforces this claim,
declaring that they “...definitely liked the scrap paper on the math though.”
Subjects 4 and 6, both who wholeheartedly preferred the Paper exam, had similar
reasonings. Paraphrasing, both subjects like the tangibility of the exam and felt more comfortable
underlining directly on the paper or flipping back and forth.
Question 4
Question 4 brought the most diverse responses. Subjects 2, 4, and 6 believed Grammar to
be the most useful class. On the other hand, Subjects 3 and 5 preferred Geometry. Finally,
Subject 1 preferred Algebra 1. Subject 2 emphasized that “...my last SAT I didn’t do so hot on
grammar..,” but that, “I think [the classes] was helpful in my grammar, especially.” Subjects 4
and 6 simply emphasized grammar as their most appreciated class, but chose not to elaborate
when follow up questions were asked.

Subject 5 preferred Geometry but also emphasized “...a bit more advanced algebra.”
When asked to elaborate, Subject 5 emphasized that they additionally benefitted from the
“problem solving, ‘tackling the test’ type strategies covered.”
Additional Results
Additional results, especially those that involve drawing comparisons with prior research
and do not directly address the gap fulfilled by this research, will be addressed in the appendices.
For those results, please proceed to Appendix G.
Discussion
This study sought to evaluate the effectiveness of extracurricular, in-person SAT
preparation classes on online and paper SATs. Additionally, this study sought to analyze score
differences between paper and online exams.
Findings
Looking at the results of all statistical tests, it can be concluded that extracurricular,
in-person SAT classes are ineffective on improving student scores on online and paper exams.
This is proven using the results of the ANOVA test, specifically the p-value, which indicates no
relationship between attendance and score improvement. This conclusion agrees with
Donaldson’s 2013 study, which concluded that preparation does not yield higher scores on the
SAT or ACT. Noted by tables 6 and 7, students improved significantly on the online exam when
compared to the initial benchmark exam. However, the results of the ANOVA test demonstrate
that this improvement cannot be connected to attendance at preparation classes. Regardless of the
ANOVA results, all six interview subjects agreed that the classes were helpful on the in-class
SATs. For example, Subject 1 stated, “...I could apply…Grammar skills and specific math
questions…,”. This analysis demonstrates the value of the qualitative and quantitative data
collection process. Without these interviews, there would be no indication as to why students
performed better on the online exam than on the paper exam. Together, the quantitative and
qualitative analyses allowed this research to find that individuals performed better on online
SATs (+1.424 overall, +1.895 math), perhaps a result of skills they took away from classes.
Tables 8 and 9 demonstrated that students performed noticeably better on online exams
than their paper counterparts. Tables 8 and 9 illustrate that collective scores increase by 2.274
points overall and 1.243 points in math. Individual students also improved by 2.273 points
overall and 1.061 points on the math. Additionally, using subject interviews it can be established
that exam formatting is the reason for score improvements. As Subject 5 mentioned, “...the
online exam is better because you can see the questions and have the article right next to you.”
Subject 1 also felt more focused on the online exam. To better understand this, visualize a
scenario where a student took a paper SAT, forgot all the questions, and then proceeded to take
an online SAT. This study finds that the aforementioned student would perform better on the
online exam, attributed to formatting and focus, and reinforced by score improvements. Together,
these results find that students do inherently better on online SATs than on paper SATs.
Gap Fulfillment
The primary gap this research fulfilled was the measurement of score changes on online
exams. All prior research has focused on paper exams. As more and more standardized tests shift
towards an online format, research focusing on online exams will become significantly more
important. By focusing on online exams, this research plugs this hole in research and contributes
to an up and coming area of research.
Implications
The SAT is a standardized test - average scores and layouts must be comparable across all
testing iterations. This online SAT must also be comparable to past SATs. This research found
that students taking this new online SAT can expect to inherently improve their scores overall
and in the math subsection. If these results held true when scaled up, the SAT would not be truly
standardized. To remedy this and maintain the comparable nature of the SAT, the College Board
might feel the need to change the difficulty of the math section (where improvement was
determined) or the curve applied to the entire exam.
In a similar vein, as this research proved that students perform inherently better on online
exams, more and more online exams might change the general difficulty or the curve applied to
their results. These changes, based on the foundation of this research, could seriously affect the
appeal of online exams.
Limitations
The primary limitation for this research was the extracurricular nature of the preparation
classes offered. Oftentimes, subjects experienced scheduling conflicts and were unable to attend
classes. Additionally, time constrictions with the AP Research course meant that make-up classes
could not be offered. Students that missed those initial classes could not apply the new skills
their peers learned, and so might not have improved as much. Greater attendance could have
changed the outcome of the ANOVA test, which could have impacted the conclusions and
implications drawn.
Areas for Further Research
This research focused on extracurricular, in-person preparation classes in a large, public,
New York High School. Within the same high school, using the same method, with the same goal
of evaluating online SATs as compared to paper SATs, future researchers could focus on
curricular (during school hour) preparation classes. Additionally, a researcher might want to
focus on online or virtual preparation, for example, the personalized preparation offered by Khan
Academy. By analyzing all three research spheres in one highschool, a very in-depth research
profile could be made. Expanding this process to other schools would allow for a national
research profile that effectively and adequately provides new literature regarding online
standardized testing.
References
ACT, Inc. (n.d.). About the ACT test. Retrieved February 20, 2022, from
https://www.act.org/content/act/en/products-and-services/the-act-educator/the-act-test.ht
ml#order-reg-materials
This source provides us with valuable information regarding statistical information
regarding the 2019 ACT, the year in which many students took the ACT. Additionally,
this source provides information that helps contextualize the ACT in terms of popularity
and usage in the realm of college admissions. This information was used to compare the
ACT with the SAT.
Appelrouth, J., & Zabrucky, K. (2017). Preparing for the SAT: A review. College and University,
92(1), 2-17.
This source provides valuable information regarding the importance of the SAT in other
aspects of higher education, and not merely admissions. The emphasis in this article lies
on the importance of SAT scores on financing and scholarships, as well as an institution's
donations from alumni and others. This new aspect to my research is something that
should be explored when establishing the importance of the SAT. As a result, this source
will be in literature review.

Briggs, D. C. (2001). The effect of admissions test preparation: Evidence from NELS:88.
Chance, 14(1), 10-18.
This source provides me with crucial conclusions and the topics used to draw these
conclusions. Referring to conclusions drawn similarly in other research, I used the
datasets
discussed in this research to collect and quantify data in my own research. For this
reason, this source is a foundational source in my literature review.
College Board. (2019, September 24). Over 2.2 million students in class of 2019 took SAT,
largest group ever. College Board Newsroom.
https://newsroom.collegeboard.org/over-22-million-students-class-2019-took-sat-largest-
group-ever#:~:text=Over%208%20million%20students%20took,than%20students%20wh
o%20did%20not.
This source provides us with valuable information regarding statistical information
regarding the 2019 SAT, the SAT in which most students took the SAT. Additionally, this
source provides information that helps contextualize the SAT in terms of popularity and
usage in the realm of college admissions.

College Board. (2022, January 25). Digital SAT brings student-friendly changes to test
experience. College Board Newsroom.
https://newsroom.collegeboard.org/digital-sat-brings-student-friendly-changes-test-experi
ence
This source provides incredibly valuable information regarding the changes to the SAT,
as stated by the College Board. This source is therefore a foundational source during my
tenure in AP Research.
Donaldson, R. F. (2013). Survey of college entrance exam preparation methods [Doctoral
dissertation, Liberty University]. ProQuest Multiple Databases.
This source analyzes the effectiveness of third-party test preparation classes on
improving SAT scores. The study concludes that there is no correlation. This study also
analyzes the correlation between high school GPA and SAT score, determining that
students in higher level classes tend to do better on the SAT. This data is collected
thoroughly and with experimental integrity and can therefore be trusted. As a result,
because I can analyze the conclusions and am working off this body of evidence, this
source is a foundational source for my research.
Internal Revenue Service. (n.d.). Return of organization exempt from income tax (Technical
Report No. Form 990). Internal Revenue Service.
https://apps.irs.gov/pub/epostcard/cor/131623965_201912_990_2021022617761073.pdf
This source is the Form 990 for the College Board in the year 2019. This source provides
information regarding the revenue of the College Board and gives significant information
contextualizing the college admissions entrance exams, specifically supporting the size
and stability of the SAT and the College Board.
Mattern, K., Kobrin, J., & Shaw, E. J. (2011). An alternative presentation of incremental validity:
Discrepant SAT and HSGPA performance. Educational and Psychological Measurement, 71(4),
638-662.
This source, through a well constructed scientific method and reinforced by sound
experimental design reports on the discrepancy between high school GPA and student
SAT scores, especially when related to marginalized groups: females, minorities, low
socioeconomic status, and nonnative English speakers. Determining whether these same
discrepancies hold true on the electronic SAT is a crucial aspect of the limitations of my
research. Therefore, this source is part of my foundational sources.
Monks, J. (2009). The impact of merit-based financial aid on college enrollment: A field
experiment. Economics of Education Review, 28(1), 99-106.
This source provides valuable information regarding the SAT and its importance as an
indicator of academic value, specifically as higher education institutions seek to provide
academically gifted students with merit-based financial aid. This source, through a
scientific experimental model, concluded that institutions that offer need-based aid
instead of merit-based aid often lose out on academically gifted students. For these
conclusions, this source will be a part of my literature review.

Moss, G. L., Chippendale, E. K., Mershon, C. W., & Carney, T. (2012). Effects of a coaching
class on the ACT scores of students at a large midwest high school. Journal of College
Admission, (217), 16-23.
This source provides a foundational research method. It also provides conclusions that I
will be able to analyze in my own research. Additionally, this source provides some
delimitations, as it discusses the ACT compared to my research's focus on the SAT.
Finally, this source provides important contextualization for the test preparation industry
and the value it provides students who want to enter college.
Powers, D. E. (1988). Preparing for the SAT (College Board Reports 88-7). College Board.
This source is the first to analyze qualitative data on the effectiveness of SAT preparation
methods. This source concludes that although qualitative data was collected regarding the
most effective test preparation strategy, these conclusions were somewhat limited.
Therefore, in an effort to expand the current research regarding the qualitative analysis of
effective test preparation strategies, this study will also include qualitative analyses.
Ringer, V. M. (2014). The effects of the Cambridge Test Preparation Program on Scholastic
Assessment Test scores [Doctoral dissertation, Walden University]. ProQuest Multiple
Databases.
This dissertation analyzes the effectiveness of SAT preparation on SAT scores and
student outcomes. Their method was very similar to the method of my study, and this
study concluded that there was a significant increase in SAT scores while the Cambridge
Test Preparation Program was active in schools. This source is important because it
provides a conclusion that I could test in my study. Also, this source will serve as a
foundational source in my research.
Robinson, M., & Monks, J. (2005). Making SAT scores optional in selective college admissions:
A case study. Economics of Education Review, 24(4), 393-405.
This source essentially discusses the impact of SAT optional pilot programs in a single
college in the Northeastern United States. The study has various takeaways, always
relating the importance of the SAT to college admissions. This source is important not
only for its conclusions, but also because it provides perspectives that prove valuable to
the holistic understanding of the SAT on the college admissions process.

Appendix A - Data Collection Form
Name: ____________________________________ Grade:____________
Data Collection Form

Anonymous research conducted on the effectiveness of different SAT preparation methods
and how the Sachem Central School District can take effective action to implement these
methods in the greater context of a 1:1 computer rollout.
- Google Classroom Code For All Consenting
Volunteers:___________________________
- Please join the Google Classroom with your private email
All human subject related data will be collected anonymously, and will only be referred to
through student ID numbers. Additionally, this form will collect data that may be considered
sensitive to some individuals. By no means are you obliged to complete this form, as all
responses to this form are voluntary. For participation in this study, participants will receive
practice on specific SAT sections and food/drink during any practices, classes, or meetings.
● Please refer to the checklist on the following page for all required information
● For all students, signatures must be collected. Please refer to the last page of this packet
for the terms that you are agreeing to.
● For all students, there are some additional, and perhaps sensitive questions/data that must
be collected. Please be aware that by signing the form on the last page, you are
committing to answering these questions truthfully and to the best of your ability.
Thank you for your time and consideration. If you have any questions, please do not hesitate to
contact me:
Email = sidsheth6@gmail.com
Phone Number = +1 631-897-9581
Checklist
Name and Grade filled in at the top of the form
Schedule Sheet Filled Out
Student ID / Pin Number written legibly
Guidance Counselor Name Written Legibly
Additional Information Pages (Pgs. 4 & 5) Filled out Completely
Access Statement
By signing this statement, you are affirming that you will fill out this form truthfully and to the
best of your ability. Additionally, by signing here, you are allowing me access to your
schedule and transcript ONLY, and ONLY if you are a selected participant of this study.
___________________________________ _______________________________
(Print Volunteer Name) (Volunteer Signature)

Required Information:
Report Card (Quarter 1) (A- Day Schedule):
CLASS TEACHER REPORT CARD

*Please list AP, Honors, or AVERAGE
Regents/Regulars
Additional Information Page
First Name : _____________________________________________________________
Last Name : _____________________________________________________________
Student ID Number (include 0s) : _____________________________________________
Guidance Counselor : ______________________________________________________
Age (years) : _____________________ Grade : _______________________
Please Check Off An Answer:
1) Which gender do you identify with?
▢Male ▢Other: _______________________
▢Female ▢Prefer Not to Say
2) What ethnicity do you identify with?
▢White ▢Asian-including Indian Subcontinent
▢Black / African-American ▢Pacific Islander
▢American Indian or Alaskan Native ▢Prefer not to say
3) What would you consider your unweighted GPA?
▢0 - 60 (F) ▢80 - 89 (B)
▢60 - 69 (D) ▢90 - 100 (A)
▢70 -79 (C)
4) Do you qualify for Free or Reduced lunch?
▢Yes ▢No
▢Prefer Not To Say

5) How many extracurriculars (clubs and sports, NOT honor societies) are you actively
involved in?
▢0 - 3 ▢7 - 10
▢4 - 6
6) Please write down one of your previous SAT or PSAT scores (estimation for scores is
acceptable)
▢SAT ▢PSAT ▢Neither
SAT Score : _________________ Math: ______/800 English: _______/800
PSAT Score : ________________ Math: ______/760 English: _______/760
7) Have you ever taken a SAT preparation course (ex. Private company, Sachem Library,
etc., NOT self studying)
▢Yes ▢No
▢Unsure: _________________________________________
Questions?
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
Contact Me: (to email me, please use your private email)
Siddharth (Sid) Sheth
11th Grade Hr. # - B227
Email : sidsheth6@gmail.com
Cell Phone Number : 631 - 897 - 9581

Appendix B - Informed Consent Form
Informed Consent
This study will examine the effectiveness of extracurricular, in-person SAT test
preparation on online SATs, as compared to paper SATs. The goal of this study will be to
evaluate the effectiveness of these classes on both SAT formats and to measure the difference
between online and paper exams.
Participation in this study will be minimally time consuming. Over the entire course of
the study, which will range from 2-3 weeks, the commitment would only be one to two ninth
periods per week. Beyond that, I completely understand if there are conflicts and absolutely
would not mandate any of the meetings. Unfortunately, there will be no make-up sessions, and
all volunteers will be given bagged snacks and water for coming to meetings.
If affirming yourself to volunteer in this study, please be sure to join the Google
Classroom. All further information, methodology, and questions will be addressed there. Thank
you, and if you have any questions, please do not hesitate to contact me!
Sincerely,
Siddharth (Sid) Sheth Email : sidsheth6@gmail.com
Grade - 11th Cell Phone : 631 - 897 - 9581
Please Turn Over

Affirmation Statement
-By signing here, you are certifying that you will fill out all future forms truthfully and to the
best of your ability. Additionally, by signing here, you consent to fulfilling the obligations
required of this study.
-By signing here, you are consenting to access to your schedule and transcript.
-To protect your privacy, none of the data collected via this survey and other implanted research
will be assigned to a name, and will instead be connected to an individual number that is not
your PIN Number.
_______________________________ ________________________________
(Print Volunteer Name) (Sign Volunteer Name)
______________________________ ________________________________
(Print Parent/Guardian Name) (Sign Parent/Guardian Name)
If Affirmed:
- PLEASE JOIN THE GOOGLE CLASSROOM ON A PRIVATE EMAIL

- Please join the Google Classroom at this code: 3zpia6w
- Please fill out the “Data Collection Form” and upload by the due date
Appendix C - ANOVA In-Depth
- Overall Score Changes, organized by absence groups
- Improvement Dot Plot Distributions, organized by absence groups

- ANOVA Summary Statistics; all numbers in terms of points changed overall
- SD > Standard Deviation
- Q1 > The median of the lower half of the data
- Q2 > The median of the upper half of the data
- ANOVA Statistical Conclusion
- P-Value > Greater than 0.05, indicates NO RELATIONSHIP between attendance
and score improvement
- Error Group > Refers to any subjects that did not improve overall. Despite the
name, this group does not imply error or impact the results of our study.
Description Test Type Test Statistic Conclusion
Analysis of Variance Better attendance 314.242 We conclude that

Calculation (ANOVA) leads directly to there is no
between Improvement improvement on relationship
(Paper Culminating - overall scores. between Student
Initial Benchmark) and Improvement and
Attendance (Calculations were Student
performed for each Attendance
group)
p - value = 0.526 0.526 > 0.05 Df = 32
- Statistical Summary of ANOVA results

Appendix D - T-Test Results
TA Results
TA Overall
- Improvement Dot Plot Distribution, Overall Scores
- TA Overall Statistical Conclusion
- P-Value > Because the p-value is greater than 0.05, there is no significant
improvement on the paper culminating SAT as compared to the Initial
Benchmark, overall
TA Math
- Improvement Dot Plot Distribution, Math Scores
- TA Math Subsection Statistical Conclusion
Benchmark, math subsection

TA English
- Improvement Dot Plot Distribution, English Scores
- TA Math Subsection Statistical Conclusion
Benchmark, english subsection

TB Results
TB Overall
- Improvement Dot Plot Distribution, Overall Scores
- TB Overall Statistical Conclusion
- P-Value > Because the p-value is less than 0.05, there is a significant
improvement on the online culminating SAT as compared to the Initial
Benchmark, overall
TB Math
- Improvement Dot Plot Distribution, Math Scores
- TB Math Subsection Statistical Conclusion
- P-Value > Because the p-value is less than 0.05, there is a significant
Benchmark, math subsection

TB English
- Improvement Dot Plot Distribution, English Scores
- TB English Subsection Statistical Conclusion
Benchmark, english subsection

TC Results
TC Overall
- Score Difference Dot Plot Distribution, Overall
- TC Overall Statistical Conclusion
- P-Value > Because the p-value is less than 0.05, there is a significant difference
and improvement on the online culminating SAT as compared to the paper
culminating SAT, overall

TC Math
- Score Difference Dot Plot Distribution, Math Scores
- TC Math Subsection Statistical Conclusion
- P-Value > Because the p-value is less than 0.05, there is a significant difference
and improvement on the online culminating SAT as compared to the paper
culminating SAT, math subsection

TC English
- Score Difference Dot Plot Distribution, English Scores
- TC English Subsection Statistical Conclusion
difference and no improvement on the online culminating SAT as compared to the
paper culminating SAT, english subsection

Appendix E - Interview Responses
Interview 1
Q1) Did you feel the content of the SAT preparation class helped you improve your math/english
skills (unrelated to the SAT)?
A - I think the classes definitely improved my SAT score because I didn’t study at all, for like,
the PSATs, and I want to get higher than that, and I think that because I used the skills that we
learned in class on the SAT, that will help my SAT score be better.
Q2) Did you take the 2022 March 12th SAT? If so, did you find that this course was helpful on
the SATs (in class and in March)?
A - Yes I did.
A - Yeah, there was some things that we learned that I did use. I used some of the grammar skills
we learned on the grammar part and for the math part, there were some questions where I was
able to apply the skills to the actual question because they were very similar, so that helped me.
Q3) Were you more comfortable with the on-paper or online exam?
A - I liked online better, to be honest. I felt like I was focused more when I took the exam,
especially on the English. I definitely liked the scrap paper on the math though.
Q4) Did you think the SAT preparation class was more helpful for math or english? Which
specific topic (Alg. 1, Geometry, Grammar, etc.)?
A - I think math because I feel like with English it’s kind of like a hit or miss. Like you either
know it or you don’t. I feel like for math you can manipulate it, where like you could, you might
not get it completely right, but you can get partial credit and stuff like that, that will get your
points
A - Algebra 1. Grammar.
Interview 2
A - I feel as though it really helped my grammar, especially that one grammar lesson that helped
me distinguish between what these, uh, punctuations do.
A - Yes I did.
A - 100% I did.
A - I thought I did better on paper but I actually didn’t.
I - What about the format?
A - I liked the on-paper more, but I did better online. I like the physical test more.
A - I think it was helpful in my grammar, especially. Grammar more than reading comprehension
because I already had a basic understanding of reading comprehension.
I - Grammar more than math, you think?
A - Yeah, because my last SAT I didn’t do so hot on grammar, but yeah.
I - Did you find this class was helpful in math at all? If so, which topic?
A - Yeah I did. I found it more helpful in geometry, just because I was pretty solid with Algebra
1 already. I needed a little refresher on Geometry.

Interview 3
A - Yes, most definitely.
I - Specifically what?
A - Certain questions that I didn’t know the answer to, the teacher kinda cleared up for me.
I - On the SAT or outside of the SAT?
A - Inside.
I - Did you see anything outside of the SAT?
A - Not as much. Nothing much in essay writing or in PreCalculus. Maybe a slight change, but
nothing big.
A - Yeah.
A - Definitely.
A - I thought that the paper was much better than the online exam.
I - Anything format related?
A - The format of eliminating choices and narrowing it down to 2, then picking the choice that
answers the question the best.
A - Math.
A - Geometry. Geometry the most out of all the topics.
I - If you did think it was helpful in English, did you think grammar or reading comprehension
was better?
A - Grammar.
Interview 4
A - For sure. Specifically, probably essay writing, like the grammar that we went over definitely
helped a lot.
I - Was there anything specific in grammar? Comma placement, colons, anything like that?
A - I would say, something that was actually really useful that I didn’t realize before was the
dangling participles. I didn’t realize that the subject had to be directly after the comma, and so I
started utilizing that a lot more once I saw it.
A - Yes.
A - Yes I did. There was a lot of, some of the, some of the trigonometry (trig) that we went over
popped up on there.
A - I definitely prefer taking it on-paper. It’s easier to go back and forth and stuff like that. And
it’s just a different feel actually underlining stuff on the paper reading section.
A - I’ll probably go English.
A - Probably the Grammar.
I - On Math, if you thought it was helpful at all, did you think it was more helpful on Algebra 1
or Geometry?
A - Geometry, probably.
I - Any other topics within Math?
A - There was a lot of the, I don’t know, I guess it would be considered precalc stuff? Like lots of
trig stuff. There was a lot of stuff about that on there. It did pop up, like there was a sin(a) =
cos(b) thing popped up.

Interview 5
A - Yes. Just making sure that I tackle problems, specifically in math, in different ways, and not
taking a strictly linear approach to problems.
A - Yes.
A - Yes.
A - I would say in terms of the reading and grammar, the online exam is better because you can
see the questions and have the article next to you.
I - Does that imply that you found the math was better on the paper exam?
A - Yes. And that was because it's just a lot easier to write on it.
I - Scrap paper?
A - Yes.
A - I would say, for me personally, it was a lot better for math.
I - And for what specific topic?
A - Geometry, and like, probably a bit more advanced algebra.
I - So like the problem solving, “tackling the test” type strategies covered?
A - Yeah.
I - Did you think that prep class was helpful in English too?
A - Yeah. I think it was more helpful in reading comprehension.
Interview 6
A - Yeah for sure. I definitely felt thorough improvement within my grammar and in older math
strategies that I have not used in a while.

A - Yes.
A - Yes.
A - On-paper.
I - Was there any reason for that? A formatting issue or something?
A - The reading passages were definitely easier to refer back from, and the on-paper exam is
simply more tangible than the online one.
A - English, and definitely grammar.
I - Did you think it was helpful at all in math?
A - Yes.
I - Which topic specifically?
A - Definitely geometry, and earlier algebra 1.

Appendix F - Interview Themes
Interview Themes
- 1) Classes improved SAT score
- 2) Classes helped my Grammar
- 3) Slight Changes, but nothing significant
- 4) Probably essay writing, specifically Grammar
- 5) Problem solving in math
- 6) I definitely felt a thorough improvement within my grammar
- 1) Yes. Grammar skills and specific math questions where I could apply what was
learned.
- 2) Yes, 100%
- 3) Yes. Definitely.
- 4) Yes I did. A lot of trig popped up.
- 5) Yes. Yes.
- 6) Yes. Yes.
Q3A) Were you more comfortable with the on-paper or online exam?
- 1) I liked online better. I felt more focused.
- 2) I liked the on-paper more, but I did better online.
- 3) I thought that the paper was much better than the online exam.
- 4) I definitely prefer taking it on-paper.
- 5) I would say in terms of the reading and grammar, the online exam is better;
Paraphrasing - Math was better on paper because you could write directly on the paper
- 6) On-paper.
Q3B) Any follow up question that asks about formatting. Often, this question was not asked, but
was answered in the initial response.
- 1) Mixed > Online/English - I felt like I was focused more when I took the exam,
especially on the English.
> Paper/Math - I definitely liked the scrap paper on the math though.
- 2) Paper - I like the physical test more.
- 3) Paper - The format of eliminating choices and narrowing it down to 2, then picking
the choice that answers the question the best.
- 4) Paper - It’s easier to go back and forth and stuff like that. And it’s just a different feel
actually underlining stuff on the paper reading section.
- 5) Mixed > Online/English - the online exam is better because you can see the questions
and have the article next to you.
> Paper/Math - Yes. And that was because it's just a lot easier to write on it.
- 6) Paper - The reading passages were definitely easier to refer back from, and the
on-paper exam is simply more tangible than the online one.

Q4A) Did you think the SAT preparation class was more helpful for math or english? Which
- 1) I think math because I feel like with English it’s kind of like a hit or miss. Algebra 1.
- 2) I think it was helpful in my grammar, especially. Grammar more than reading
comprehension because I already had a basic understanding of reading comprehension.
- 3) Geometry. Geometry the most out of all the topics.
- 4) Probably the Grammar.
- 5) Geometry, and like, probably a bit more advanced algebra.
- 6) English, and definitely grammar.
Q4B) Asked students whether they felt Math/English (opposite to their response in 4A) was
helpful in any way even if significantly less than their stated response in 4A.
- 1) Grammar.
- 2) Yeah I did. I found it more helpful in geometry, just because I was pretty solid with
Algebra 1 already. I needed a little refresher on Geometry.
- 3) Grammar.
- 4) Geometry, probably. There was a lot of the, I don’t know, I guess it would be
considered precalc stuff? Like lots of trig stuff. There was a lot of stuff about that on
there. It did pop up, like there was a sin(a) = cos(b) thing popped up.
- 5) Yeah. I think it was more helpful in reading comprehension.
- 6) Definitely geometry, and earlier Algebra 1.

Appendix G - Additional Results
A) Appendix G
Appendix G takes important claims from the Literature Review and connects the results
of this study in comparison with those claims. This study concludes that comparisons between
different variables and improvement on the SAT established using paper exams can be applied to
online exams as well.
B) HSGPA and the Number of Students that Improved in Each Section, Per Each Test
HSGPA # of # of # of # of # of # of
(unweighted) subjects subjects subjects subjects subjects subjects
Imp. Imp. Math Imp. Imp. Imp. Math Imp.
Overall (P) English Overall (O) English
(P) (P) (O) (O)
99, 100 4 5 1 5 6 1
96 - 98 8 7 2 6 6 0
93 - 95 7 9 0 9 6 3
90 - 92 3 1 5 4 4 4
88, 89 0 1 2 0 4 3
86 - 88 0 2 1 0 3 2
83 - 85 0 3 1 1 3 2
80 - 82 0 4 0 2 0 2
This table categorized students based on their High School GPA. A relationship was then drawn
between student HSGPAs and whether or not they improved on either exam. This table clearly
indicates that students with a higher HSGPA (90+) improved more often on both paper and
online SATs than their counterparts with lower HSGPAs. This analysis agrees with Mattern et.
al’s 2012 study and Donaldson’s 2013 study, which claimed that students with greater HSGPAs
were more likely to improve and perform better on SATs. This conclusion can also be extended
to online exams.
C) Math Grades and the Number of Students that Improved in Each Section, Per Test
Math # of # of # of # of # of # of
Class subjects subjects subjects subjects subjects subjects
Average Imp. Imp. Math Imp. Imp. Imp. Math Imp.
(P) (P) (O) (O)
99, 100 3 6 2 2 8 1
96 - 98 4 8 1 4 7 2
93 - 95 7 6 1 3 4 2
90 - 92 2 0 3 3 5 1
88, 89 0 1 4 0 2 1
86 - 88 1 1 2 0 2 0
83 - 85 1 0 3 2 1 0
80 - 82 0 3 0 4 1 0
This table categorized students based on their Math class averages. A relationship was then
drawn between math class performance and improvement on each section of each exam. The
goal was to look for a connection between higher averages and improvement on the math
section. Generally, students with a higher math average (90+) improved more often on the math
section than their peers with lower math averages. This agrees with the results of Briggs’s 2001
study, which stated that coaching would be the most effective on math sections among students
with a higher math average. Furthermore, that same conclusion can be extended to online SATs.
D) Academic Track and the Number of Students that Improved in Each Section, Per
Test
Academic # of # of # of # of # of # of
Track subjects subjects subjects subjects subjects subjects
Imp. Imp. Math Imp. Imp. Imp. Math Imp.
(P) (P) (O) (O)
Advanced 8 9 6 10 10 5
Placement
Honors 4 7 5 6 6 3
Regulars/ 2 4 0 3 4 2
Regents
This table categorized students based on academic track. The method for classification is
expanded upon in the Method Section (pg. 15). This table shows that students in higher academic
tracks tend to show improvements in score more often than their counterparts in lower academic
tracks. Furthermore, students in all tracks show improvement more often on the online exam,
reinforcing the claim that students perform inherently better on online exams. The results of this
table also imply that the SAT measures a student's academic merit to a certain extent. This table
shows that as students take more difficult classes, they show a greater ability to improve on the
SAT. However, although students in more difficult classes improve more, they may not
necessarily be better performing students. It is possible that students with higher GPAs are in
regents/regulars or honors classes. For that reason, although we can assume that a more difficult
academic track is connected to an ability to improve, we cannot conclude the SAT measures
student merit. To draw that conclusion, it may be more valuable to look at weighted and
unweighted GPAs more in-depth.

Updated Research Paper

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Updated Research Paper

Uploaded by

Copyright:

Available Formats

ANALYZING THE SCORE DIFFERENCE BETWEEN ON-PAPER AND ONLINE SAT EXAMS 1

In-Person SAT Classes In a Large, Public, New York High School

Word Count: 5496

scores on online and paper SATs.

interviews were conducted to collect data.

including accessibility to tests and new aspects of or approaches to college admissions.

Gap in the Research

score improvements on an online examination. Additionally, previous research similar to this

on-paper SATs in a large, public New York highschool?

entrance exams, financial aid, and test optional.

College Entrance Exams As Measures of Student Academic Merit

essentially equated academic merit to entrance exam performance. In fact, according to

$90,000 more, illustrating the significance of standardized testing performance to scholarships.

fiscal consequences becomes clear.

tracking and score improvement.

Data Collected in Prior Research

collected in this research.

conclusion: “the discrepancy...increasingly overpredicts…[first year college GPA]...as the

In summary, HSGPA, socioeconomic status, performance in math classes specifically,

The Impact of Test Preparation on Student SAT Scores

In 2013, educational researchers at a private Virginia university conducted a survey of

Program showed a significant increase in verbal and math SAT scores.

in-person, extracurricular preparatory classes on SAT outcomes.

2017; Mattern et al., 2011; Ringer, 2014).

different factors necessary to draw valid conclusions.

Research Design and Method

new SAT specifically.

on a previously limited and underrepresented way to analyze effectiveness, firmly positioning

effectiveness of individual SAT preparation (Powers, 1988, p. 15). Additionally, although

emphasizes the scope and importance of this experimental design.

summarizing informational sheets through this public school’s History/Social Studies

collection, and statistically/testing data collection. Organizational (also referred to as correlative)

practice SATs, as provided through Khan Academy 2.

Organizational Data Instruments

Institutional Review Board (IRB).

Q1 Please provide your Report Card/Schedule

Q2 Which gender do you identify with?

Q3 What ethnicity do you identify with?

Q4 What would you consider your unweighted

Q5 Do you qualify for Free or Reduced Lunch?

Q6 How many extracurriculars are you actively

Q7 Please provide one of your previous PSAT

Q8 Have you ever taken a SAT preparation

different class classifications: regents/regulars, honors, or Advanced Placement (AP). In an effort

example, if Student A is enrolled in AP Physics, Honors US History, AP Calculus, Honors

measurements of academic merit.

Statistical Data Instruments

collected in order to connect attendance to student improvement.

Qualitative Interview Instruments

Table 2: Coordinate Questions asked during qualitative Interviews

Form and Data Collection Form, students were inducted subjects.

whether there is a significant difference between the means of two groups.

ANOVA Test Results

examination as well as the on-paper and online culminating exams.

absences (Table 3).

Group Name n (sample size) Median Score Change

Table 4: ANOVA Test Conclusions, summarized

Question Conclusion p-value Df

Is there a relationship between Attendance DOES NOT p = 0.526 32

Table 5: Average Scores for Each Exam (out of 20)

Benchmark Benchmark On-Paper On-Paper Online Online

12.000 14.909 11.818 14.242 13.061 15.273

Table 6: TB T-Test Conclusions, overall scores, summarized

Name: ________________________ Grade: