ECS-308

Data Science and Machine Learning

Project Report
Automated Essay Grading Using Machine Learning
By
Debjit De Sarkar
Roll No.- 18079
1. Introduction

1.1 Brief Historical Overview

Most historical summaries of AES trace the origins of the field to
the work of Ellis Batten Page. In 1966, he argued for the possibility
of scoring essays by computer, and in 1968 he published his
successful work with a program called Project Essay Grade (PEG).
Using the technology of that time, computerized essay scoring
would not have been cost-effective, so Page abated his efforts for
about two decades. Eventually, Page sold PEG to Measurement
Incorporated.
By 1990, desktop computers had become so powerful and so
widespread that AES was a practical possibility. As early as 1982,
a UNIX program called Writer's Workbench was able to offer
punctuation, spelling and grammar advice. In collaboration with
several companies (notably Educational Testing Service), Page
updated PEG and ran some successful trials in the early 1990s.
After that several other companies like- Pearson Educational
Technologies, Vantage Learning further developed this technology.
In 2012, the Hewlett Foundation sponsored a competition
on Kaggle called the Automated Student Assessment Prize
(ASAP). 201 challenge participants attempted to predict, using
AES, the scores that human raters would give to thousands of
essays written to eight different prompts. The intent was to
demonstrate that AES can be as reliable as human raters, or more
so.
 1.2 Kaggle Problem Statement
The William and Flora Hewlett Foundation (Hewlett) is sponsoring
the Automated Student Assessment Prize (ASAP).  Hewlett is
appealing to data scientists and machine learning specialists to help
solve an important social problem.  We need fast, effective and
affordable solutions for automated grading of student-written
essays.
You are provided access to hand scored essays, so that you can
build, train and test scoring engines against a wide field of
competitors.  Your success depends upon how closely you can
deliver scores to those of human expert graders. 

1.3 Motivation

Today, many education companies and government agencies are

developing new forms of testing and grading methods, to assess the
new common core standards.  In this environment the need for
more sophisticated and affordable options is vital.  For example,
we know that essays are an important expression of academic
achievement, but they are expensive and time consuming for states
to grade them by hand.  So, we are frequently limited to multiple-
choice standardized tests.  We believe that automated scoring
systems can yield fast, effective and affordable solutions that
would allow states to introduce essays and other sophisticated
testing tools.  
We are going to use various machine learning models which can
take any essay and automatically output the grade of that essay.
Using feature extraction and ML models this task is being
automated.

2. Feature extraction and data pre-processing

2.1 Dataset
The data provided by Hewlett Foundation contain 12978 essays
which contains essays written by students from grade 7 to 10. Each
essay was graded by many graders out of which the scores of only
 graders are provided and their net score is also provided in the
dataset.

2.2 Input Features Considered

The essay provided cannot be directly used as an input to any of

machine learning models hence we need to extract some features
out of it. The features that we are considering are as follows:
i. Word Count
ii. Sentence Count
iii. Large word count
iv. Noun Count
v. Verb Count
vi. Pronoun Count
vii. Adjective Count
viii. Adverb Count
ix. Preposition Count
x. Comparative Adjective Count
xi. Personal Pronoun Count
xii. Plural Noun Count
xiii. Quotes Count
xiv. Word to Sentence Ratio
xv. Spelling Error Count
2.1.1 Word Count
We are using the Regex module present in python to separate
the words in the essay and then we count them.

2.1.2 Sentence Count

We are using sent_tokenize module present in nltk library to
tokenize all the sentences present and then we count the output
list obtained.

2.1.3 Large Word Count

We are using same method as before to obtain the list of words
in each essay and then run an if function to check for size.

2.1.4 Parts of Speech count

It includes all the procedures required for counting various parts
of speech present like – Noun, Verb, Pronoun etc. We are using
the pos_tag method of nltk module to find which part of the
parts of speech they belong to.
 2.1.4 Quotes Count
We are using the Regex module to find all the quotes and count
them

2.1.5 Spelling error count

We are using the SpellChecker method from pyspellschecker module
to do this.
Final input file obtained after pre-processing the essays