1

Spam Filtering Using Bayesian

Approach
Presented by: Nitin Kumar
2
References
What you need to know about Bayesian Spam filtering
http://email.about.com/cs/bayesianfilters/a/bayesian_filter_2.htm
A plan for Spam by Paul Graham
http://www.paulgraham.com/spam.html
Genetic Algorithm Tutorial
http://cs.felk.cvut.cz/%7Exobitko/ga/
Spam filtering using Bayesian Approach

3
Advantages of Bayesian Method
Bayesian approach is self adapting. It keeps learning
from the new spams.
Bayesian method takes whole message into account.
Bayesian method is easy to use and very accurate
(Claimed Accuracy Percentage is 97).
Bayesian approach is multi-lingual.
Reduces the number of false positives.
4
Project Goal
1. Getting a learning set of various Spam and Ham emails.
2. Parsing the individual mails to extract the words of
interest.
3. Implementing the Nave Bayesian Method.
5
Learning Set
The learning set will consist of:
2500 Ham emails & 4000 Spam emails
The testing set will consist of:
2700 Ham emails & 1400 Spam emails
Spam Mail archive: http://spamassassin.org/publiccorpus/

6
Parsing Words of Interest To Form
Tokens
Remove the common words (e.g. you, I, for, is, are, etc).
Strip off the HTML tags and punctuation marks.
Multiple occurrence of a word in a single mail should not
increase its count.
Token may be formed from a pair of words.
7
Nave Bayesian Algorithm
Nave Bayesian method is used for the learning process.
Analyze a mail to calculate its probability of being a Spam
using individual characteristic of words in the mail.
For each word in the mail, Calculate the following:
S(w)=(number of Spam emails containing the word)/(total
number of Spam emails)
H(w)=(number of Ham emails containing the word)/(total
number of Ham emails)
P(w)=S(w)/(S(w)+H(w))
P(w) can be interpreted as the probability that a randomly chosen
email containing the word w is Spam.
8
Exceptions
Say a word w =success appears only once and it is a
Spam email. Then the above formula calculates P(w)=1.
This doesnt mean that all future mails containing this
word will be considered as Spam. It will rather depend
upon its degree of belief. The Bayesian method allow us to
combine our intuitive background information with this
collected data.
Degree of belief f(w)= [(s*x)+(n*p(w))]/(s+n)
s=Assumed strength of the background information.
x= Assumed probability of the background information.
n= no of emails received containing word w.

9
Combining The Probabilities
Each email is represented by a set of probabilities.
Combining these individual probabilities gives the overall
indicator of spamminess.
Fishers Method:
H= Chi_inverse(-2*ln(Product of all(f(w)), 2*n)
S= Chi_inverse(-2*ln(Product of all(1-f(w)), 2*n)
I= [1+H-S]/2
Here, I is the Indicator of Spamminess.
10
Genetic Algorithm
A mail can be divided into three parts:
Body
From
Subject
Genetic Algorithm can be used to get an appropriate
weights say , and for body part, frompart and
subject part.
I
Final
= *I
Body
+ *I
From
+ *I
Subject
The overall accuracy is a function of , and . Genetic
Algorithm maximize the above function.
11
Result
A mail is declared as SPAM, if the value of I
Final
is greater
than a threshold value.