,erformance (valuation and mprovement in (ducation -ector using 'ayesian "lassification %lgorithm.

% -urvey
(.# 9akshmikanth., 8.K. %angareddy:
.

5.1. %tudent, Department of 2omputer %cience and $ngineering, *dichunchanagiri "nstitute of (echnology, 2hikmagalur, Karnataka (lkwodeyar;gmail.com) : *sst.5rofessor, Department of 2omputer %cience and $ngineering, *dichunchanagiri "nstitute of (echnology, 2hikmagalur, Karnataka (sbkurtakoti;gmail.com)

Abstract Nowadays there is an evaluation of educational systems and there is a great importance to the educational field. The amount of data stored in educational database is increasing rapidly. These databases contain hidden information for evaluation and improvement of students performance. The ability to predict a students performance is very important in educational environments. The feasible technique to achieve this prediction is Data Mining. The academic performance of a student is based upon diverse factors like personal, social, psychological and other environmental variables. n this paper, we have given an overview of different classification algorithms namely D!, "#.$, "%&T and 'ayesian, to predict students behavior. Keywords Data Mining, (ducational database, D!, "#.$, "%&T, 'ayesian algorithm.

several well'established areas of research including e' learning, adaptive hypermedia, intelligent tutoring systems, web mining and data mining etc. *s we know, large amount of data is stored in educational database, data mining is the process of discovering interesting knowledge from these large amounts of data stored in database, data warehouse or other information repositories. (o assist the low academic achievers in higher education the ob+ectives are -./0 (a) 1eneration of data source of predictive variables (b) "dentification of different factors, which effects a student)s learning behavior and performance during academic career (c) 2onstruction of a prediction model using classification data mining techni!ues on the basis of identified predictive variables (d) 3alidation of the developed model for higher education students studying in "ndian 4niversities or "nstitutions. 5redictive classification enhances the !uality of higher education system to increase number of loyal students to evaluate students6 data to study the main attributes that may affect the enrollment factor. * number of data mining techni!ues have already been done on educational data mining to improve the performance of students like 7egression, 1enetic algorithm, 8ayes classification, k'means clustering, associate rules, prediction etc. Data mining techni!ues can be used in educational field to enhance our understanding of learning process to focus on identifying, extracting and evaluating variables related to the learning process of students. 2lassification is one of the most fre!uently studied problems by data mining and machine learning researchers. "t consists of predicting the value of a categorical attribute based on the value of other attributes. 2lassification methods like decision trees, rule mining, 8ayesian network etc. can be applied on the educational data for predicting the students behavior, performance in examination etc. Decision tree is a flow' chart'like tree structure, where each internal node is denoted by rectangles and the leaf nodes are denoted by ovals. "t the most commonly used algorithm because of its ease of implementation and easier to understand compared to other classification algorithms. (he outcome of the decision tree predicted the number of students who

). NT&*D+"T *N Data mining sometimes is also called knowledge discovery in databases (KDD). Data mining can also be used to find the existing relationships and patterns. Data mining combines machine learning, statistics and visuali ation techni!ues to discover and extract knowledge. Data mining techni!ues are used to operate on large amount of data to discover hidden patterns and relationships helpful in decision making. "t can be used in educational field to enhance our understanding of learning process to focus on identifying, extracting and evaluating variables related to the learning process of students #ining in educational environment is called $ducational Data #ining. %tudent retention has become an indication of academic performance and enrollment management. &ne of the most useful data mining techni!ues for e'learning is classification. (he ability to predict a student)s performance is very important in educational environments. %tudents) academic performance is based upon diverse factors like personal, social, psychological and other environmental variables. * very promising tool to attain this ob+ective is the use of Data #ining. $ducation is an essential element for the betterment and progress of a country. "t makes the people of a country civili ed and well mannered. #ining in educational environment is called educational data mining. $ducational data mining is an upcoming field related to

are likely to pass, fail or promoted to next year. Decision tree can be constructed relatively fast compared to other methods of classification. (rees can be easily converted into %<9 statements that can be used to access databases efficiently. Decision tree classifiers obtain similar and sometimes better accuracy when compared with other classification methods. Decision tree algorithm can be implemented in a serial or parallel fashion based on the volume of data, memory space available on the computer resource and scalability of the algorithm. (he 2=.>, "D?, 2*7( and @aAve 8ayesian, decision tree algorithms are applied on the data of students to predict their performance. (he prediction of students6 performance with high accuracy is more beneficial for identifying low academic achievements students at the beginning. (o improve their performance the teacher will monitor the students6 performance carefully. /. 0iterature -urvey 4mesh Kumar 5andey, 8ri+esh Kumar 8hardwa+ and %aurabh pal -:/ in their paper Data #ining as a (orch 8earer in $ducation %ector, discussed the different type of researches used in the education sector using data mining. %tudents, educators and academic responsible person can use these findings to improve the !uality of education. 8ri+esh Kumar 8hardwa+ and %aurabh 5al -?/ in their paper Data #ining0 * prediction for performance improvement using classification, Discussed an experimental methodology those are used to generate a database, where the raw data was preprocessed in terms of filling up missing values, transforming values in one form into another and relevant attribute selection. (hey only proved that the academic performances of the students are not always depending on their own effort. 8oumedyen %hanna!, Busupov 7afael and 3. *lexandro -=/ in their paper %tudent 7elationship in Cigher $ducation 4sing Data #ining (echni!ues, aimed to improve the current trends in the higher education systems to understand from the system which factors might create loyal students. (heir research work concentrated only on the number of students will enroll in the upcoming years.

implementation and easier to understand compared to ther classification algorithms. (he outcome of the decision tree predicted the number of students who are likely to pass, fail or what will be the marks one can score. Cere are some of the interesting things about the decision tree. "t divides up the data on each branch point without losing any of the data (the number of total records in a given parent node is e!ual to the sum of the records contained in its two children). "t is pretty easy to understand how the model is being built (in contrast to the models from neural networks or from standard statistics). "t can be constructed relatively fast compared to other methods of classification. (rees can be easily converted into %<9 statements that can be used to access database efficiently. !.). D! %lgorithm "terative Dichotomiser?, it is a decision tree algorithm introduced in .DEF by <uinlan 7oss. "t is based on Cunts algorithm. (he tree is constructed in two phases. (he two phases are tree building and pruning. 5runing means to change the model by deleting the child nodes of a branch node. 5runed node is regarded as a leaf node, leaf node cannot be pruned. "t only accepts categorical attributes in building a tree model. "t does not give accurate result when there is noise. (o remove the noise pre'processing techni!ue has to be used. (o build decision tree -D/, information gain is calculated for each and every attribute and select the attribute with the highest information gain to designate as a root node. 9abel the attribute as a root node and the possible values of the attribute are represented as arcs. (hen all possible outcome instances are tested to check whether they are falling under the same class or not. "f all the instances are falling under the same class, the node is represented with single class name, otherwise choose the splitting attribute to classify the instances. 2ontinuous attributes can be handled using the "D? algorithm by discreti ing or directly, by considering the values to find the best split point by taking a threshold on the attribute values. "D? does not support pruning. Data understanding Data 5reprocessing Different algorithms

!. M(T1*D*0*23 Decision tree is a predictive model that, as its name implies, can be viewed as a tree. %pecifically, each branch of the tree is a classification !uestion, and the leaves of the tree are partitions of the data set with their classification. Decision tree is a flow'chart'like tree structure, where each internal node is denoted by rectangles and the leaf nodes are denoted by ovals. "t is the most commonly used algorithm because of its ease of

$valuating algorithms Data base Deployment

Gig...Data #ining algorithms.

(echni!ue

for

evaluating

different

!./. "#.$ %lgorithm (his algorithm is a successor to "D? developed by <uinlan 7oss. "t is also based on Cunt)s algorithm. 2=.> handles both categorical and continuous attributes to build a decision tree. "n order to handle continuous attributes, 2=.> splits the attribute values into two partitions based on the selected threshold such that all the values above the threshold as one child and the remaining as another child -E/. "t also handles missing attribute values. 2=.> uses 1ain 7atio as an attribute selection measure to build a decision tree. "t removes the biasness of information gain when there are many outcome values of an attribute. *t first, calculate the gain ratio of each attribute. (he root node will be the attribute whose gain ratio is maximum. 2=.> uses pessimistic pruning to remove unnecessary branches in the decision tree to improve the accuracy of classification. !./.). 0imitations of "#.$ %lgorithm. %ome of the limitations of 2=.> *lgorithm are listed below0 (mpty branches0 2onstructing tree with meaningful value is one of the crucial steps for rule generation by 2=.> algorithm. 7a+ Kumar and D7. 7a+esh 3erma ->/ in their experiment, they have found many nodes with ero values or close to ero values. (hese values neither contribute to generate rules nor help to construct any class for classification task. 7ather it makes the tree bigger and more complex. nsignificant branches0 @umbers of selected discrete attributes create e!ual number of potential branches to build a decision tree. 8ut all of them are not significant for classification task. (hese insignificant branches not only reduce the usability of decision *ver fitting0 &ver fitting happens when algorithm model picks up data with uncommon characteristics. (his cause many fragmentations is the process distribution. %tatistically insignificant nodes with very few samples are known as fragmentations -F/. 1enerally 2=.> algorithm constructs trees and grows it branches H+ust deep enough to perfectly classify the training examples). (his strategy performs well with noise free data. 8ut most of the time this approach over fits the training examples with noisy data. !.!. "%&T

2*7( stands for 2lassification and 7egression (rees introduced by 8reiman. "t is also based on Cunt)s algorithm. 2*7( handles both categorical and continuous attributes to build a decision tree. "t handles missing values. 2*7( uses 1ini "ndex as an attribute selection measure to build a decision tree .4nlike "D? and 2=.> algorithms, 2*7( produces binary splits. Cence, it produces binary trees. 1ini "ndex measure does not use probabilistic assumptions like "D?, 2=.>. 2*7( uses cost complexity pruning to remove the unreliable branches from the decision tree to improve the accuracy. (he pruning process is completed in one pass through the tree 2=.>)s tree'construction algorithm differs in several respects from 2*7( -I/ (ests in 2*7( are always binary, but 2=.> allows two or more outcomes. 2*7( uses the 1ini diversity index to rank tests, whereas 2=.> uses information'based criteria. 2*7( prunes trees using a cost'complexity model whose parameters are estimated by cross' validation. 2=.> uses a single'pass algorithm derived from binomial confidence limits.

!.#. Na4ve 'ayes %lgorithm * @aive 8ayes algorithm is one of the most effective methods in the field of text classification, but only in the large training sample set can it get a more accurate result. (he re!uirement of a large number of samples not only brings heavy work for previous manual classification, but also puts forward a higher re!uest for storage and computing resources during the computer post'processing -.J/. @aive 8ayes classifier is a simple probabilistic classifier based on applying 8ayes6 theorem (from 8ayesian statistics) with strong (naive) independence assumptions. * more descriptive term for the underlying probability model would be Kindependent feature modelK. "n simple terms, a naive 8ayes classifier assumes that the presence (or absence) of a particular feature of a class is unrelated to the presence (or absence) of any other feature. Gor example, a fruit may be considered to be an apple if it is red, round, and about =K in diameter. $ven if these features depend on each other or upon the existence of the other features, a naive 8ayes classifier considers all of these properties to independently contribute to the probability that this fruit is an apple. Depending on the precise nature of the probability model, naive 8ayes classifiers can be trained very efficiently in a supervised learning setting. "n many practical applications, parameter estimation for naive 8ayes models uses the method of maximum likelihood, in other words, one can work with the naive 8ayes model without believing in 8ayesian probability or using any 8ayesian methods. "n spite of their naive design and apparently over'simplified

assumptions, naive 8ayes classifiers have worked !uite well in many complex real'world situations. "n :JJ=, analysis of the 8ayesian classification problem has shown that there are some theoretical reasons for the apparently unreasonable efficiency of naive 8ayes classifiers. %till, a comprehensive comparison with other classification methods in :JJF showed that 8ayes classification is outperformed by more current approaches, such as boosted trees or random forests. *n advantage of the naive 8ayes classifier is that it only re!uires a small amount of training data to estimate the parameters (means and variances of the variables) necessary for classification. 8ecause independent variables are assumed, only the variances of the variables for each class need to be determined and not the entire covariance matrix. 2lassification is enduring to be one of the most researched problems due to continuously'increasing amount of electronic documents and digital data. @aive 8ayes is an effective and a simple classifier for data mining tasks -../. (he naive 8ayes classifier6s beauty is in its simplicity, computational efficiency, and good classification performance. "n fact, it often outperforms more sophisticated classifiers even when the underlying assumption of (conditionally) independent predictors is far from true. (his advantage is especially pronounced when the number of predictors is very large. (hree issues should be kept in mind, however. Girst, the naive 8ayes classifier re!uires a very large number of records to obtain good results. %econd, where a predictor category is not present in the training data, naive 8ayes assumes that a new record with that category of the predictor has ero probability. (his can be a problem if this rare predictor value is important. !.#.). %dvantages of Na4ve 'ayes Gast to train (single scan). Gast to classify. @ot sensitive to irrelevant features. Candles real and discrete data. Candles streaming data well. #. "*N"0+- *N (hrough this survey paper, the 4niversity will have the ability to predict the students) loyalty (numbers of enrolled (students) so they can manage and prepare necessary resources for the new enrolled students. (his helps the teacher to improve the performance of students) and decides on those students who need special attention. (hat helps in reducing the failing ratio and to take actions at right time. 8ayesian classification method is used on student database to predict the students division on the basis of previous year database. Data mining extracts hidden information with the help different mining techni!ue. 5rediction, results and recommendation are provided by this information, which help the user to take further decision.

&(5(&(N"(-./ #. %ukanya, %. 8iruntha, Dr.%. Karthik and (. Kalaikumaran, LData Mining: Performance Improvement in Education Sector using Classification and Clustering Algorit m! International Conference on Computing and
Control Engineering (ICCCE 2012), 12 & 13 April, 2012

-:/ 4mesh Kumar 5andey, 8ri+esh Kumar 8hardwa+, %aurabh pal LData Mining as a "orc #earer in Education Sector! "nternational Mournal of *dvanced 7esearch in 2omputer %cience and %oftware $ngineering Volume 2, Issue 1, Manuary :J.: -?/ 8ri+esh Kumar 8hardwa+, %aurabh 5al $Data Mining: A prediction for performance improvement using classification! ("M2%"%) "nternational Mournal of 2omputer %cience and "nformation %ecurity, 3ol. D, @o. =, *pril :J..

-=/ 8oumedyen %hanna!, Busupov 7afael, 3. *lexandro $Student %elations ip in &ig er Education 'sing Data Mining "ec ni(ues! 3ol. .J "ssue .. (3er. ..J) &ctober :J.J ->/ 7a+ Kumar. Dr. 7a+esh 3erma, $Classification Algorit m for Data Mining: A Survey!. International Journal of
Inno ations in Engineering and !ec"nolog# (IJIE!) Vol$ 1 Issue 2 August 2012.

-F/ M. Can and #. Kamber, LData Mining: Concepts and "ec ni(ues!, #organ Kaufmann 5ublish, :JJ.. -I/ 8reiman 9, Griedman MC, &lshen 7*, %tone 2M (.DE=) LClassification and regression trees)*adswort +! -E/ %ur+eet Kumar Badav, 8ri+esh 8haradwa+, %aurabh 5al , %Mining Education Data to Predict Student,s %etention: A comparative StudyN, "M2%"%) "nternational Mournal of 2omputer %cience and "nformation %ecurity, 3ol. .J, @o. :, :J.:. -D/ D.9avanya, Dr.K.4sha 7ani, LPerformance Evaluation of Decision "ree Classifiers on Medical DatasetsN,

"nternational Mournal of 2omputer *pplications (JDI> EEEI),3olume :F@o.=, Muly :J...

-.J/

Buguang Cuang 8ei+ing 4niv. of 5osts O, (elecommunication., 8ei+ing, china 9ei 9i $-a.ve
#ayes classification algorit m based on small sample set!)

2loud 2omputing and "ntelligence %ystems (22"%), :J.. "$$$ "nternational 2onference.

-../ #eena, #.M. 2handran, K.7. Dept. of 2%$, 5%1 2oll. of (echnol., 2oimbatore, "ndia L-a.ve #ayes te/t classification wit positive features selected by statistical met od! *dvanced 2omputing, :JJD. "2*2 :JJD.

Girst "nternational 2onference.