International Journal of Computer Science and Management Research Vol 1 Issue 4 November 2012

ISSN 2278-733X

Student Performance Prediction by Using Data

Mining Classification Algorithms
Dorina Kabakchieva
University of National and World Economy
Sofia, Bulgaria

Abstract— This paper presents the results from data mining Data mining, generally defined as the process of
research, performed at one of the famous and prestigious discovering meaningful patterns in large quantities of data,
Bulgarian universities, with the main goal to reveal the high offers a great variety of techniques, methods and tools for
potential of data mining applications for university management thorough analysis of available data in various fields. The
and to contribute to more efficient university enrolment
implementation of data mining in the educational sector,
campaigns and to attracting the most desirable students. The
research is focused on the development of data mining models for recently defined as “educational data mining” (EDM) [1], is a
predicting student performance, based on their personal, pre- new stream in the data mining research field. The educational
university and university-performance characteristics. The data mining research community is constantly growing,
dataset used for the research purposes includes data about starting by organizing workshops since 2004, then conducting
students admitted to the university in three consecutive years. an annual International Conference on EDM beginning since
Several well known data mining classification algorithms, 2008, and now already having a Journal on EDM (the first
including a rule learner, a decision tree classifier, a neural issue being published in October 2009).
network and a Nearest Neighbour classifier, are applied on the There are already a large number of research papers
dataset. The performance of these algorithms is analyzed and
discussing various problems within the higher education
compared.
sector and providing examples for successful solutions
reached by using data mining. Extensive literature reviews of
Keywords— Educational Data Mining, Student Profiling,
the EDM research field are provided by Romero and Ventura
Predicting Student Performance, Classification
in 2007 [1], covering the research efforts in the area between
I. INTRODUCTION 1995 and 2005, and by Baker and Yacef in 2009 [2], for the
Universities today, similar to business organizations, are period after 2005. The problems that are most often attracting
the attention of researchers and becoming the reasons for
operating in a very dynamic and strongly competitive
initiating data mining projects at higher education institutions
environment. The education globalization leads to more and
are focused mainly on retention of students (by better
better opportunities for students to receive high quality
education at institutions all over the world. Universities are knowing their peculiarities and needs, and by providing
proper support in advance), more effective targeted marketing,
confronted with a severe competition among each other,
improving institutional efficiency, and alumni management.
trying to attract the most appropriate students who will
The performed research work, presented in this paper,
successfully pass through the university educational process,
focuses on the development of data mining models for
and making efforts to cope with student retention. University
predicting student performance by using four data mining
management is very often forced to take quickly important
algorithms for classification – a Rule Learner, a Decision tree
decisions, and therefore timely and high quality information is
needed. algorithm, a Neural network, and a K-Nearest Neighbour
Modern universities are collecting large volumes of data method. It is a continuation of previous research, carried out
with the same dataset and with similar data mining algorithms,
referring to their students, the organization and management
but for a different format of the predicted target variable. The
of the educational process, and other managerial issues.
achieved results from the performed research, using a target
However, the available data is usually used for producing
simple queries and traditional reports that are rarely reaching variable with five distinct values – Bad, Average, Good, Very
Good, and Excellent, are previously published in [3]. Current
the right people at the right time for making informed
research is implemented for a binary target variable.
decisions. Moreover, much of the data remains unused due to
The rest of the paper is organized in four sections. The
the inability of the university administration to handle it
research motivation and the state-of-the-art are presented in
because of the large volumes and the increasing complexity.
Section 2. The adopted methodological approach, the
Advanced information technologies have to be introduced to
experimentation data selection and pre-processing are
effectively transform available data into information and
knowledge to support decision making. described in Section 3. The obtained results from the
application of the selected data mining algorithms are

Dorina Kabakchieva 686 www.ijcsmr.org

International Journal of Computer Science and Management Research Vol 1 Issue 4 November 2012
ISSN 2278-733X

presented in Section 4. The paper concludes with a summary students, identifying the most important factors for student
of the achievements and discussion of further research. success and developing a profile of the typical successful and
unsuccessful students. Ramaswami et al. in [17] focus on
II. RESEARCH MOTIVATION AND STATE OF THE ART developing predictive data mining model to identify the slow
The rationale behind the research work described in this learners and study the influence of the dominant factors on
paper is based on the great potential that is seen in using data their academic performance, using the popular CHAID
mining methods and techniques for effective usage of decision tree algorithm.
university data. The discussions with high level managers and
administrators of a famous and prestigious Bulgarian III. RESEARCH APPROACH, DATA SELECTION AND PRE-
university have lead to the identification of existing needs for PROCESSING
better knowing the students and performing more effective The performed research is based on the CRISP-DM (Cross-
university marketing policy. Industry Standard Process for Data Mining) model, a non-
The literature review reveals that these problems have been propriety, freely available, and application-neutral standard
of interest for various researchers during the last few years. for data mining project implementation, widely used by
The development of data mining models for predicting student researchers in the data mining field during the last ten years
performance at various levels, and comparison of those [18]. It is a cyclic approach, including six main phases –
models, are discussed in a number of research papers. In 2000 Business understanding, Data understanding, Data preparation,
the results of a study are described [4] aimed at finding weak Modelling, Evaluation and Deployment, with a number of
students and involving them in additional courses for internal feedback loops between the phases, resulting from the
advanced support by extracting association rules from data. very complex non-linear nature of the data mining process and
The retention of students is a problem discussed also by Luan, ensuring the achievement of consistent and reliable results.
who implemented clustering, neural network and decisions The open source software WEKA, offering a wide range of
tree methods to predict the students in risk of failure [5], [6]. classification methods for data mining [19], is used as a data
Data mining methods are implemented for modeling online mining tool for the research implementation.
student grades [7], using three classification approaches used First of all, the business problem is identified – it is the
(binary: pass/fail; 3-level: low, middle, high; and 9-level: from growing need of university management for better knowing
1 - lowest grade to 9 - highest score). Kotsiantis et al. [8] also the university students and predicting their performance in
deal with predicting student performance, recognizing order to approach in the marketing campaigns exactly those
dropout-prone students based on demographic characteristics students that will be most successful in the university
(e.g. sex, age, marital status) and performance attributes (e.g. education process. The stated business problem is transformed
mark in a given assignment). Pardos et al. [9] use data from an into a data mining task – the task for classifying students into
online tutoring system for teaching Math and implement a two categories – successful and unsuccessful, by analysing the
regression approach for predicting the math test score based available student data with selected data mining methods for
on individual skills. Superby et al. [10] predict students at risk classification.
of drop-out, determining factors influencing the achievement The next phase in the research implementation includes the
of the first-year university students, classifying students into data selection and pre-processing, crucial activities within
three classes – low-risk, medium-risk and high-risk, using each data mining project, highly influencing the quality of the
Decision trees, Random forest method, Neural networks and final results. After studying the application process for student
Linear discriminant analysis. Vandamme et al. [11] also deal enrollment at the University and reviewing the procedures for
with early identification of three categories of students: low, collecting and storing data about the academic performance of
medium and high-risk students using Decision trees, Neural the university students, it is established that the university data
networks and Linear discriminant analysis. Cortez and Silva is generally organized in two databases. All the data related to
in [12] attempt to predict student failure by applying and the university admission campaigns is stored in the University
comparing four data mining algorithms, Decision Tree, Admission database, including personal data of university
Random Forest, Neural Network and Support Vector Machine. applicants (names, addresses, secondary education scores,
The implementation of predictive modelling for maximizing selected admission exams, etc.), data about the organization
student recruitment and retention is presented in the study of and performance of the admission exams, scores achieved by
Noel-Levitz [13]. The development of enrolment prediction the applicants at the admission exams, data related to the final
models based on student admissions data by applying different classification of applicants and student admission, etc. All the
data mining methods (Decision trees, Rule induction, Feature data concerning student performance at the university is stored
subset selection) is the research focus of Nandeshwar [14]. in the University Students Performance database, including
Dekker et al. [15] focus on predicting students drop out. student personal and administrative data, the grades achieved
Kovačić in [16] uses data mining techniques (feature selection at the exams on the different subjects, etc. For the purposes of
and classification trees) to explore the socio-demographic the study, student data from both databases is carefully
variables (age, gender, ethnicity, education, work status, and selected, extracted and combined in a new flat file (in this case
disability) and study environment (course programme and Excel file) used for the data mining analysis in the WEKA
course block) that may influence persistence or dropout of software tool.

Dorina Kabakchieva 687 www.ijcsmr.org

International Journal of Computer Science and Management Research Vol 1 Issue 4 November 2012
ISSN 2278-733X

The provided flat file contains data about 10330 students TABLE I
that have been enrolled as university students during the FINAL DATASET USED FOR THE DATA MINING ANALYSIS
period between 2007 and 2009, described by 20 parameters, Type of Data Attribute Name Attribute
Type
Values

including gender, birth year, birth place, living place and Personal
Data
Gender Nom Male (49%), Female (51%)
Age Nom 29 distinct values (17-43,46,48,53) (18-21–95%, 19–78%)
country, type of previous education, profile and place of BirthYear Nom 29 distinct values
Pre-University PlacePrevEdu Nom 7 distinct values (Sofia, NE, NC, NW, SE, SC, SW)
previous education, total score from previous education, Data ProfilePrevEdu Nom 9 distinct values (Language, Natural_Math, Humanitarian,
university admittance exam and achieved score, total Economics, Technological, Business_Management, Arts,
Sports, General)
university score at the end of the first year, etc. The data is ScorePrevEdu Num 3.40-6.00
carefully studied and subjected to many transformations. AdmissionYear
AdmissionExam
Nom
Nom
2007, 2008, 2009
5 distinct values (BG,Math,Geography, History,Economics)
Some of the parameters are removed, e.g. the “Birth place” AdmissionExamScore Num 0.00-6.00
AdmissionScore Num 0.00-35.98
and the “Place of living” fields containing data that is of no University UnivSpecialtyName Nom 10 distinct values
Data
interest to the research, the “Country” field containing only CurrentSemester Nom 1-10
NumFailures Nom 0-12 (0 – 86,4%)
one value (Bulgaria) because the data concerns only Bulgarian StudentClass Nom Weak (5340), Strong (4727)
students, the “Type of previous education” field which has
only one value as well because concerns only students who Most of the attributes (11), including the predicted class
have finished secondary education. Some of the variables, variable, are nominal variables accepting a certain number of
containing important data for the research, are text fields distinct values, and only 3 of the attributes are numeric
where free text is being entered at the data collection stage. variables.
Therefore, these variables are processed and turned into
nominal variables with a limited number of distinct values. IV. ACHIEVED RESULTS FROM THE DATA MINING
Such a parameter is the “Profile of the secondary education” ALGORITHMS IMPLEMENTATION
which is turned into a nominal variable with 9 distinct values During the Modeling Phase, the algorithms for building
(e.g. language, math, natural sciences, economics, technical, models that would classify the students into the two classes –
sports, arts, etc.). The “Place of secondary education” field is Weak and Strong, depending on their university performance
also preprocessed and transformed into a nominal variable and based on the student pre-university data, are considered
with 7 distinct values, corresponding to the capital city and the and selected. Popular WEKA classifiers (with their default
6 geographic regions in Bulgaria – North-East, North-Central, settings unless specified otherwise) are used in the
North-West, South-East, South-Central, and South-West. A experimental study, including a rule learner (OneR), a
new numeric variable is added – the “Student age at common decision tree algorithm C4.5 (J48), a neural network
enrollment”, calculated by subtracting the values contained in (MultiLayer Perceptron), and a Nearest Neighbour algorithm
the “Admission year” and “Birth year” fields. Another (IBk). These classification algorithms are selected because
important operation during the preprocessing phase is also the they are very often used for research purposes and have
transformation of some variables from numeric to nominal potential to yield good results. Moreover, they use different
(e.g. age, admission year, current semester, total university approaches for generating the classification models, which
score, etc.) because they are much more informative when increases the chances for finding a prediction model with high
interpreted with their nominal values. The data is also being classification accuracy. The OneR Rule Learner algorithm
studied for missing values, which are very few and could not produces a one-level decision tree expressed in the form of a
affect the results, and obvious mistakes, which are corrected. set of rules that all test one particular attribute – the minimum-
Essentially, the challenge in the presented data mining classification-error attribute. It is a simple, cheap method that
research is to predict the student university performance based often produces good rules with high accuracy. The Decision
on the available student pre-university and university Tree algorithms generate models in the form of a tree-like
performance data. This is achieved by solving a classification structure, which starts from root attributes and ends with leaf
data mining task. A binary categorical target variable is nodes, describing the relationship among attributes and the
constructed, based on the original numeric parameter relative importance of attributes. They represent rules which
“University average score” (the average numeric score could easily be understood and interpreted by users, do not
achieved by the students at the end of the first year at the require complex data preparation, and perform well for
University). The predicted variable has two distinct values, numerical and categorical variables. Neural networks produce
corresponding to the two classes in which the students are classification models in the form of a mathematical model,
classified – Weak and Strong. Since a six-level scale is used in consisting of interconnected computational elements (neurons)
the Bulgarian educational system for evaluation of student and processing information using a connectionist approach to
performance at schools and universities, the students with computation. They are used to model complex relationships
average university score that is lower than 4.50 are classified between inputs and outputs and very often yield very good
as “Weak”, and the students with average university score results. The K-Nearest Neighbor algorithm (k-NN) is a
equal or higher than 4.50 are classified as “Strong”. method for classifying instances based on measuring the
The final dataset, on which the selected classification data distance between the classified instance and the closest
mining algorithms are applied, contains 10067 instances and training examples in the feature space. It is easily understood
14 attributes (summarized in Table 1).

Dorina Kabakchieva 688 www.ijcsmr.org

International Journal of Computer Science and Management Research Vol 1 Issue 4 November 2012
ISSN 2278-733X

by users, often provides good classification results and correctly classified instances, Fig.2a) is achieved for the
performs well for large datasets. Neural Network algorithm – 73.59%. The Neural Network
The selected data mining algorithms are applied to the model is also the only model that predicts the “Strong” class
dataset using the holdout method (WEKA “Percentage Split” with higher accuracy (TP Rate=77%) than the “Weak” class
test option, 66%/34%), as shown on Fig.1. The dataset is (TP Rate=70%), which means that this model could most
divided into 3 parts and, each time an algorithm is run, 2/3 of successfully be used to predict the strong students based on
the data is used for training of the classification model and 1/3 their pre-university and university-performance characteristics.
of the data is used for testing and evaluation of the model. The disadvantages of that model are its complexity and the
difficult understanding and interpretation by users. The
Modeling models, generated with the other three algorithms, are
predicting the “Weak” class with higher accuracy than the
Data Mining
Training Data
Algorithm “Strong” class, and they could be used for early identification
2/3
Dataset of students in risk that might need additional support. The
3/3
Decision Tree classification model also reveals high accuracy
of prediction - 72.74%. The advantages of this model are that
Data Exploration
Classification it is easily interpretable because it produces a set of
Testing Data Model
and Data Pre-Processing 1/3 understandable rules, and that it is working well with both,
nominal and numeric variables. The attributes, which appear
Results
at the upper part of the decision tree and therefore are
Model Evaluation
considered most informative for the instance distribution into
the two classes, are the “Number of Failures” and the
Fig. 1 Data Mining Algorithm Implementation - Classification Model
“Admission Score”. The K-NN model provides 70.5%
The results from the evaluation of the classification models accuracy of classification, working with similar accuracies for
generated with the selected four data mining algorithms are both classes – Weak and Strong. The OneR classifier is the
presented in Table 2. least accurate, performing better for the Weak class, as the
TABLE III
Decision Tree classifier. The OneR algorithm uses the
ACHIEVED RESULTS FROM THE DATA MINING ALGORITHM IMPLEMENTATION “Admission Score” attribute for the classification which once
Data Mining Rule Learner (OneR) Decision Tree (J48) Neural Network K-Nearest Neighbour
again proves that the Admission Score parameter is very
Algorithms (MultiLayer Perceptron -
1 hidden layer with 7
(IBk, k=50)
informative for recognizing strong and weak students.
neurons)
Evaluation Weak Strong Weigh. Weak Strong Weigh. Weak Strong Weigh. Weak Strong Weigh.
Parameters Av. Av. Av. Av.
Corr. Classified 67.4554% 72.7432% 73.5904% 70.47 0.9
Instances 0.8
Kappa Statistic and ROC Area

Incorr. Classified 32.5446% 27.2568% 26.4096% 29.53 0.823

Instances 0.7 0.784 0.784
Kappa Statistic 0.3439 0.4524 0.4730 0.4085
TP Rate 0.73 0.61 0.68 0.75 0.70 0.73 0.70 0.77 0.74 0.71 0.70 0.71 0.6 0.671
FP Rate 0.39 0.27 0.33 0.30 0.25 0.28 0.23 0.30 0.26 0.30 0.29 0.30 0.5
Precision 0.68 0.67 0.67 0.74 0.72 0.73 0.78 0.70 0.74 0.73 0.68 0.71
Recall 0.73 0.61 0.68 0.75 0.70 0.73 0.70 0.77 0.74 0.71 0.70 0.71 0.4 0.473
0.4524 Kappa Statistic
F-Measure 0.70 0.64 0.67 0.75 0.71 0.73 0.74 0.73 0.74 0.72 0.69 0.71 0.4085
0.3
ROC Area 0.67 0.67 0.67 0.78 0.78 0.78 0.82 0.82 0.82 0.78 0.78 0.78 0.3439 ROC Area
0.2
0.1

The four classification models, generated with the selected 0

1R J48 NN kNN
data mining algorithms, are compared by using the following
Classifiers
evaluation measures: % of correctly/incorrectly classified
instances, Kappa Statistic, True Positive (TP) and False Fig. 3B Classifiers Accuracy Comparison
Positive (FP) Rates, Precision, Recall, F-Measure and ROC
The results for the Kappa Statistic (Fig.2b), an index that
Area. These are well known measures for evaluation of data
compares correct classifications against chance classifications
mining models for classification.
and taking values in the range from -1 for complete
100 disagreement, to 1 for perfect agreement, also reveal that the
% Correctly Classified Instances

90 Neural Network model outperforms the other three

80
70 classification models with the maximum achieved value of
60 0.473. The ROC curve plots the true positives against the false
Accuracy,

50 Accuracy
40 positives and the area under the curve represents the accuracy
30 Accuracy-Weak
20 Accuracy-Strong
of the model – the larger the area, the more accurate the model.
10 The achieved results for three of the generated classification
0
models (Fig.2b), the Neural Network, the Decision Tree and
1R J48 NN kNN
the k-NN, reveal values of the ROC Area above 0.7, highest
Classifiers
for the Neural Network Model - 0.82, which means that all the
Fig. 2A Classifiers Accuracy Comparison models are reliable for prediction (if the ROC area is below
0.5, random guesses outperform the model).
The results for the classification model comparison are
presented on Fig.2. The highest classification accuracy (% of

Dorina Kabakchieva 689 www.ijcsmr.org

International Journal of Computer Science and Management Research Vol 1 Issue 4 November 2012
ISSN 2278-733X

V. CONCLUSIONS held at the 8th International Conference on Intelligent Tutoring

Systems (ITS2006), June 26, 2006, Taiwan.
The classification models, generated by applying the [10] Superby, J. Vandamme, J., Meskens, N. (2006). Determination of
selected four data mining algorithms – OneR Rule Learner, factors influencing the achievement of the first-year university students
Decision Tree, Neural Network and K-Nearest Neighbour, on using data mining methods. Proceedings of the Workshop on
Educational Data Mining at the 8th International Conference on
the available and carefully pre-processed student data, reveal Intelligent Tutoring Systems (ITS 2006). Jhongli, Taiwan, pp37-44.
classification accuracy between 67.46% and 73.59%. The [11] Vandamme, J., Meskens, N., Superby, J. (2007). Predicting Academic
highest accuracy is achieved for the Neural Network model Performance by Data Mining Methods. Education Economics, 15(4),
(73.59%), followed by the Decision Tree model (72.74%) and pp405-419.
[12] Cortez, P., Silva, A. (2008). Using Data Mining to Predict Secondary
the k-NN model (70.49%). The Neural Network model School Student Performance. EUROSIS, A. Brito and J. Teixeira
predicts with higher accuracy the “Strong” class, while the (Eds.), 2008, pp.5-12.
other three models perform better for the “Weak” class. The [13] Noel-Levitz White Paper (2008). Qualifying Enrollment Success:
data attributes related to the students’ University Admission Maximizing Student Recruitment and Retention Through Predictive
Modeling. Noel-Levitz, Inc., 2008.
Score and Number of Failures at the first-year university [14] Nandeshwar, A., Chaudhari, S. (2009). Enrollment Prediction Models
exams are among the factors influencing most the Using Data Mining. Available at: http://nandeshwar.info/wp-
classification process. content/uploads/2008/11/DMWVU_Project.pdf
The presented results will be compared to previous results, [15] Dekker, G., Pechenizkiy, M., Vleeshouwers, J. (2009). Predicting
Students Drop Out: A Case Study. Conference Proceedings of the 2nd
achieved for the same dataset but for a different format of the International Conference on Educational Data Mining (EDM’09), 1-3
predicted target variable (a nominal variable with five distinct July 2009, Cordoba, Spain, pp.41-50.
values – Bad, Average, Good, Very Good, and Excellent). The [16] Kovačić, Z. (2010). Early Prediction of Student Success: Mining
results and conclusions will also help to define the further Students Enrolment Data. Proceedings of Informing Science & IT
Education Conference (InSITE) 2010, pp.647-665.
steps and directions for continuing the university data mining [17] Ramaswami, M., Bhaskaran, R. (2010). A CHAID Based Performance
research, including possible transformations of the dataset, Prediction Model in Educational Data Mining. IJCSI International
adding new data, tuning the classification algorithms’ Journal of Computer Science Issues, Vol. 7, Issue 1, No.1, January
parameters, etc., in order to achieve better prediction. 2010, pp.10-18.
[18] Chapman, P., et al. (2000). CRISP-DM 1.0: Step-by-step data mining
Recommendations will also be provided to the university guide. 2000 SPSS Inc. CRISPWP-0800. Available at:
management, concerning the sufficiency and availability of http://www.crisp-dm.org/CRISPWP-0800.pdf
university data, and related to the improvement of the data [19] Witten, I., Frank, E. (2005). Data Mining: Practical Machine Learning
collection process. Tools and Techniques. Morgan Kaufmann Publishers, Elsevier Inc.
2005.
REFERENCES
[1] Romero, C., Ventura, S. (2007). Educational Data Mining: A Survey
from 1995 to 2005. Expert Systems with Applications 33, 2007,
pp.135-146.
[2] Baker, R., Yacef, K. (2009). The State of Educational Data mining in
2009: A Review and Future Visions. Journal of Educational Data
Mining, Vol.1, Issue 1, Oct. 2009, pp.3-17.
[3] Kabakchieva, D., Stefanova, K., Kisimov, V. (2011). Analyzing
University Data for Determining Student Profiles and Predicting
Performance. Conference Proceedings of the 4th International
Conference on Educational Data Mining (EDM 2011), 6-8 July 2011,
Eindhoven, The Netherlands, pp.347-348.
[4] Ma, Y., Liu, B., Wong, C. K., Yu, P. S., Lee, S. M. (2000). Targeting
the right students using data mining. Proceedings of the sixth ACM
SIGKDD international conference on Knowledge discovery and data
mining, Boston, pp 457-464.
[5] Luan, J. (2002). Data Mining and Its Applications in Higher Education.
New Directions for Institutional Research, Special Issue titled
Knowledge Management: Building a Competitive Advantage in Higher
Education, Vol. 2002, Iss.113, pp.17–36.
[6] Luan, J. (2004). Data Mining Applications in Higher Education. SPSS
Executive Report, SPSS Inc.
[7] Minaeli-Bidgoli, B., Kashy, D., Kortemeyer, G., Punch, W. (2003).
Predicting Student Performance: An Application of Data Mining
Methods with the Educational Web-Based System LON-CAPA. 33rd
ASEE/IEEE Frontiers in Education Conference, 5-8 Nov 2003,
Boulder, CO.
[8] Kotsiantis, S., Pierrakeas, C., Pintelas, P. (2004). Prediction of
Student’s Performance in Distance Learning Using Machine Learning
Techniques. Applied Artificial Intelligence, Vol. 18, No. 5, 2004, pp.
411-426.
[9] Pardos Z., Heffernan N., Anderson B., and Heffernan C. (2006). Using
Fine-Grained Skill Models to Fit Student Performance with Bayesian
Networks. In Proceedings of the Workshop in Educational Data Mining

Dorina Kabakchieva 690 www.ijcsmr.org