Volume 4, Issue 4, April – 2019 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Modernistic Approach to Clustering Algorithms

Ateeq ur Rehman Abirami T
MCA (IT Infrastructure management services) Asst. Professor & Project Guide
Jain Deemed-to-be University Jain Deemed-to-be University
Bangalore, Karnataka, India Bangalore, Karnataka, India

Abstract :- Cluster is a group of objects that are similar I. INTRODUCTION

amongst themselves but dissimilar to the objects in
other clusters. Identifying meaningful clusters and Cluster analysis is a study of algorithms and methods
thereby a structure in a large un-labelled dataset is an of classifying objects. Cluster analysis does not label or tag
important unsupervised data mining task. and assign an object into a pre-existent structure; instead,
Technological progress leads to enlarging volumes of the objective is to find a valid organization of the existing
data that require clustering. Clustering large datasets is data and thereby to identify a structure in the data. It is an
a challenging resource-intensive task and the key to important tool to explore pattern recognition and artificial
scalability and performance benefits is to use parallel or learning.
concurrent clustering algorithms. Data mining is the
process of discovering patterns in large data sets This project involves a complete study, evaluation and
involving methods at the intersection of machine investigation of machine learning algorithms and compare
learning, statistics, and database systems. Machine their results. In [4] several computational intelligence
learning techniques are widely used in the medical field techniques are studied for heart disease detection, and their
for the diagnosis of various diseases. A machine study is based in comparing six known classifiers with the
learning technique enables us to identify patterns in data provided by the University of Cleveland. In that
observed data, build a model according to the observed research the method with better precision levels was
pattern and predict things without pre-programmed support vector machines (89,49%) surpassing Decision
rules. There are two types of machine learning Trees, K-Nearest neighbors and Part. According to [6],
techniques namely supervised and unsupervised diagnosis of cardiac disease it is a relevant issue and many
machine learning techniques. To diagnose these diseases researchers have developed intelligent decision support
the data must be clustered before segmentation. systems to improve the capacity of medical staff. In that
In this project we analyzed Hard C Means (HCM), research, a novelty methodology is presented using SAS
Fuzzy C Means (FCM) clustering technique and Naive 9.1.3 software. The results had 89.01% of precision based
Bayes classification techniques to diagnose the CVD. on the heart disease dataset of the University of Cleveland,
The medical dataset is analyzed for all the three and also obtained 80.95% in sensitivity and 95.91% in
techniques and the results are evaluated based on specificity in the diagnosis of heart conditions. The title of
metrics like cluster quality, time consumed, true the project is “Modernistic approach to clustering
positive rate, false positive rate, precision, recall and algorithms”. For the comparison of the clustering
accuracy. The results indicated that FCM clustering algorithms, cardiovascular patient dataset is used.
technique provided a better accuracy of 91% and
precision and recall of 91% and 89% respectively in Cardiovascular diseases are the main cause of death
comparison with other two techniques in presence of around the world. Every year, more people die from these
noise. diseases than from any other cause. According to World
Health Organization data, in 2012 more than 17.5 million
Keywords:- Cluster; Data Mining; Machine Learning; people died from this cause, and that represents 31% of all
Supervised; Unsupervised; Hard C Means; Fuzzy C deaths registered worldwide. Data mining techniques are
Means; Naïve Bayes; Classification; Dataset; Cluster widely used for the analysis of diseases, including
Quality; Time Consumed; True Positive Rate; False cardiovascular conditions. It is necessary to use a clustering
Positive Rate; Precision; Recall; Accuracy method for data segmentation according to their diagnosis.

A. Objective
To study, evaluate and investigate the two
unsupervised machine learning technique and one
supervised machine learning technique with the common
dataset as the input. The results of the algorithms are
evaluated based on the attributes like cluster quality, time
consumed, true positive rate, false positive rate, precision,
recall and accuracy. Finally the results are compared and
the algorithm which gives better result for the dataset is
observed.

IJISRT19AP590 www.ijisrt.com 884

Volume 4, Issue 4, April – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
B. Scope  Thal: in case of normal value is 3, by default is 6, in
This project serves as a baseline to analyze and case of reversible defects is 7.
evaluate different machine learning algorithms for medical  Heart disease diagnosis (angiographic disease status): 0
dataset, consequently to determine the suitable algorithm to in case of less than 50%, 1 otherwise.
be chosen for a particular data input. This project helps in
diagnosing the cardiovascular disease for different machine IV. DESIGN
learning algorithms and evaluates the precision of the
diagnosis. The block diagram specifies the architecture, high
level design of a computer program. They aid the
C. Motivation programmer in dividing and conquering a large software
There are several different algorithms to analyze the problem by recursively breaking down the problem into
input dataset and produce results as clusters or classes. But parts that are small enough to be understood by a human
different algorithms produce variable results for the brain. The process is called a top-down design, or
different input dataset. Because our project revolves around functional decomposition.
diagnosing the cardiovascular disease whose results are
very crucial, the results are required to be precise. Hence Figure 1 shows the block diagram of the entire
we compare different types of machine learning algorithm system. It shows the decomposition of the entire task and
to learn which algorithm can improve the diagnosis flow from one subtask to another and the interaction
accuracy. between them. It has the following modules.

II. ASSUMPTIONS AND DEPENDENCIES

The various assumption and dependencies that affect

the project are as follows:

 The project depends on the input dataset and the

intended results.
 The results that are obtained from the HCM, FCM and
Naive Bayes technique are assumed to be true.
 The user of this project will be concerned to the medical
field who wished to know about the persons of being
affected by any heart disease symptoms.

III. DATASET ANALYSIS AND

PREPARATION

The dataset analyzed for this project is the “Heart

disease dataset” which is contained in the Machine
Learning Repository UCI [9]. There are 303 records in the
dataset and each record consists of 14 attributes. The
description of all the attributes of the dataset is given
below:

 Age: values of the age of a person in years.

 Sex: male takes the value of 1 and female is 0. Fig 1
 Chest Pain Type: In case of typical angina the value is1,
atypical angina is 2, other kind of pain is 3 and V. IMPLEMENTATION
asymptomatic is 4.
 Resting Blood pressure: Value calculated in Hg at the A. Unsupervised Machine Learning Technique
time of hospital admission.
 Cholesterol: mg/dl.  Hard C-Means (K-Means Clustering) Algorithm
 Blood sugar > 120 mg/dl: 1, in case is true, 0 otherwise. Let X = {x1,x2,x3,……..,xn} be the set of data points and
 Electrocardiogram Result: In case of normal value, is 0, V = {v1,v2,…….,vc} be the set of centers.
anomaly is 1 and ventricular hypertrophy is 2. Randomly select ‘c’ cluster centers.
 Maximum heart rate achieved
 Exercise induced angina: In case of negative is 0,  Calculate the distance between each data point and
otherwise is 1. cluster centers.
 Induced depression.  Assign the data point to the cluster center whose
 Slope peak exercise: distance from the cluster center is minimum of all the
 Number of major vessels (0 - 3) colored by fluoroscopy. cluster centers.

IJISRT19AP590 www.ijisrt.com 885

Volume 4, Issue 4, April – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Recalculate the new cluster center using: vi =
1 ci
( ) ∑j=1 xi where, ci represents the number of data
ci
points in the ith cluster.
 Recalculate the distance between each data point and
new obtained cluster centers.
 If no data point was reassigned then stop, otherwise
repeat from step 3).

 Fuzzy C-Mean Algorithm

Let X = {x1, x2, x3 ..., xn} be the set of data points and
V = {v1, v2, v3 ..., vc} be the set of centers.

 Randomly select ‘c’ cluster centers.

 Calculate the fuzzy membership ‘uij’ using: uij = Fig 2
dij 2
1/ ∑ck=1(d )m−1
ik B. Time Consumed for Clustering
 Compute fuzzy centers ‘vj’ using : vj = (∑ni=1(uij )m )/ The time consumed for clustering begins from the
(∑ni=1(uij )m ) for all j=1,2,3......c time the dataset input for the first iteration of the clustering.
 Repeat step 2) and 3) until minimum j value is achieved The time consumed for clustering should increase as and
or ||u(k+1)-u(k)||<𝛽 when the number of records in the dataset increases.
Where, ‘k’ is the iteration step
‘𝛽’ is the termination criterion between [0,1]
‘U=(uij )n∗c ’ is the fuzzy membership matrix
‘j’ is the objective function

B. Supervised Machine Learning Technique

 Naïve Bayes Classification

Bayes theorem provides a way of calculating posterior
probability P(c|x) from P(c), P(x) and P(x|c). Look at the
equation below:
P(x|c)P(c)
P(c|x) =
P(x)
P(c|X) = P(x1|c) ∗ P(x2|c) ∗ … … … ∗ P(xn |c)P(c) where,
Fig 3
 P(c|x) is the posterior probability of class(c, target)
given predictor (x, attributes). C. Performance
 P(c) is the prior probability of class. In this section the performance of all the three
 P(x|c) is the likelihood which is the probability machine learning techniques are compared based on some
of predictor given class. more metrics such as true positive rate, false positive rate,
P(x) is the prior probability of predictor. precision, recall and accuracy. The metrics such as cluster
quality and time consumed are very trivial and do not give
VI. RESULT accurate results for ambiguous dataset that is used.

A. Cluster Quality  True Positive Rate

The cluster qualities of the proposed techniques are The following is the formula to calculate the true
evaluated based on the distance between the clusters and positive rate of the data points.
the number of data points that were clustered. The cluster
quality should increase as the number of data points in the 𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒
cluster increases. 𝑇𝑝 𝑟𝑎𝑡𝑒 =
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 + 𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒

 False Positive Rate

The following is the formula to calculate the false
positive rate of the data points.

𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒
𝐹𝑝 𝑟𝑎𝑡𝑒 =
𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 + 𝑉𝑒𝑟𝑑𝑎𝑑𝑒𝑟𝑜𝑠 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒

IJISRT19AP590 www.ijisrt.com 886

Volume 4, Issue 4, April – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Precision
The following is the formula to calculate the precision
of the data points.

𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 + 𝐹𝑎𝑙𝑠𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒

 Recall
The following is the formula to calculate the recall of
the data points.

𝑉𝑒𝑟𝑑𝑎𝑑𝑒𝑟𝑜𝑠 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑜𝑠
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑉𝑒𝑟𝑑𝑎𝑑𝑒𝑟𝑜𝑠 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑜𝑠 + 𝑓𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑒𝑡𝑖𝑣𝑜𝑠

Fig 7

Figure above shows the comparison among the

different techniques based on the quality metrics that were
used. This figure clearly indicated that without aggregation,
if classification technique is compared against the soft
clustering technique then soft clustering technique yields
Fig 4
the better result.

VII. CONCLUSION

The different machine learning techniques that were

used for the implementation are HCM, FCM clustering and
the Naive Bayes classification. The clustering techniques
HCM and FCM were evaluated based on the cluster
quality. All the three techniques were compared for the
time consumed for clustering and classification. The time
consumption does not stand as a strong base for the
evaluation of the clusters. Cluster quality with different
metrics are calculated and it was observed that chevbychev
distance gives varied results for the same dataset and that
metric cannot be used for medical dataset.
Fig 5
We further compare the FCM clustering technique and
HCM clustering technique with classification technique
Naive Bayes. The techniques were compared based on the
true positive rate, false positive rate, precision, recall and
accuracy. The results obtained indicate that FCM clustering
technique provided a better accuracy of 91% and a
precision and recall of 91% and 89% respectively in
comparison with the other two techniques in presence of
noise. FCM provides better accuracy and precision than the
Naive Bayes technique when the data is not aggregated
before it is classified.

Fig 6

IJISRT19AP590 www.ijisrt.com 887

Volume 4, Issue 4, April – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
REFERENCES

[1]. Fabio Mendoza Palechor*, Alexis De la Hoz Manotas,

Paola Ariza Colpas , Jorge Sepulveda Ojeda, Roberto
Morales Ortega, Marlon Piñeres Melo Universidad de
la Costa, Barranquilla, Atlantico,
Colombia.(November 2, 2016). Cardiovascular
Disease Analysis Using Supervised and Unsupervised
Data Mining Techniques
[2]. Burke, A. P., Farb, A., Malcom, G. T., Liang, Y. H.,
Smialek, J., & Virmani, R. (1997). Coronary risk
factors and plaque morphology in men with coronary
disease who died suddenly. New England Journal of
Medicine.
[3]. El-Hanjouri, M., Alkhaldi, W., Hamdy, N., & Alim,
O. A. (2002). Heart diseases diagnosis using HMM.
Proceedings of the Electrotechnical Conference on
Mediterranean.
[4]. Nahar, J., Imam, T., Tickle, K. S., & Chen, Y. P. P.
(2013). Computational intelligence for heart disease
diagnosis: A medical knowledge driven approach.
Expert Systems with Applications.
[5]. Detrano, R., Steinbrunn, W., Pfisterer, M., Schmid, J.
J., Sandhu, S., Guppy, K. H., Lee, & Froelicher, V.
(1989). International application of a new probability
algorithm for the diagnosis of coronary artery disease.
American. Journal of Cardiology.
[6]. Das, R., Turkoglu, I., & Sengur, A. (2009). Effective
diagnosis of heart disease through neural networks
ensembles. Expert systems with applications.
[7]. Avci, E., & Turkoglu, I. (2009). An intelligent
diagnosis system based on principle component
analysis and ANFIS for the heart valve diseases.
Expert Systems with Applications.
[8]. Kurt, I., Ture, M., & Kurum, A. T. (2008). Comparing
performances of logistic regression, classification and
regression tree, and neural networks for predicting
coronary artery disease. Expert Systems with
Applications.
[9]. Janosi, A., Steinbrunn, W., Pfisterer, M., & Detrano,
R. Machine Learning Repository: Fertility Data Set.
Recupedao.

IJISRT19AP590 www.ijisrt.com 888