International Journal of Science and Research (IJSR)

ISSN (Online): 2319-7064

Index Copernicus Value (2013): 6.14 | Impact Factor (2015): 6.391

Implementing K-Means Clustering Algorithm Using

MapReduce Paradigm
Botcha Chandrasekhara Rao1, Medara Rambabu2
1
M.Tech Scholar, Department of Computer Science & Engineering, Pydah Kaushik College of Engineering, Gambheeram Village,
Anandapuram, Mandal-531163.Visakhapatnam, AP, India
2
HOD &Associate Professor, Department of Computer Science & Engineering, Pydah Kaushik College of Engineering, Gambheeram Village,
Anandapuram, Mandal-531163.Visakhapatnam, AP, India

Abstract: Clustering is a useful data mining technique which groups data points such that the points within a single group have similar
characteristics, while the points in different groups are dissimilar. Partitioning algorithm methods such as k-means algorithm is one kind
of widely used clustering algorithms. As there is an increasing trend of applications to deal with vast amounts of data, clustering such big
data is a challenging problem. Recently, partitioning clustering algorithms on a large cluster of commodity machines using the
MapReduce framework have received a lot of attention. Traditional way of clustering text documents is Vector space model, in which tf-idf
is used for k-means algorithm with supportive similarity measure. This project exhibits an approach to cluster text documents in which
results obtained by executing map reduce k-means algorithm on single node cluster show that the performance of the algorithm increases
as the text corpus increases.

Keywords: Vector space model, map reduce, text, clustering, map reduce k-means, Hadoop

1. Introduction importance in the whole corpus rather cannot compare with

other documents. Term frequency -Inverse document
Big Data[1] is the term applied to data sets whose size is frequency (TF-IDF) overcomes the limitations of term
beyond the ability of thecommonly used software tools to frequency and calculated term weighting to find how
capture, manage, and process within a tolerableelapsed time. important a word is to a document in a collection or corpus.
Big data can be analysed with the software tools commonly
used as part of advanced analytics disciplines such The weight vector for document d isVd=[w1,d, w2,d, w3,d,..,
as predictiveanalytics, datamining, textanalytics and statistica wN,d]T, where
l analysis. The semi-structured and unstructured data may not
fit well in traditional data warehouses based on relational
databases.Hadoop is a platform that provides both distributed tfi,d is the term frequency of term t in document d (a local
storage and computational capabilities. It is an open source parameter)
software project that enables the distributed processing of
large data sets across clusters of commodity servers. It is For any two documents dj and dk, their similarity is
designed to scale up from a single server to thousands of
machines, with a very high degree of fault tolerance. Hadoop
is a distributed master-slave architecture that consists of
Hadoop distributed file system (HDFS) for storage and Map-
Reduce for computational capabilities.
where wi is the weight of the term (Here tf-idf value)
n is the number of dimensions of document vector
Mathematical models like Boolean model, probabilistic
model, vector space model are proposed to use in information
B. K-Means Clustering
retrieval systems. Vector space model (VSM) [2] is most
K-Means is one of the simplest unsupervised learning
popular and widely used model.
algorithms that solve the well known clustering problem.
The procedure follows a simple and easy way to classify a
2. Literature Survey given data set through a certain number of clusters (assume k
clusters).The number of clusters should match the data. An
A. Vector Space Model incorrect choice of the number of clusters will invalidate the
In VSM [3], A term document matrix t X d is created, where whole process. An empirical way to find the best number of
t is the terms(words) of the documents and d represents clusters is to try K-means clustering with different number of
documents and find the frequency of the terms in each and clusters and measure the resulting sum of squares.
every document..This term frequency cannot able to find its

Volume 5 Issue 7, July 2016

www.ijsr.net
Licensed Under Creative Commons Attribution CC BY
Paper ID: 14071601 1240
 International Journal of Science and Research (IJSR)
ISSN (Online): 2319-7064
Index Copernicus Value (2013): 6.14 | Impact Factor (2015): 6.391
The basic K-means Algorithm is as follows: one value. The returns of all calls are collected as the desired
Step 1: Select K points as initial centroids result list.
Step2: repeat
Step 3: Form K clusters by assigning each point to its Thus the MapReduce framework transforms a list of (key,
Closest centroid value) pairs into a list of values.
Step 4: Recompute the centroid of each cluster
Step 5: Until centroids do not change.

The key step of Basic K-means algorithm is selection of

proper initial centroids. Initial clusters are formed by random
initialization of centroids.

C. Hadoop
Hadoop is a platform that provides both distributed storage
and computational capabilities. It is an open source software
project that enables the distributed processing of large data
sets across clusters of commodity servers. It is designed to
scale up from a single server to thousands of machines, with
a very high degree of fault tolerance. Hadoop[7] is a Figure 1: MapReduce Tasks
distributed master-slave architecture that consists of Hadoop
distributed file system (HDFS) for storage and Map-Reduce 3. Mapreduce K-Means Clustering
for computational capabilities. Hadoop can handle all types
of data from disparate systems: structured, unstructured, log The mapreduce k-means clustering approach for processing
files, pictures, audio files, communications records, email big text corpus [4] can be done by the following steps:
just about anything you can think of, regardless of its native 1) Give the Sequence file from directory of text documents
format. Even when different types of data have been stored in as input.
unrelated systems, you can dump it all into your Hadoop 2) Tokenize and generate TF-IDF vector for each document
cluster with no prior need for a schema. from sequence file.
3) Apply Map-Reduce K-Means algorithm to form k clusters.
Hadoop Distributed File System (HDFS)[5] is a file system
that spans all the nodes in a Hadoop cluster for data storage. A. Sequence File from Directory of Text Documents
The HDFS splits large data files into chunks that are Map reduce programming is coined to process huge data sets
managed by different nodes in the cluster. Each chunk is in parallel and distributed environment. Suppose, we select
replicated across several nodes to address single node outage the input data from a document set, where the text files in the
or fencing scenarios. directory are small in size. Since HDFS and Mapreduce are
optimized for large files, convert the small text files into
D. MapReduce Programming larger file i.e., SequenceFile format. SequenceFile is a
MapReduce[6] runs as a series of jobs, with each job hadoop class, which allows us to write document data in
essentially a separate Java application that goes out into the terms of binary <key, value> pairs, where key is a Text with
data and starts pulling out information as needed. Based on unique document id and value is Text content within the
the MapReduce design, records are processed in isolation via document in UTF-8 format. SequenceFile packs the small
tasks called Mappers. The output from the Mapper tasks is files and process whole file as a record. Since the
further processed by a second set of tasks, the Reducers, SequenceFile is in binary format, we could not able to read
where the results from the different Mapper tasks are merged the content directly but faster for read /write operations.
together. Using MapReduce instead of a query gives data
seekers a lot of power and flexibility, but also adds a lot of B. Creating TF-IDF Vectors
complexity. The Map and Reduce functions The sequence file from the previous step is fed as Input to
of MapReduce are both defined with respect to data create vectors. The TF-IDF vectors are calculated in
structured in (key, value) pairs. Map takes one pair of data Mapreduce by the following steps:
with a type in one data domain, and returns a list of pairs in a
different domain: Step 1: Tokenization: The input fed to map function is in
Map(k1,v1) list(k2,v2) format of <key, value> pairs, where key is the document
The Map function is applied in parallel to every pair in the name and value as document content. The outcome of reduce
input dataset. This produces a list of pairs for each call. After function is also <key, value> pair where key is document
that, the MapReduce framework collects all pairs with the name and value are tokens (words) present in that document.
same key from all lists and groups them together, creating Ex: Key: /acq1.txt: Value: [macandrews, forbes, holdings,
one group for each key. bids, revlon, mcandrews, forbes, holdings, inc,said, offer,
The Reduce function is then applied in parallel to each group, dlrs, per, share, all, revlon, group]
which in turn produces a collection of values in the same
domain: Step 2: Dictionary file: This step assign unique number to
Reduce(k2, list (v2)) list(v3) each token in all documents. The input format for the map
Each Reduce call typically produces either one value v3 or an function is <document name, wordlist> and the output of
empty return, though one call is allowed to return more than reduce function is <Word, uniqueid>.
Volume 5 Issue 7, July 2016
www.ijsr.net
Licensed Under Creative Commons Attribution CC BY
Paper ID: 14071601 1241
 International Journal of Science and Research (IJSR)
ISSN (Online): 2319-7064
Index Copernicus Value (2013): 6.14 | Impact Factor (2015): 6.391
Ex: Key: accounts: Value: 152.
Step 5 : Calculate tf-idf value : The output of step 4 is taken
Step 3: Frequency count: The number of times the word as input to map function of this step and calculates the weight
appears globally in all documents is calculated in this step. vector as tf X tf-idf value of each term ti in each document dj
The input to this map function is <docname, words>and as specified in equation(1). The output format of the result is
output format is <word id, 1>.The value of output of the map <docname, {t1: tf-idf1, t2:tf-idf2 ...ti:tf-idfi}>.
function is accumulated and find the sum in reduce function. For example: The output format of thisstep is as follows:
The output format of reduce function is <word id, count>. Key: acq1.txt: Value: {3258:0.12728, 3257:0.12728,
Ex: Key: 50: Value: 2 462:0.08060 ...}

Step 4 : Calculate term frequency : The map reduce function C. MapReduce K-Means Algorithm
in this step takes input as <docname, wordlist > and counts The implementation of map reduce k-means accepts two
the number of times each word or term ti occurs in that input files. One input file contains the documents with each
document dj. The outcome of this step is in the format of term and its tf-idf values, and the second is k initial centroids
<docname, {ti: count}>. file. The set of k initial centroids are selected randomly. In
every iteration, the map reduce framework splits the input
Ex: In the below example, acq1.txt is document name and the data into M splits and then processed in parallel as shown in
values are in the format of list of (wordid:count) Figure 2.
Key:/acq1.txt: Value: {3258:1.0, 3257:1.0, 157:2.0 ...}

Figure 2: MapReduce Framework for K-Means Algorithm

The map function read the document in the format of bestCentroid c

<docname, {t1 :tf-idf1, t2:tf-idf2, ...ti:tf-idfi}> along with end if
randomly selected set of k initial centroids. The map function end for
determines the document set which are closer to the centroids emit (bestCentroid, xi)
by calculating cosine similarity measure and emits records i+=1
containing all documents data with kcentroids in the format end for
as <k-center, {docname, {t1 :tfidf1, t2:tf-idf2, ...ti: tf-idfi}>. return Outputlist
The reducer function receives the output of map function k-
centroids along with closest documents bound to it. and Algorithm for Reducer
calculates new k-centroid. The mapper and reducer algorithm
of k-means is explained as below: Input: (Key, Value), where key = bestCentroid and Value
Algorithm for Mapper =Objects assigned to the lpgr'; 1\] x centroid by the mapper

Input: A set of objects X = {x1, x2 xn}, A Set ofinitial Output: (Key, Value), where key = oldCentroid and value =
Centroids C = {c1, c2, ,ck} newBestCentroid which is the new centroid value calculated
Output: An output list which contains pairs of (Ci, xj)where for that bestCentroid
1 i n and 1 j k
Procedure
Procedure
M1{x1, x2 xm} Outputlistoutputlist from mappers
current_centroidsC {}
Distance (p, q) =di=1(pi qi)2(where pi (or qi)is the newCentroidList null
coordinate of p (or q) in dimension i) for all outputlist do
for all xi M1 such that 1im do centroid .key
bestCentroidnull object .value
minDist [centroid] object
for all c current_centroids do end for
dist distance (xi, c) for all centroid do
if (bestCentroid = null || dist<minDist) newCentroid, sumofObjects,
then sumofObjects null
minDistdist for all object [centroid] do

Volume 5 Issue 7, July 2016

www.ijsr.net
Licensed Under Creative Commons Attribution CC BY
Paper ID: 14071601 1242
 International Journal of Science and Research (IJSR)
ISSN (Online): 2319-7064
Index Copernicus Value (2013): 6.14 | Impact Factor (2015): 6.391
sumofObjects += object tested in single node cluster with 20_newsgroups database
numofObjects += 1 and the performance of the map reduce algorithm increases
end for efficiently by increasing the number of input files and
newCentroid (sumofObjects + increasing number of nodes in the cluster. The runtime
numofObjects) execution in terms of seconds can be given as the CPU time
emit (centroid, newCentroid) spent for each iteration.
end for
end E. Output

The outcome of the k-means map reduce algorithm is the

cluster points along with bounded documents as <key, value>
pairs, where key is the cluster id and value contains in the
form of vector: weight. The weight indicates the probability
of vector be a point in that cluster. For Example: Key: 92:
Value: 1.0: [32:0.127,79:0.114, 97:0.114, 157:0.148 ...].

The final output of the program will be the cluster name, file
name: number of text documents that belong to that cluster.

4. Experimental Results
This section presents the results obtained by executing map
reduce k-means clustering algorithm on cluster of machines. Time taken for iteration 0
The experimentation with 20_newsgroups dataset is
explained in detailed in below sections.

A. Environment setup
The experimentation is conducted in single node cluster. The
single node is configured with I7processor, 4 GB memory,
64GB hard disk along with JDK 1.7.0 and hadoop 2.4.1
version. The Operating system used is Ubuntu 14.04LTS.

B. Dataset description
The 20_Newsgroups data set is a collection of approximately
20,000 newsgroup documents, partitioned (nearly) evenly
across 20 different newsgroups.It was originally collected by
Time taken for iteration 1
Ken Lang, probably for his Newsweeder: Learning to filter
netnews paper, though he does not explicitly mention this
collection. The 20 newsgroups collection has become a
popular data set for experiments in text applications of
machine learning techniques, such as text classification and
text clustering. The data is organized into 20 different
newsgroups, each corresponding to a different topic. Some of
the newsgroups are very closely related to each other
(e.g. comp.sys.ibm.pc.hardware /
comp.sys.mac.hardware), while others are highly unrelated
(e.g. misc.forsale / soc.religion.christian).

C. Data pre-processing
The data available here are in .tar.gz bundles. You will
need tar and unzip to open them. Each subdirectory in the Time taken for iteration 2
bundle represents a newsgroup; each file in a subdirectory is
the text of some newsgroup document that was posted to that
newsgroup. Create a sequence file from 20_newsgroups text
documents. Sequence file is passed as input to find tf-idf
value of each term in every text document. The tf-idf file is
fed as input to map reduce k-means algorithm for form k
number of clusters.

D. Results
In a single node cluster, the map reduce k-means algorithm is
executed with 20_newsgroups dataset with selected number
of text files of 20,000 from 20 different topics. The algorithm
Volume 5 Issue 7, July 2016
www.ijsr.net
Licensed Under Creative Commons Attribution CC BY
Paper ID: 14071601 1243
 International Journal of Science and Research (IJSR)
ISSN (Online): 2319-7064
Index Copernicus Value (2013): 6.14 | Impact Factor (2015): 6.391
[2] G. Salton, A. Wong, C. S. Yang.A vector space model for
automatic indexing. Communications of the ACM,
version.18 n.11, pages.613-620, Nov. 1975.
[3] GeorgeTsatsaronis and Vicky Panagiotopoulou. A
generalized vector space model for text retrieval based on
semantic relatedness. Proceedings of the EACL 2009
student research workshop, pages 70-78, April 2009.
[4] J Dittrich, JA Quian-Ruiz. Efficient big data
processingin Hadoop MapReduce. Proceedings of the
VLDB Endowment, 2012 - dl.acm.org, Volume 5 Issue
12, August 2012 ,Volume 5 Issue 12, August 2012.
[5] Sanjay, G., G. Howard, and L. Shun-Tak, The Google file
system, in Proceedings of the nineteenth ACM
symposium on Operating systems principles. ACM:
Bolton Lan,ding, NY, USA 2003,
[6] Jeffrey, D. and G. Sanjay,. MapReduce: simplified
dataprocessing on large clusters. Commun. ACM, 51(1):
pages 107-113 2008.
[7] Apache Lucene Hadoop[EB/OL].
http://hadoop.apache.org/.

Author Profile
Botcha Chandrasekhara raois currently pursuing his
2 years M.Tech in CSE at Pydah Kaushik College of
Engineering, Gambheeram Village, Anandapuram
Mandalam, Visakhapatnam. His area of interest
includes Cloud Computing.
Clusters
Medara Rambabu is currently working as HOD &
5. Conclusions and Future Work Associate Professor in Department of Computer
Science & Engineering at Pydah Kaushik College of
Engineering, Gambheeram Village, Anandapuram
Information retrieval techniques are widely popular in most Mandalam, Visakhapatnam. He completed his M.Tech from
of the search engines to efficiently organize and retrieve JNTUK in 2010, and he completed his B.Tech from AU, in 2004.He
information systems. Most of the data in internet is in the has more than 6 years of teaching experience in various engineering
format of unstructured and semi structured. Currently colleges. His research interest includes Computer Organization,
clustering techniques are used to organize and group the Data Communication Systems, Computer Networks and DM & DW.
similar data objects to retrieve search results faster.
Traditional way of clustering text documents is Vector space
model, in which tf-idf is used for k-means algorithm with
supportive similarity measure. As the data is enormously
increasing day by day, elastic resources are required to store
and compute. Hadoop framework supports to store and
compute big data in parallel and distributed platform with the
help of HDFS and Map reduce.

This paper exhibits an approach to cluster text documents

using vector space model. The results obtained by executing
map reduce k-means algorithm on single node cluster shows
that the performance of the algorithm increases as the text
corpus increases. In the current approach of map reduce k-
means algorithm, the initial centroids are selected randomly.
In future, Canopy clustering algorithm can be used as initial
step to get initial centroids and then fed this as input to map
reduce k-means algorithm. In future, we can try to cluster
text corpus by map reduced text clustering based on frequent
itemsets.

References
[1] Hadoop The definitive guide.

Volume 5 Issue 7, July 2016

www.ijsr.net
Licensed Under Creative Commons Attribution CC BY
Paper ID: 14071601 1244