Abstract
One of the most important aspects to consider when computing large data sets is to distribute and parallelize the analysis algorithms. A distributed system presents a good performance if the workload is properly balanced. It is expected that the computing time is directly related to the processing time on the node where the processing takes longer. This paper aims at proposing a data partitioning strategy that takes into account partition balance and that is generic for spatial data. Our proposed solution is based on a grid model data structure that is further transformed into a graph partitioning problem, where we finally compute the partitions. Our proposed approach is used on the distributed DBSCAN algorithm and it is focused on finding density areas in a large data set using MapReduce. We call our approach G2P (Grid and Graph Partitioning) and we show via massive experiments that G2P presents great quality data partitioning for the distributed DBSCAN algorithm compared to the competitors. We believe that G2P is not only suitable for DBSCAN algorithm, but also to execute spatial join operations and distance based range queries to name to a few.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Schadt, E.E., Linderman, M.D., Sorenson, J., Lee, L., Nolan, G.P.: Computational solutions to large-scale data management and analysis. In: Nature Reviews Genetics, pp. 647–657. Nature Publishing Group (2010)
Ankerst, M., Breunig, M.M., Kriegel, H.-P., Sander, J.: Optics: ordering points to identify the clustering structure. ACM SIGMOD Record 28(2), 49–60 (1999)
Bentley, J.L.: Multidimensional binary search trees used for associative searching. In: Communications of the ACM, vol. 18, pp. 509–517. ACM (1975)
Coelho da Silva, T.L., Araujo, A.C.N., Magalhaes, R.P., Farias, V.A.E., de Macedo, J.A., Machado, J.C.: Efficient and distributed dbscan algorithm using mapreduce to detect density areas on traffic data. In: ICEIS (2014)
Dai, B.-R., Lin, I.-C.: Efficient map/reduce-based dbscan algorithm with optimized data partition. In: 2012 IEEE 5th International Conference on Cloud Computing (CLOUD), pp. 59–66. IEEE (2012)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Communications of the ACM 51(1), 107–113 (2008)
Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. KDD 96, 226–231 (1996)
Giannotti, F., Nanni, M., Pedreschi, D., Pinelli, F., Renso, C., Rinzivillo, S., Trasarti, R.: Unveiling the complexity of human mobility by querying and mining massive trajectory data. VLDB J. 20(5), 695–719 (2011)
He, Y., Tan, H., Luo, W., Mao, H., Ma, D., Feng, S., Fan, J.: Mr-dbscan: an efficient parallel density-based clustering algorithm using mapreduce. In: 2011 IEEE 17th International Conference on Parallel and Distributed Systems (ICPADS), pp. 473–480. IEEE (2011)
Jensen, C.S., Lin, D., Ooi, B.-C.: Continuous clustering of moving objects. IEEE Transactions on Knowledge and Data Engineering 19(9), 1161–1174 (2007)
Jeung, H., Yiu, M.L., Zhou, X., Jensen, C.S., Shen, H.T.: Discovery of convoys in trajectory databases. Proceedings of the VLDB Endowment 1(1), 1068–1080 (2008)
Kisilevich, S., Mansmann, F., Keim, D.: P-dbscan: a density based clustering algorithm for exploration and analysis of attractive areas using collections of geo-tagged photos. In: Proceedings of the 1st International Conference and Exhibition on Computing for Geospatial Research & Application, p. 38. ACM (2010)
Li, X., Ceikute, V., Jensen, C.S., Tan, K.-L.: Effective online group discovery in trajectory databases. IEEE Transactions on Knowledge and Data Engineering 25(12), 2752–2766 (2013)
Li, Y., Han, J., Yang, J.: Clustering moving objects. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 617–622 (2004)
Lin, J., Dyer, C.: Data-intensive text processing with mapreduce. Synthesis Lectures on Human Language Technologies 3(1), 1–177 (2010)
Uncu, O., Gruver, W.A., Kotak, D.B., Sabaz, D., Alibhai, Z., Ng, C.: Gridbscan: grid density-based spatial clustering of applications with noise. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2006, vol. 4, pp. 2976–2981. IEEE (2006)
Vieira, M.R., Bakalov, P., Tsotras, V.J.: On-line discovery of flock patterns in spatio-temporal data. In: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 286–295. ACM (2009)
Pavlo, A., Paulson, E., Rasin, A., Abadi, D.J., DeWitt, D.J., Madden, S., Stonebraker, M.: A comparison of approaches to large-scale data analysis. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pp. 165–178. ACM (2009)
Welton, B., Samanas, E., Miller, B.P.: Mr. scan: extreme scale density-based clustering using a tree-based network of gpgpu nodes. In: Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis, p. 84. ACM (2013)
Kim, Y., Shim, K., Kim, M.-S., Lee, J.S.: DBCURE-MR: an efficient density-based clustering algorithm for large data using MapReduce. In: Information Systems, pp. 15–35. Elsevier (2014)
Karypis, G., Kumar, V.,: Metis-unstructured graph partitioning and sparse matrix ordering system, version 2.0. Citeseer (1995)
Andreev, K., Racke, H.: Balanced graph partitioning. In: Theory of Computing Systems, pp. 15–35. Springer (2006)
Yahoo! Webscope. Yahoo! Webscope dataset YFCC-100M (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Araujo Neto, A.C., Coelho da Silva, T.L., de Farias, V.A.E., Macêdo, J.A.F., de Castro Machado, J. (2015). G2P: A Partitioning Approach for Processing DBSCAN with MapReduce. In: Gensel, J., Tomko, M. (eds) Web and Wireless Geographical Information Systems. W2GIS 2015. Lecture Notes in Computer Science(), vol 9080. Springer, Cham. https://doi.org/10.1007/978-3-319-18251-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-18251-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18250-6
Online ISBN: 978-3-319-18251-3
eBook Packages: Computer ScienceComputer Science (R0)