research-article

Processing multi-way spatial joins on map-reduce

Authors:

Himanshu Gupta,

Bhupesh Chawda,

Tanveer A. Faruquie,

L. V. Subramaniam,

Mukesh MohaniaAuthors Info & Claims

EDBT '13: Proceedings of the 16th International Conference on Extending Database Technology

Pages 113 - 124

https://doi.org/10.1145/2452376.2452390

Published: 18 March 2013 Publication History

Abstract

In this paper we investigate the problem of processing multi-way spatial joins on map-reduce platform. We look at two common spatial predicates - overlap and range. We address these two classes of join queries, discuss the challenges and outline novel approaches for executing these queries on a map-reduce framework. We then discuss how we can process join queries involving both overlap and range predicates. Specifically we present a Controlled-Replicate framework using which we design the approaches presented in this paper. The Controlled-Replicate framework is carefully engineered to minimize the communication among cluster nodes. Through experimental evaluations we discuss the complexity of the problem under investigation, details of Controlled-Replicate framework and demonstrate that the proposed approaches comfortably outperform naive approaches.

References

[1]

Census 2000 Tiger/Line Data www.esri.com/data/download/census2000-tigerline.

[2]

OpenMap Library http://openmap.bbn.com/.

[3]

F. N. Afrati and J. D. Ullman. Optimizing joins in a map-reduce environment. In EDBT, 2010.

Digital Library

[4]

S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian. A comparison of join algorithms for log processing in mapreduce. In SIGMOD, 2010.

Digital Library

[5]

T. Brinkhoff, H. P. Kriegal, and B. Seeger. Efficient processing of spatial joins using R-trees. In SIGMOD, 1993.

Digital Library

[6]

T. Brinkhoff, H.-P. Kriegel, R. Schneider, and B. Seeger. Multi-step processing of spatial joins. In SIGMOD, 1994.

Digital Library

[7]

T. Brinkhoff, H.-P. Kriegel, and B. Seeger. Parallel processing of spatial joins using r-trees. In ICDE, 1996.

Digital Library

[8]

J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Commun. ACM, 51(1), 2008.

Digital Library

[9]

J.-P. Dittrich and B. Seeger. Data redundancy and duplicate detection in spatial join processing. In ICDE, 2000.

[10]

O. Günther. Efficient computation of spatial joins. In ICDE, 1993.

Digital Library

[11]

M.-L. Lo and C. V. Ravishankar. Spatial joins using seeded trees. In SIGMOD, 1994.

Digital Library

[12]

M.-L. Lo and C. V. Ravishankar. Spatial hash-joins. In SIGMOD, pages 247--258, 1996.

Digital Library

[13]

W. Lu, Y. Shen, S. Chen, and B. C. Ooi. Efficient processing of k nearest neighbor joins using mapreduce. PVLDB, 5(10), 2012.

Digital Library

[14]

N. Mamoulis and D. Papadias. Integration of spatial join algorithms for processing multiple inputs. In SIGMOD, 1999.

Digital Library

[15]

N. Mamoulis and D. Papadias. Multiway spatial joins. ACM Trans. Database Syst., 26(4), 2001.

Digital Library

[16]

A. Okcan and M. Riedewald. Processing theta-joins using mapreduce. In SIGMOD, 2011.

Digital Library

[17]

D. Papadias and D. Arkoumanis. Approximate processing of multiway spatial joins in very large databases. In EDBT, 2002.

Digital Library

[18]

J. M. Patel and D. J. DeWitt. Partition based spatial-merge join. In SIGMOD, 1996.

Digital Library

[19]

J. M. Patel and D. J. DeWitt. Clone join and shadow join: two parallel spatial join algorithms. In ACM-GIS, 2000.

Digital Library

[20]

R. Vernica, M. J. Carey, and C. Li. Efficient parallel set-similarity joins using mapreduce. In SIGMOD, 2010.

Digital Library

[21]

K. Wang, J. Han, B. Tu, J. Dai, W. Zhou, and X. Song. Accelerating spatial data processing with mapreduce. In ICPADS, 2010.

Digital Library

[22]

S. Zhang, J. Han, Z. Liu, K. Wang, and S. Feng. Spatial queries evaluation with mapreduce. In GCC, 2009.

Digital Library

[23]

S. Zhang, J. Han, Z. Liu, K. Wang, and Z. Xu. Sjmr: Parallelizing spatial join with mapreduce on clusters. In CLUSTER, 2009.

Cited By

de Oliveira TCosta FFoulds LLongo H(2023)Scheduling distributed multiway spatial join queries: optimization models and algorithmsInternational Journal of Geographical Information Science10.1080/13658816.2023.217038037:6(1388-1419)Online publication date: 6-Feb-2023
https://doi.org/10.1080/13658816.2023.2170380
Eken SSayar A(2021)A MapReduce-based distributed and scalable framework for stitching of satellite mosaic imagesArabian Journal of Geosciences10.1007/s12517-021-07500-w14:18Online publication date: 23-Aug-2021
https://doi.org/10.1007/s12517-021-07500-w
Ma YZhang RCui ZLin C(2020)Projection Based Large Scale High-Dimensional Data Similarity Join Using MapReduce FrameworkIEEE Access10.1109/ACCESS.2020.30070288(121665-121677)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3007028
Show More Cited By

Index Terms

Processing multi-way spatial joins on map-reduce
1. Information systems
  1. Data management systems
    1. Database management system engines
    2. Query languages
  2. Information systems applications
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query languages (principles)

Recommendations

Multi-way Spatial Joins Using R-Trees: Methodology and Performance Evaluation
SSD '99: Proceedings of the 6th International Symposium on Advances in Spatial Databases

We propose a new multi-way spatial join algorithm called M-way R-tree join which synchronously traverses M R-trees. The M-way R-tree join can be considered as a generalization of the 2-way R-tree join. Although a generalization of the 2-way R-tree join ...
On Multi-way Spatial Joins with Direction Predicates
SSTD '01: Proceedings of the 7th International Symposium on Advances in Spatial and Temporal Databases

Spatial joins are fundamental in spatial databases. Over the last decade, the primary focus of research has been on joins with the predicate "region intersection." In modern database applications involving geospatial data such as GIS, efficient ...
Multi-way spatial join selectivity for the ring join graph

Efficient spatial query processing is very important since the applications of the spatial DBMS (e.g. GIS, CAD/CAM, LBS) handle massive amount of data and consume much time. Many spatial queries contain the multi-way spatial join due to the fact that ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EDBT '13: Proceedings of the 16th International Conference on Extending Database Technology

March 2013

793 pages

ISBN:9781450315975

DOI:10.1145/2452376

General Chair:
Giovanna Guerrini
Università di Genova, Italy
,
Program Chair:
Norman W. Paton
University of Manchester, UK

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

EDBT/ICDT '13

EDBT/ICDT '13: Joint 2013 EDBT/ICDT Conferences

March 18 - 22, 2013

Genoa, Italy

Acceptance Rates

Overall Acceptance Rate 7 of 10 submissions, 70%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
416
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

de Oliveira TCosta FFoulds LLongo H(2023)Scheduling distributed multiway spatial join queries: optimization models and algorithmsInternational Journal of Geographical Information Science10.1080/13658816.2023.217038037:6(1388-1419)Online publication date: 6-Feb-2023
https://doi.org/10.1080/13658816.2023.2170380
Eken SSayar A(2021)A MapReduce-based distributed and scalable framework for stitching of satellite mosaic imagesArabian Journal of Geosciences10.1007/s12517-021-07500-w14:18Online publication date: 23-Aug-2021
https://doi.org/10.1007/s12517-021-07500-w
Ma YZhang RCui ZLin C(2020)Projection Based Large Scale High-Dimensional Data Similarity Join Using MapReduce FrameworkIEEE Access10.1109/ACCESS.2020.30070288(121665-121677)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3007028
Bhattu SPotluri AKadari PR. B. V. S(2020)Generalized communication cost efficient multi-way spatial join: revisiting the curse of the last reducerGeoInformatica10.1007/s10707-019-00387-624:3(557-589)Online publication date: 14-Jan-2020
https://doi.org/10.1007/s10707-019-00387-6
Kadari PPotluri ASristy NSubramanyam RKumar N(2020)Skew Aware Partitioning Techniques for Multi-way Spatial JoinMining Intelligence and Knowledge Exploration10.1007/978-3-030-66187-8_6(52-61)Online publication date: 20-Dec-2020
https://doi.org/10.1007/978-3-030-66187-8_6
Zou WJing WChen GLu YSong H(2019)A Survey of Big Data Analytics for Smart ForestryIEEE Access10.1109/ACCESS.2019.29079997(46621-46636)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2907999
Gupta HHans SAggarwal KMehta SChatterjee BJayachandran P(2018)Efficiently Processing Temporal Queries on Hyperledger Fabric2018 IEEE 34th International Conference on Data Engineering (ICDE)10.1109/ICDE.2018.00167(1489-1494)Online publication date: Apr-2018
https://doi.org/10.1109/ICDE.2018.00167
García-García FCorral AIribarne LVassilakopoulos MManolopoulos Y(2018)Efficient large-scale distance-based join queries in spatialhadoopGeoinformatica10.1007/s10707-017-0309-y22:2(171-209)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s10707-017-0309-y
Du ZZhao XYe XZhou JZhang FLiu R(2017)An Effective High-Performance Multiway Spatial Join Algorithm with SparkISPRS International Journal of Geo-Information10.3390/ijgi60400966:4(96)Online publication date: 26-Mar-2017
https://doi.org/10.3390/ijgi6040096
Eldawy AMokbel M(2017)The era of big spatial dataProceedings of the VLDB Endowment10.14778/3137765.313782810:12(1992-1995)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.14778/3137765.3137828
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents