research-article

Geospatial Big Data Analytics Engine for Spark

Authors:

Shaohua Wang,

Yang Zhong,

Hao Lu,

Erqi Wang,

Weiying Yun, and

Wenwen CaiAuthors Info & Claims

BigSpatial '17: Proceedings of the 6th ACM SIGSPATIAL Workshop on Analytics for Big Geospatial Data

November 2017

Pages 42 - 45

https://doi.org/10.1145/3150919.3150923

Published: 07 November 2017 Publication History

Get Access

Abstract

With the rapid development of geospatial data acquisition and processing technology, the scale of spatial data is expanding. Mass production applications put forward higher requirements for the performance of geospatial data analysis. In this study, we developed a geospatial big data analytics engine based on SuperMap iObject for Java and Apache Spark. The geospatial big data analytics engine can increase the RDD representation ability of spatial data. The spatial indexing can make the spatial calculation on the nodes of the Spark cluster distributed and efficient. The experimental results show that compared with the traditional algorithm, the geospatial big data analytics engine for Spark has better execution efficiency.

References

[1]

Q. Li. and D. Li. Big data GIS. Geomatics and Information Science of Wuhan University, 39(6):641--644, 2014.

Google Scholar

[2]

M. R. Evans, D. Oliver, K. Yang, X. Zhou, and S. Shekhar, Enabling spatial big data via CyberGIS: Challenges and opportunities. CyberGIS: Fostering a New Wave of Geospatial Innovation and Discovery. Springer Book, 2013.

Google Scholar

[3]

E. Wang. and S. Wang. Technology Trends Prospects of the Future Development of GIS. Bulletin of Surveying and Mapping, 2015(S2):66--69, 2015.

Google Scholar

[4]

J. Yu, J. Wu and M. Sarwat. GeoSpark: A Cluster Computing Framework for Processing Large-Scale Spatial Data. In Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems ACM, 70, 2015.

Digital Library

Google Scholar

[5]

J. Zhang, S. You, and L. Gruenwald. Large-scale spatial data processing on GPUs and GPU-accelerated clusters. SIGSPATIAL Special, 6(3):27--34, 2015.

Digital Library

Google Scholar

[6]

S. You, J. Zhang, and L. Gruenwald. Spatial Join Query Processing in Cloud: Analyzing Design Choices and Performance Comparisons. In Parallel Processing Workshops (ICPPW), 44th International Conference on IEEE, 90--97.2015.

Digital Library

Google Scholar

[7]

M. Tang, Y. Yu, Q. M. Malluhi, M.O. uzzani, and W. G. Aref. Locationspark: a distributed in-memory data management system for big spatial data. Proceedings of the VLDB Endowment, 9(13):1565--1568, 2016.

Digital Library

Google Scholar

[8]

D. Xie, F. Li, B. Yao, G. Li, L. Zhou and M. Guo. Simba: Efficient In-Memory Spatial Analytics. In Proceedings of the 2016 International Conference on Management of Data ACM, 1071--1085.2016.

Digital Library

Google Scholar

[9]

S. Wang, E. Zhong, H. Lu, H. Guo, and L. Long. An effective algorithm for lines and polygons overlay analysis using uniform spatial grid indexing. In Spatial Data Mining and Geographical Knowledge Services (ICSDM), 2015 2nd IEEE International Conference on IEEE, 175--179, 2015.

Crossref

Google Scholar

[10]

S. Zhao and C. Zhou. Accelerating polygon overlay analysis by GPU. Progress in Geography, 32(1):114--120, 2013.

Google Scholar

[11]

R. K. Lenka, R. K. Barik, N. Gupta, S. M. Ali, A. Rath, and H. Dubey, Comparative analysis of SpatialHadoop and GeoSpark for geospatial big data analytics. In Contemporary Computing and Informatics (IC3I), 2016 2nd International Conference on IEEE, 484--488, 2016.

Google Scholar

[12]

W. Cai., S. Wang, E. Zhong, C. Hu, and X. Liu. Design and Implementation of a New Cross-platform Open Source GIS Desktop Software. Bulletin of Surveying and Mapping, 2017 (1):122--125, 2017.

Google Scholar

Cited By

View all

Wang WLi XWang TWang SWang RXu DZhou J(2024)Spatial-Temporal Evolution Characteristics Analysis of Color Steel Buildings in Lanzhou CityISPRS International Journal of Geo-Information10.3390/ijgi1306017913:6(179)Online publication date: 29-May-2024
https://doi.org/10.3390/ijgi13060179
Sun LJin B(2023)Improving NoSQL Spatial-Query Processing with Server-Side In-Memory R*-Tree Indexes for Spatial Vector DataSustainability10.3390/su1503244215:3(2442)Online publication date: 30-Jan-2023
https://doi.org/10.3390/su15032442
Fabris-Rotelli IStein A(2021)Use of fractals to measure anisotropy in point patterns extracted with the DPT of an imageSpatial Statistics10.1016/j.spasta.2020.10045242(100452)Online publication date: Apr-2021
https://doi.org/10.1016/j.spasta.2020.100452

Index Terms

Geospatial Big Data Analytics Engine for Spark
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Geographic visualization
2. Information systems
  1. Information systems applications
    1. Spatial-temporal systems

Recommendations

A CyberGIS-Jupyter Framework for Geospatial Analytics at Scale
PEARC '17: Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact

The interdisciplinary field of cyberGIS (geographic information science and systems (GIS) based on advanced cyberinfrastructure) has a major focus on data- and computation-intensive geospatial analytics. The rapidly growing needs across many application ...
Read More
A brief introduction to geospatial big data analytics with apache AsterixDB
SpatialAPI '21: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on APIs and Libraries for Geospatial Data Science

There is immense potential with spatial data, which is even more significant when combined with temporal or textual features, or both. However, it is expensive to store and analyze spatial data, and it is even more challenging with the combined features ...
Read More
Geospatial Big Data

Geospatial big data refers to spatial data sets exceeding capacity of current computing systems. A significant portion of big data is actually geospatial data, and the size of such data is growing rapidly at least by 20% every year. In this paper, we ...
Read More

Comments

Information & Contributors

Information

Published In

BigSpatial '17: Proceedings of the 6th ACM SIGSPATIAL Workshop on Analytics for Big Geospatial Data

November 2017

51 pages

ISBN:9781450354943

DOI:10.1145/3150919

Editors:
Varun Chandola
State University of New York at Buffalo, NY, USA
,
Ranga Raju Vatsavai
North Carolina State University, NC, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSPATIAL'17

Sponsor:

SIGSPATIAL

SIGSPATIAL'17: 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 7 - 10, 2017

CA, Redondo Beach, USA

Acceptance Rates

Overall Acceptance Rate 32 of 58 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
250
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang WLi XWang TWang SWang RXu DZhou J(2024)Spatial-Temporal Evolution Characteristics Analysis of Color Steel Buildings in Lanzhou CityISPRS International Journal of Geo-Information10.3390/ijgi1306017913:6(179)Online publication date: 29-May-2024
https://doi.org/10.3390/ijgi13060179
Sun LJin B(2023)Improving NoSQL Spatial-Query Processing with Server-Side In-Memory R*-Tree Indexes for Spatial Vector DataSustainability10.3390/su1503244215:3(2442)Online publication date: 30-Jan-2023
https://doi.org/10.3390/su15032442
Fabris-Rotelli IStein A(2021)Use of fractals to measure anisotropy in point patterns extracted with the DPT of an imageSpatial Statistics10.1016/j.spasta.2020.10045242(100452)Online publication date: Apr-2021
https://doi.org/10.1016/j.spasta.2020.100452

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A CyberGIS-Jupyter Framework for Geospatial Analytics at Scale

A brief introduction to geospatial big data analytics with apache AsterixDB

Geospatial Big Data