poster

DISTIL: a distributed in-memory data processing system for location-based services

Authors:

Md Mahbub Alam,

Virendra C. Bhavsar,

Kenneth B. Kent,

Gerhard W. DueckAuthors Info & Claims

SIGSPATIAL '18: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Pages 496 - 499

https://doi.org/10.1145/3274895.3274961

Published: 06 November 2018 Publication History

Abstract

Location-based services (LBS) have become an ubiquitous technology and spatio-temporal data generated by LBS is characterized by high volume and velocity. In recent times several projects, such as GeoSpark, SpatialSpark and LocationSpark, have focused on developing spatial data systems that take advantage of the distributed in-memory data processing capability of Spark. However, most of these systems assume immutable spatial data, and they do not support high throughput location data updates that are common in LBS. On the other hand, a few HBase-based systems, such as MD-HBase, have been proposed that support data updates. However, these systems do not take advantage of any distributed in-memory query processing frameworks.

To address the challenges of high velocity location data, we propose DISTIL, a distributed in-memory spatio-temporal data processing system. Our system includes a distributed in-memory index and storage infrastructure that are built on a distributed in-memory programming paradigm called APGAS (Asynchronous Partitioned Global Address Space). In our system, the location records are distributed across a cluster of nodes, using the producer-consumer model. Our experimental evaluation demonstrates that DISTIL can support high throughput location updates, and low latency concurrent processing of spatio-temporal range queries.

References

[1]

Michael Anderson, Shaden Smith, Narayanan Sundaram, Mihai CapotĂ, Zheguang Zhao, Subramanya Dulloor, Nadathur Satish, and Theodore L. Willke. 2017. Bridging the Gap Between HPC and Big Data Frameworks. PVLDB (2017).

Digital Library

[2]

Philippe Charles, Christian Grothoff, Vijay Saraswat, Christopher Donawa, Allan Kielstra, Kemal Ebcioglu, Christoph von Praun, and Vivek Sarkar. 2005. X10: An Object-oriented Approach to Non-uniform Cluster Computing. In OOPSLA.

Digital Library

[3]

Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. of the ACM (2008).

Digital Library

[4]

Ahmed Eldawy and Mohamed F Mokbel. 2015. Spatialhadoop: A mapreduce framework for spatial data. In ICDE.

[5]

Chen Feng, Xi Yang, Fan Liang, Xian-He Sun, and Zhiwei Xu. 2015. LCIndex: A Local and Clustering Index on Distributed Ordered Tables for Flexible Multidimensional Range Queries. In ICPP. 719--728.

Digital Library

[6]

Anthony Fox, Chris Eichelberger, James Hughes, and Skylar Lyon. 2013. Spatio-temporal indexing in non-relational distributed databases. In IEEE Big Data.

[7]

Hadoop {n. d.}. Apache Hadoop. http://hadoop.apache.org/. ({n. d.}).

[8]

Hadoop-GIS {n. d.}. http://bmidb.cs.stonybrook.edu/hadoopgis/index. ({n. d.}).

[9]

Stefan Hagedorn, Philipp Götze, and Kai-Uwe Sattler. 2017. The STARK Framework for Spatio-Temporal Data Analytics on Spark. BTW 2017 (2017).

[10]

HBase {n. d.}. https://hbase.apache.org/. ({n. d.}).

[11]

MOTO {n. d.}. . http://moto.sourceforge.net/. ({n. d.}).

[12]

Shoji Nishimura, Sudipto Das, Divyakant Agrawal, and Amr El Abbadi. 2011. MD-HBase: A Scalable Multi-dimensional Data Infrastructure for Location Aware Services. In MDM.

Digital Library

[13]

O'Neil, Patrick and Cheng, Edward and Gawlick, Dieter and O'Neil, Elizabeth. 1996. The log-structured merge-tree (LSM-tree). Acta Informatica (1996).

Digital Library

[14]

Suprio Ray, Rolando Blanco, and Anil K. Goel. 2013. Enhanced Database Support for Location-based Services. In IWGS @ ACM AIGSPATIAL.

Digital Library

[15]

Suprio Ray, Rolando Blanco, and Anil K. Goel. 2014. Supporting Location-Based Services in a Main-Memory Database. In MDM.

Digital Library

[16]

Vijay Saraswat, George Almasi, Ganesh Bikshandi, Calin Cascaval, David Cunningham, David Grove, Sreedhar Kodali, Igor Peshansky, and Olivier Tardieu. 2010. The asynchronous partitioned global address space model. In AMP.

[17]

Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. 2010. The hadoop distributed file system. In IEEE MSST.

Digital Library

[18]

Stefan Hagedorn and Philipp Götze and Kai-Uwe Sattler. 2017. Big Spatial Data Processing Frameworks: Feature and Performance Evaluation. In EDBT.

[19]

Mingjie Tang, Yongyang Yu, Qutaibah M Malluhi, Mourad Ouzzani, and Walid G Aref. 2016. Locationspark: A distributed in-memory data management system for big spatial data. PVLDB (2016).

Digital Library

[20]

Tiger® {n. d.}. http://www.census.gov/geo/www/tiger. ({n. d.}).

[21]

Dong Xie, Feifei Li, Bin Yao, Gefei Li, Liang Zhou, and Minyi Guo. 2016. Simba: Efficient in-memory spatial analytics. In SIGMOD.

Digital Library

[22]

Simin You, Jianting Zhang, and Le Gruenwald. 2015. Large-scale spatial join query processing in cloud. In ICDEW.

[23]

Jia Yu, Jinxuan Wu, and Mohamed Sarwat. 2015. Geospark: A cluster computing framework for processing large-scale spatial data. In SIGSPATIAL.

Digital Library

[24]

Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster computing with working sets. HotCloud.

Digital Library

Cited By

Komati V(2023)Spatiotemporal Data EngineeringEmerging Trends, Techniques, and Applications in Geospatial Data Science10.4018/978-1-6684-7319-1.ch002(15-62)Online publication date: 7-Apr-2023
https://doi.org/10.4018/978-1-6684-7319-1.ch002
Guroob A(2023)EA2-IMDG: Efficient Approach of Using an In-Memory Data Grid to Improve the Performance of Replication and Scheduling in Grid Environment SystemsComputation10.3390/computation1103006511:3(65)Online publication date: 20-Mar-2023
https://doi.org/10.3390/computation11030065
Liu KTong PLi MWu YHuang J(2023)ST4ML: Machine Learning Oriented Spatio-Temporal Data Processing at ScaleProceedings of the ACM on Management of Data10.1145/35889411:1(1-28)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588941
Show More Cited By

Index Terms

DISTIL: a distributed in-memory data processing system for location-based services
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
  2. Information systems applications
    1. Spatial-temporal systems
      1. Data streaming
      2. Location based services

Recommendations

A Survey on Spatio-temporal Data Analytics Systems
Due to the surge of spatio-temporal data volume, the popularity of location-based services and applications, and the importance of extracted knowledge from spatio-temporal data to solve a wide range of real-world problems, a plethora of research and ...
Enhanced database support for location-based services
IWGS '13: Proceedings of the 4th ACM SIGSPATIAL International Workshop on GeoStreaming

The ubiquity of GPS-enabled mobile devices and sensors have led to the explosive growth of time-stamped location data. Consequently Location-Based Services (LBS) has become a popular technology impacting various aspects of our lives. LBS applications ...
Hilbert Index-based Outlier Detection Algorithm in Metric Space

Big data is profoundly changing the lifestyles of people around the world in an unprecedented way. Driven by the requirements of applications across many industries, research on big data has been growing. Methods to manage and analyze big data to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '18: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2018

655 pages

ISBN:9781450358897

DOI:10.1145/3274895

General Chairs:
Farnoush Banaei-Kashani
University of Colorado, Denver
,
Erik Hoel
Esri
,
Program Chairs:
Ralf Hartmut Güting
FernUniversität in Hagen, Germany
,
Roberto Tamassia
Brown University
,
Li Xiong
Emory University

Copyright © 2018 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2018

Check for updates

Author Tags

Qualifiers

Poster

Conference

SIGSPATIAL '18

Sponsor:

SIGSPATIAL

SIGSPATIAL '18: 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 6 - 9, 2018

Washington, Seattle

Acceptance Rates

SIGSPATIAL '18 Paper Acceptance Rate 30 of 150 submissions, 20%;

Overall Acceptance Rate 220 of 1,116 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
156
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Komati V(2023)Spatiotemporal Data EngineeringEmerging Trends, Techniques, and Applications in Geospatial Data Science10.4018/978-1-6684-7319-1.ch002(15-62)Online publication date: 7-Apr-2023
https://doi.org/10.4018/978-1-6684-7319-1.ch002
Guroob A(2023)EA2-IMDG: Efficient Approach of Using an In-Memory Data Grid to Improve the Performance of Replication and Scheduling in Grid Environment SystemsComputation10.3390/computation1103006511:3(65)Online publication date: 20-Mar-2023
https://doi.org/10.3390/computation11030065
Liu KTong PLi MWu YHuang J(2023)ST4ML: Machine Learning Oriented Spatio-Temporal Data Processing at ScaleProceedings of the ACM on Management of Data10.1145/35889411:1(1-28)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588941
Ren YHuang DWang WYu X(2023)BSMDFuture Generation Computer Systems10.1016/j.future.2022.09.008138:C(328-338)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.future.2022.09.008
Richly KSchlosser RBoissier M(2022)Budget-Conscious Fine-Grained Configuration Optimization for Spatio-Temporal ApplicationsProceedings of the VLDB Endowment10.14778/3565838.356585815:13(4079-4092)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.14778/3565838.3565858
Tian RZhai HZhang WWang FGuan Y(2022)A Survey of Spatio-Temporal Big Data Indexing Methods in Distributed EnvironmentIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2022.317565715(4132-4155)Online publication date: 2022
https://doi.org/10.1109/JSTARS.2022.3175657
Dai HWang YXu C(2022)Osprey: a heterogeneous search framework for spatial-temporal similarityComputing10.1007/s00607-022-01075-4104:9(1949-1975)Online publication date: 4-Apr-2022
https://doi.org/10.1007/s00607-022-01075-4
Dritsas EKanavos ATrigka MVonitsanos GSioutas STsakalidis A(2020)Trajectory Clustering and k-NN for Robust Privacy Preserving k-NN Query Processing in GeoSparkAlgorithms10.3390/a1308018213:8(182)Online publication date: 28-Jul-2020
https://doi.org/10.3390/a13080182
Badarneh HRavana SMansoor A(2020)A survey on indexing techniques for mobility in Internet of Things'International Journal of Network Management10.1002/nem.209730:4Online publication date: 7-Jul-2020
https://dl.acm.org/doi/10.1002/nem.2097
Memarzia PPatrou MAlam MRay SBhavsar VKent K(2019)Toward Efficient Processing of Spatio-Temporal Workloads in a Distributed In-Memory System2019 20th IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM.2019.00-66(118-127)Online publication date: Jul-2019
https://doi.org/10.1109/MDM.2019.00-66
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents