research-article

iSPEED: an Efficient In-Memory Based Spatial Query System for Large-Scale 3D Data with Complex Structures

Authors:

Fusheng WangAuthors Info & Claims

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Article No.: 17, Pages 1 - 10

https://doi.org/10.1145/3139958.3139961

Published: 07 November 2017 Publication History

Abstract

Recent advances in digital pathology make it possible to support 3D tissue-based investigation of human diseases at extremely high resolutions. Exploring spatial relationships and patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a critical role in studying human diseases. In this paper, we present our work on building an effective and scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. To achieve low latency, iSPEED stores data in memory with effective progressive compression for each 3D object with successive levels of detail. To minimize search space and computation cost, iSPEED pre-generates global spatial indexes in memory and employs on-demand indexing at run-time. In particular, iSPEED exploits structural indexing for complex structured objects in distance based queries. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. iSPEED builds in-memory indexes and decompresses data on-demand, which has minimal memory footprint. We evaluate iSPEED with two representative queries: 3D spatial joins and 3D spatial proximity estimation. Our experiments demonstrate that iSPEED significantly improves the performance over traditional non-memory based spatial query systems.

References

[1]

3D Modelling. https://en.wikipedia.org/wiki/3D_modeling.

[2]

The Computational Geometry Algorithms Library (CGAL). http://www.cgal.org/.

[3]

ESRI 3D GIS. http://www.esri.com/products/arcgis-capabilities/3d-gis.

[4]

FDAWSI. https://www.fda.gov/newsevents/newsroom/ucm552742.htm.

[5]

OFF Format. https://en.wikipedia.org/wiki/OFF_(file_format).

[6]

OpenTopography. http://www.opentopography.org/.

[7]

pbEncom. http://www.pitneybowes.com/pbencom/homepage.html.

[8]

Spatial Index Library. http://libspatialindex.github.com.

[9]

Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, and Joel Saltz. 2013. Hadoop GIS: a high performance spatial data warehousing system over mapreduce. Proceedings of the VLDB Endowment 6, 11 (2013), 1009--1020.

Digital Library

[10]

Lars Arge, Octavian Procopiuc, Sridhar Ramaswamy, Torsten Suel, Jan Vahrenhold, and Jeffrey Scott Vitter. 2000. A unified approach for indexed and non-indexed spatial joins. In International Conference on Extending Database Technology. Springer, 413--429.

Digital Library

[11]

Norbert Beckmann, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1990. The R*-tree: An Efficient and Robust Access Method for Points and Rectangles. In SIGMOD. 322--331.

Digital Library

[12]

J Bercken and Bernhard Seeger. 2001. An evaluation of generic bulk loading techniques. In Proc. of VLDB. 461--470.

Digital Library

[13]

Thomas Brinkhoff, Hans-Peter Kriegel, and Bernhard Seeger. 1996. Parallel processing of spatial joins using R-trees. In Data Engineering, 1996. IEEE, 258--265.

Digital Library

[14]

Nikki A. Charles, Eric C. Holland, Richard Gilbertson, Rainer Glass, and Helmut Kettenmann. 2012. The brain tumor microenvironment. Glia 60, 3 (2012), 502--514.

[15]

A. Davidson and A. Or. 2013. Optimizing Shuffle Performance in Spark. Technique Report. UC Berkeley.

[16]

Ahmed Eldawy and Mohamed F Mokbel. 2015. SpatialHadoop: A MapReduce framework for spatial data. In Data Engineering (ICDE) 2015. IEEE, 1352--1363.

[17]

Hugues Hoppe. 1996. Progressive meshes. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 99--108.

Digital Library

[18]

Yanhui Liang, Hoang Vo, Ablimit Aji, Jun Kong, and Fusheng Wang. 2016. Scalable 3D spatial queries for analytical pathology imaging with MapReduce. In The 24th ACM SIGSPATIAL GIS. ACM, 52.

Digital Library

[19]

Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. A 3D Primary Vessel Reconstruction Framework with Serial Microscopy Images. In MICCAI 2015. Springer, 251--259.

[20]

Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. Liver whole slide image analysis for 3D vessel reconstruction. In ISBI 2015. IEEE, 182--185.

[21]

Adrien Maglo, CléMent Courbet, Pierre Alliez, and Céline Hudelot. 2012. Progressive compression of manifold polygon meshes. Computers & Graphics 36, 5 (2012), 349--359.

Digital Library

[22]

Kenton McHenry and Peter Bajcsy. 2008. An Overview of 3D Data Content, File Formats and Viewers.

[23]

Jingliang Peng, Chang-Su Kim, and C-C Jay Kuo. 2005. Technologies for 3D mesh compression: A survey. Journal of Visual Communication and Image Representation 16, 6 (2005), 688--733.

Digital Library

[24]

Tohru Takahashi. 2011. Pathology of organ structure by analysis and interpretation of images (second edition ed.). SciPress, Tokyo.

[25]

Mingjie Tang, Yongyang Yu, Qutaibah M Malluhi, Mourad Ouzzani, and Walid G Aref. 2016. Locationspark: a distributed in-memory data management system for big spatial data. Proceedings of the VLDB Endowment 9, 13 (2016), 1565--1568.

Digital Library

[26]

Pierre Terdiman. OPCODE 3D collision detection library, 2005.

[27]

Hoang Vo, Ablimit Aji, and Fusheng Wang. 2014. SATO: a spatial data partitioning framework for scalable query processing. In The 22nd ACM SIGSPATIAL GIS. ACM, 545--548.

Digital Library

[28]

Fusheng Wang, Jun Kong, Jingjing Gao, Lee AD Cooper, Tahsin Kurc, Zhengwen Zhou, David Adler, Cristobal Vergara-Niedermayr, Bryan Katigbak, Daniel J Brat, et al. 2013. A high-performance spatial database based approach for pathology imaging algorithm evaluation. J. of pathology informatics 4, 1 (2013), 5.

[29]

Kaibo Wang, Yin Huai, Rubao Lee, Fusheng Wang, Xiaodong Zhang, and Joel H. Saltz. 2012. Accelerating Pathology Image Data Cross-comparison on CPU-GPU Hybrid Systems. Proc. VLDB Endow. 5, 11 (July 2012), 1543--1554.

Digital Library

[30]

Dong Xie, Feifei Li, Bin Yao, Gefei Li, Liang Zhou, and Minyi Guo. 2016. Simba: Efficient in-memory spatial analytics. In ICMD 2016. ACM, 1071--1085.

Digital Library

[31]

Simin You, Jianting Zhang, and Le Gruenwald. 2015. Large-scale spatial join query processing in cloud. In Data Engineering Workshops (ICDEW). IEEE, 34--41.

[32]

Jia Yu, Jinxuan Wu, and Mohamed Sarwat. 2015. Geospark: A cluster computing framework for processing large-scale spatial data. In The 23rd ACM SIGSPATIAL GIS. ACM, 70.

Digital Library

[33]

Xiaofang Zhou, David J Abel, and David Truffet. 1998. Data partitioning for parallel spatial join processing. Geoinformatica 2, 2 (1998), 175--204.

Digital Library

Cited By

Wang FLee RTeng DZhang XSaltz J(2024)High-Performance Spatial Data Analytics: Systematic R&D for Scale-Out and Scale-Up Solutions from the Past to NowProceedings of the VLDB Endowment10.14778/3685800.368591217:12(4507-4520)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685912
Dey ARajan SLambadaris I(2024)Detection of Abnormality in Deterministic Compressive Sensed Breast Thermograms Using Bilateral AsymmetryIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.348814473(1-13)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3488144
Teng DBaig FPeng ZKong JWang F(2023)Efficient spatial queries over complex polygons with hybrid representationsGeoInformatica10.1007/s10707-023-00508-228:3(459-497)Online publication date: 27-Dec-2023
https://doi.org/10.1007/s10707-023-00508-2
Show More Cited By

Recommendations

Efficient 3D Spatial Queries for Complex Objects
3D spatial data has been generated at an extreme scale from many emerging applications, such as high definition maps for autonomous driving and 3D Human BioMolecular Atlas. In particular, 3D digital pathology provides a revolutionary approach to map human ...
SparkGIS: Resource Aware Efficient In-Memory Spatial Query Processing
SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Much effort has been devoted to support high performance spatial queries on large volumes of spatial data in distributed spatial computing systems, especially in the MapReduce paradigm. Recent works have focused on extending spatial MapReduce frameworks ...
Hadoop GIS: a high performance spatial data warehousing system over mapreduce

Support of high performance queries on large volumes of spatial data becomes increasingly important in many application domains, including geospatial problems in numerous fields, location based services, and emerging scientific applications that are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2017

677 pages

ISBN:9781450354905

DOI:10.1145/3139958

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSPATIAL'17

Sponsor:

SIGSPATIAL

SIGSPATIAL'17: 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 7 - 10, 2017

CA, Redondo Beach, USA

Acceptance Rates

SIGSPATIAL '17 Paper Acceptance Rate 39 of 193 submissions, 20%;

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
266
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang FLee RTeng DZhang XSaltz J(2024)High-Performance Spatial Data Analytics: Systematic R&D for Scale-Out and Scale-Up Solutions from the Past to NowProceedings of the VLDB Endowment10.14778/3685800.368591217:12(4507-4520)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685912
Dey ARajan SLambadaris I(2024)Detection of Abnormality in Deterministic Compressive Sensed Breast Thermograms Using Bilateral AsymmetryIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.348814473(1-13)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3488144
Teng DBaig FPeng ZKong JWang F(2023)Efficient spatial queries over complex polygons with hybrid representationsGeoInformatica10.1007/s10707-023-00508-228:3(459-497)Online publication date: 27-Dec-2023
https://doi.org/10.1007/s10707-023-00508-2
Chen LTeng DZhu TKong JHerr BBueckle ABörner KWang F(2022)Real-time spatial registration for 3D human atlasProceedings of the 10th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data10.1145/3557917.3567618(27-35)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1145/3557917.3567618
Xiao MWang HGeng LLee RZhang X(2022)An RDMA-enabled In-memory Computing Platform for R-tree on ClustersACM Transactions on Spatial Algorithms and Systems10.1145/35035138:2(1-26)Online publication date: 12-Feb-2022
https://dl.acm.org/doi/10.1145/3503513
Teng DLiang YVo HKong JWang F(2022)Efficient 3D Spatial Queries for Complex ObjectsACM Transactions on Spatial Algorithms and Systems10.1145/35022218:2(1-26)Online publication date: 12-Feb-2022
https://dl.acm.org/doi/10.1145/3502221
Liu ZHua WLiu XLiang DZhao YShi M(2021)An Efficient Group-Based Replica Placement Policy for Large-Scale Geospatial 3D Raster Data on HadoopSensors10.3390/s2123813221:23(8132)Online publication date: 5-Dec-2021
https://doi.org/10.3390/s21238132
Kim HYoon HThakur NHwang GLee EKim CChong Y(2021)Deep learning-based histopathological segmentation for whole slide images of colorectal cancer in a compressed domainScientific Reports10.1038/s41598-021-01905-z11:1Online publication date: 18-Nov-2021
https://doi.org/10.1038/s41598-021-01905-z
Xiao MWang HGeng LLee RZhang X(2019)Catfish: Adaptive RDMA-enabled R-Tree for Low Latency and High Throughput2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS.2019.00025(164-175)Online publication date: Jul-2019
https://doi.org/10.1109/ICDCS.2019.00025
Vo HLiang YKong JWang F(2018)iSPEEDProceedings of the VLDB Endowment10.14778/3229863.323626411:12(2078-2081)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.14778/3229863.3236264

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten