research-article

iSPEED: an Efficient In-Memory Based Spatial Query System for Large-Scale 3D Data with Complex Structures

Authors:

Fusheng WangAuthors Info & Claims

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

Article No.: 17, Pages 1 - 10

https://doi.org/10.1145/3139958.3139961

Published: 07 November 2017 Publication History

Abstract

Recent advances in digital pathology make it possible to support 3D tissue-based investigation of human diseases at extremely high resolutions. Exploring spatial relationships and patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a critical role in studying human diseases. In this paper, we present our work on building an effective and scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. To achieve low latency, iSPEED stores data in memory with effective progressive compression for each 3D object with successive levels of detail. To minimize search space and computation cost, iSPEED pre-generates global spatial indexes in memory and employs on-demand indexing at run-time. In particular, iSPEED exploits structural indexing for complex structured objects in distance based queries. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. iSPEED builds in-memory indexes and decompresses data on-demand, which has minimal memory footprint. We evaluate iSPEED with two representative queries: 3D spatial joins and 3D spatial proximity estimation. Our experiments demonstrate that iSPEED significantly improves the performance over traditional non-memory based spatial query systems.

References

[1]

3D Modelling. https://en.wikipedia.org/wiki/3D_modeling.

[2]

The Computational Geometry Algorithms Library (CGAL). http://www.cgal.org/.

[3]

ESRI 3D GIS. http://www.esri.com/products/arcgis-capabilities/3d-gis.

[4]

FDAWSI. https://www.fda.gov/newsevents/newsroom/ucm552742.htm.

[5]

OFF Format. https://en.wikipedia.org/wiki/OFF_(file_format).

[6]

OpenTopography. http://www.opentopography.org/.

[7]

pbEncom. http://www.pitneybowes.com/pbencom/homepage.html.

[8]

Spatial Index Library. http://libspatialindex.github.com.

[9]

Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, and Joel Saltz. 2013. Hadoop GIS: a high performance spatial data warehousing system over mapreduce. Proceedings of the VLDB Endowment 6, 11 (2013), 1009--1020.

Digital Library

[10]

Lars Arge, Octavian Procopiuc, Sridhar Ramaswamy, Torsten Suel, Jan Vahrenhold, and Jeffrey Scott Vitter. 2000. A unified approach for indexed and non-indexed spatial joins. In International Conference on Extending Database Technology. Springer, 413--429.

Digital Library

[11]

Norbert Beckmann, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1990. The R*-tree: An Efficient and Robust Access Method for Points and Rectangles. In SIGMOD. 322--331.

Digital Library

[12]

J Bercken and Bernhard Seeger. 2001. An evaluation of generic bulk loading techniques. In Proc. of VLDB. 461--470.

Digital Library

[13]

Thomas Brinkhoff, Hans-Peter Kriegel, and Bernhard Seeger. 1996. Parallel processing of spatial joins using R-trees. In Data Engineering, 1996. IEEE, 258--265.

Digital Library

[14]

Nikki A. Charles, Eric C. Holland, Richard Gilbertson, Rainer Glass, and Helmut Kettenmann. 2012. The brain tumor microenvironment. Glia 60, 3 (2012), 502--514.

[15]

A. Davidson and A. Or. 2013. Optimizing Shuffle Performance in Spark. Technique Report. UC Berkeley.

[16]

Ahmed Eldawy and Mohamed F Mokbel. 2015. SpatialHadoop: A MapReduce framework for spatial data. In Data Engineering (ICDE) 2015. IEEE, 1352--1363.

[17]

Hugues Hoppe. 1996. Progressive meshes. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 99--108.

Digital Library

[18]

Yanhui Liang, Hoang Vo, Ablimit Aji, Jun Kong, and Fusheng Wang. 2016. Scalable 3D spatial queries for analytical pathology imaging with MapReduce. In The 24th ACM SIGSPATIAL GIS. ACM, 52.

Digital Library

[19]

Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. A 3D Primary Vessel Reconstruction Framework with Serial Microscopy Images. In MICCAI 2015. Springer, 251--259.

[20]

Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. Liver whole slide image analysis for 3D vessel reconstruction. In ISBI 2015. IEEE, 182--185.

[21]

Adrien Maglo, CléMent Courbet, Pierre Alliez, and Céline Hudelot. 2012. Progressive compression of manifold polygon meshes. Computers & Graphics 36, 5 (2012), 349--359.

Digital Library

[22]

Kenton McHenry and Peter Bajcsy. 2008. An Overview of 3D Data Content, File Formats and Viewers.

[23]

Jingliang Peng, Chang-Su Kim, and C-C Jay Kuo. 2005. Technologies for 3D mesh compression: A survey. Journal of Visual Communication and Image Representation 16, 6 (2005), 688--733.

Digital Library

[24]

Tohru Takahashi. 2011. Pathology of organ structure by analysis and interpretation of images (second edition ed.). SciPress, Tokyo.

[25]

Mingjie Tang, Yongyang Yu, Qutaibah M Malluhi, Mourad Ouzzani, and Walid G Aref. 2016. Locationspark: a distributed in-memory data management system for big spatial data. Proceedings of the VLDB Endowment 9, 13 (2016), 1565--1568.

Digital Library

[26]

Pierre Terdiman. OPCODE 3D collision detection library, 2005.

[27]

Hoang Vo, Ablimit Aji, and Fusheng Wang. 2014. SATO: a spatial data partitioning framework for scalable query processing. In The 22nd ACM SIGSPATIAL GIS. ACM, 545--548.

Digital Library

[28]

Fusheng Wang, Jun Kong, Jingjing Gao, Lee AD Cooper, Tahsin Kurc, Zhengwen Zhou, David Adler, Cristobal Vergara-Niedermayr, Bryan Katigbak, Daniel J Brat, et al. 2013. A high-performance spatial database based approach for pathology imaging algorithm evaluation. J. of pathology informatics 4, 1 (2013), 5.

[29]

Kaibo Wang, Yin Huai, Rubao Lee, Fusheng Wang, Xiaodong Zhang, and Joel H. Saltz. 2012. Accelerating Pathology Image Data Cross-comparison on CPU-GPU Hybrid Systems. Proc. VLDB Endow. 5, 11 (July 2012), 1543--1554.

Digital Library

[30]

Dong Xie, Feifei Li, Bin Yao, Gefei Li, Liang Zhou, and Minyi Guo. 2016. Simba: Efficient in-memory spatial analytics. In ICMD 2016. ACM, 1071--1085.

Digital Library

[31]

Simin You, Jianting Zhang, and Le Gruenwald. 2015. Large-scale spatial join query processing in cloud. In Data Engineering Workshops (ICDEW). IEEE, 34--41.

[32]

Jia Yu, Jinxuan Wu, and Mohamed Sarwat. 2015. Geospark: A cluster computing framework for processing large-scale spatial data. In The 23rd ACM SIGSPATIAL GIS. ACM, 70.

Digital Library

[33]

Xiaofang Zhou, David J Abel, and David Truffet. 1998. Data partitioning for parallel spatial join processing. Geoinformatica 2, 2 (1998), 175--204.

Digital Library

Cited By

Teng DBaig FPeng ZKong JWang F(2023)Efficient spatial queries over complex polygons with hybrid representationsGeoInformatica10.1007/s10707-023-00508-228:3(459-497)Online publication date: 27-Dec-2023
https://doi.org/10.1007/s10707-023-00508-2
Chen LTeng DZhu TKong JHerr BBueckle ABörner KWang F(2022)Real-time spatial registration for 3D human atlasProceedings of the 10th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data10.1145/3557917.3567618(27-35)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1145/3557917.3567618
Xiao MWang HGeng LLee RZhang X(2022)An RDMA-enabled In-memory Computing Platform for R-tree on ClustersACM Transactions on Spatial Algorithms and Systems10.1145/35035138:2(1-26)Online publication date: 12-Feb-2022
https://dl.acm.org/doi/10.1145/3503513
Show More Cited By

Index Terms

iSPEED: an Efficient In-Memory Based Spatial Query System for Large-Scale 3D Data with Complex Structures
1. Information systems
  1. Data management systems
    1. Database management system engines

Recommendations

Scalable 3D spatial queries for analytical pathology imaging with MapReduce
SIGSPACIAL '16: Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

3D analytical pathology imaging examines high resolution 3D image volumes of human tissues to facilitate biomedical research and provide potential effective diagnostic assistance. Such approach - quantitative analysis of large- scale 3D pathology image ...
iSPEED: a scalable and distributed in-memory based spatial query system for large and structurally complex 3D data

The recent technological advancement in digital pathology has enabled 3D tissue-based investigation of human diseases at extremely high resolutions. Discovering and verifying spatial patterns among massive 3D micro-anatomic biological objects such as ...
Dynamic Management of In-memory Storage for Efficiently Integrating Compute- and Data-intensive Computing on HPC Systems
CCGrid '17: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

In order to boost the performance of data-intensive computing on HPC systems, in-memory computing frameworks, such as Apache Spark and Flink, use local DRAM for data storage. Optimizing the memory allocation to data storage is critical to delivering ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '17: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2017

677 pages

ISBN:9781450354905

DOI:10.1145/3139958

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGSPATIAL'17

Sponsor:

SIGSPATIAL

SIGSPATIAL'17: 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 7 - 10, 2017

CA, Redondo Beach, USA

Acceptance Rates

SIGSPATIAL '17 Paper Acceptance Rate 39 of 193 submissions, 20%;

Overall Acceptance Rate 220 of 1,116 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
262
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Teng DBaig FPeng ZKong JWang F(2023)Efficient spatial queries over complex polygons with hybrid representationsGeoInformatica10.1007/s10707-023-00508-228:3(459-497)Online publication date: 27-Dec-2023
https://doi.org/10.1007/s10707-023-00508-2
Chen LTeng DZhu TKong JHerr BBueckle ABörner KWang F(2022)Real-time spatial registration for 3D human atlasProceedings of the 10th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data10.1145/3557917.3567618(27-35)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1145/3557917.3567618
Xiao MWang HGeng LLee RZhang X(2022)An RDMA-enabled In-memory Computing Platform for R-tree on ClustersACM Transactions on Spatial Algorithms and Systems10.1145/35035138:2(1-26)Online publication date: 12-Feb-2022
https://dl.acm.org/doi/10.1145/3503513
Teng DLiang YVo HKong JWang F(2022)Efficient 3D Spatial Queries for Complex ObjectsACM Transactions on Spatial Algorithms and Systems10.1145/35022218:2(1-26)Online publication date: 12-Feb-2022
https://dl.acm.org/doi/10.1145/3502221
Liu ZHua WLiu XLiang DZhao YShi M(2021)An Efficient Group-Based Replica Placement Policy for Large-Scale Geospatial 3D Raster Data on HadoopSensors10.3390/s2123813221:23(8132)Online publication date: 5-Dec-2021
https://doi.org/10.3390/s21238132
Kim HYoon HThakur NHwang GLee EKim CChong Y(2021)Deep learning-based histopathological segmentation for whole slide images of colorectal cancer in a compressed domainScientific Reports10.1038/s41598-021-01905-z11:1Online publication date: 18-Nov-2021
https://doi.org/10.1038/s41598-021-01905-z
Xiao MWang HGeng LLee RZhang X(2019)Catfish: Adaptive RDMA-enabled R-Tree for Low Latency and High Throughput2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS.2019.00025(164-175)Online publication date: Jul-2019
https://doi.org/10.1109/ICDCS.2019.00025
Vo HLiang YKong JWang F(2018)iSPEEDProceedings of the VLDB Endowment10.14778/3229863.323626411:12(2078-2081)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.14778/3229863.3236264

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents