Efficient 3D Spatial Queries for Complex Objects
Article No.: 14, Pages 1 - 26
Abstract
3D spatial data has been generated at an extreme scale from many emerging applications, such as high definition maps for autonomous driving and 3D Human BioMolecular Atlas. In particular, 3D digital pathology provides a revolutionary approach to map human tissues in 3D, which is highly promising for advancing computer-aided diagnosis and understanding diseases through spatial queries and analysis. However, the exponential increase of data at 3D leads to significant I/O, communication, and computational challenges for 3D spatial queries. The complex structures of 3D objects such as bifurcated vessels make it difficult to effectively support 3D spatial queries with traditional methods. In this article, we present our work on building an efficient and scalable spatial query system, iSPEED, for large-scale 3D data with complex structures. iSPEED adopts effective progressive compression for each 3D object with successive levels of detail. Further, iSPEED exploits structural indexing for complex structured objects in distance-based queries. By querying with data represented in successive levels of details and structural indexes, iSPEED provides an option for users to balance between query efficiency and query accuracy. iSPEED builds in-memory indexes and decompresses data on-demand, which has a minimal memory footprint. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. We evaluate iSPEED with three representative queries: 3D spatial joins, 3D nearest neighbor query, and 3D spatial proximity estimation. The extensive experiments demonstrate that iSPEED significantly improves the performance of existing spatial query systems.
References
[1]
Wikipedia. 3D Modelling. Retrieved from https://en.wikipedia.org/wiki/3D_modeling.
[2]
CARMERA. Retrieved from https://www.carmera.com/.
[3]
Civil Maps. Retrieved from https://civilmaps.com/.
[4]
The Computational Geometry Algorithms Library (CGAL). Retrieved from http://www.cgal.org/.
[5]
Nvidia. Deepmap. Retrieved from https://www.deepmap.ai/.
[6]
ESRI. ESRI 3D GIS. Retrieved from http://www.esri.com/products/arcgis-capabilities/3d-gis.
[7]
U. S. Food and Drug. FDAWSI. Retrieved from https://www.fda.gov/newsevents/newsroom/ucm552742.htm.
[8]
Nanalyze. HD Map. Retrieved from https://www.nanalyze.com/2018/11/hd-mapping-autonomous-vehicles.
[9]
Mapbox.ai. Retrieved from https://www.mapbox.com/.
[10]
Mapper. Retrieved from https://mapper.ai/.
[11]
Wikipedia. OFF Format. Retrieved from https://en.wikipedia.org/wiki/OFF_(file_format).
[12]
OpenTopography. Retrieved from http://www.opentopography.org/.
[13]
Pitney Bowes Inc. pbEncom. Retrieved from http://www.pitneybowes.com/pbencom/homepage.html.
[14]
The Human Protein Atlas project. Proteinatlas. Retrieved from https://www.proteinatlas.org/.
[15]
National Library of Medicine. Spatial Epigenome. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/31127980.
[16]
Marios Hadjieleftheriou. Spatial Index Library. Retrieved from http://libspatialindex.github.com.
[17]
Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, and Joel Saltz. 2013. Hadoop GIS: A high performance spatial data warehousing system over MapReduce. Proc. VLDB Endow. 6, 11 (2013), 1009–1020.
[18]
Mohammed Eunus Ali, Egemen Tanin, Rui Zhang, and Lars Kulik. 2010. A motion-aware approach for efficient evaluation of continuous queries on 3D object databases. VLDB J. 19, 5 (2010), 603–632.
[19]
Mohammed Eunus Ali, Rui Zhang, Egemen Tanin, and Lars Kulik. 2008. A motion-aware approach to continuous retrieval of 3D objects. In IEEE 24th International Conference on Data Engineering. IEEE, 843–852.
[20]
Lars Arge, Octavian Procopiuc, Sridhar Ramaswamy, Torsten Suel, Jan Vahrenhold, and Jeffrey Scott Vitter. 2000. A unified approach for indexed and non-indexed spatial joins. In International Conference on Extending Database Technology. Springer, 413–429.
[21]
Franz Aurenhammer, Rolf Klein, Der-Tsai Lee, and Rolf Klein. 2013. Voronoi Diagrams and Delaunay Triangulations, Vol. 8. World Scientific.
[22]
Furqan Baig, Hoang Vo, Tahsin Kurc, Joel Saltz, and Fusheng Wang. 2017. SparkGIS: Resource aware efficient in-memory spatial query processing. In 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 1–10.
[23]
Norbert Beckmann, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1990. The R*-tree: An efficient and robust access method for points and rectangles. In SIGMOD. 322–331.
[24]
J. Bercken and Bernhard Seeger. 2001. An evaluation of generic bulk loading techniques. In Proc. of VLDB. 461–470.
[25]
Kevin Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. 1999. When is “nearest neighbor” meaningful? In Database Theory - ICDT 99. Springer, 217–235.
[26]
Thomas Brinkhoff, Hans-Peter Kriegel, and Bernhard Seeger. 1996. Parallel processing of spatial joins using R-trees. In 12th International Conference on Data Engineering. IEEE, 258–265.
[27]
Nikki A. Charles, Eric C. Holland, Richard Gilbertson, Rainer Glass, and Helmut Kettenmann. 2012. The brain tumor microenvironment. Glia 60, 3 (2012), 502–514. DOI:DOI:
[28]
Paolo Cignoni, Claudio Rocchini, and Roberto Scopigno. 1998. Metro: Measuring error on simplified surfaces. In Computer Graphics Forum, Vol. 17. Wiley Online Library, 167–174.
[29]
HuBMAP Consortium et al. 2019. The human body at cellular resolution: The NIH human BioMolecular Atlas Program. Nature 574, 7777 (2019), 187.
[30]
A. Davidson and A. Or. 2013. Optimizing Shuffle Performance in Spark. Technique Report. UC Berkeley.
[31]
Ahmed Eldawy and Mohamed F. Mokbel. 2015. SpatialHadoop: A MapReduce framework for spatial data. In IEEE 31st International Conference on Data Engineering. IEEE, 1352–1363.
[32]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In 2nd International Conference on Knowledge Discovery and Data Mining. 226–231.
[33]
Soohee Han. 2013. Design of memory-efficient octree to query large 3D point cloud. J. Kor. Societ. Survey., Geodes., Photogram. Cartog. 31, 1 (2013), 41–48.
[34]
Hugues Hoppe. 1996. Progressive meshes. In 23rd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 99–108.
[35]
Edwin H. Jacox and Hanan Samet. 2007. Spatial join techniques. ACM Trans. Datab. Syst. 32, 1 (2007), 7.
[36]
Ibrahim Kamel and Christos Faloutsos. 1993. Hilbert R-tree: An Improved R-tree Using Fractals. Technical Report.
[37]
Andrei Khodakovsky, Peter Schröder, and Wim Sweldens. 2000. Progressive geometry compression. In 27th Annual Conference on Computer Graphics and Interactive Techniques. 271–278.
[38]
Yanhui Liang, Hoang Vo, Ablimit Aji, Jun Kong, and Fusheng Wang. 2016. Scalable 3D spatial queries for analytical pathology imaging with MapReduce. In 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 1–4.
[39]
Yanhui Liang, Hoang Vo, Jun Kong, and Fusheng Wang. 2017. iSPEED: An efficient in-memory-based spatial query system for large-scale 3D data with complex structures. In 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 17.
[40]
Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. A 3D primary vessel reconstruction framework with serial microscopy images. In Medical Image Computing and Computer-assisted Intervention–MICCAI 2015. Springer, 251–259.
[41]
Yanhui Liang, Fusheng Wang, Darren Treanor, Derek Magee, George Teodoro, Yangyang Zhu, and Jun Kong. 2015. Liver whole slide image analysis for 3D vessel reconstruction. In IEEE 12th International Symposium on Biomedical Imaging (ISBI). IEEE, 182–185.
[42]
Yanhui Liang, Fusheng Wang, Pengyue Zhang, Joel H. Saltz, Daniel J. Brat, and Jun Kong. 2017. Development of a framework for large-scale three-dimensional pathology and biomarker imaging and spatial analytics. AMIA Summ. Translat. Sci. Proc. 2017 (2017), 75.
[43]
Harvey Lodish, Arnold Berk, S. Lawrence Zipursky, Paul Matsudaira, David Baltimore, and James Darnell. 2000. Viruses: structure, function, and uses. In Molecular Cell Biology. 4th edition. WH Freeman.
[44]
David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. 2003. Level of Detail for 3D Graphics. Morgan Kaufmann.
[45]
Adrien Maglo, Clment Courbet, Pierre Alliez, and Céline Hudelot. 2012. Progressive compression of manifold polygon meshes. Comput. Graph. 36, 5 (2012), 349–359.
[46]
Kenton McHenry and Peter Bajcsy. 2008. An Overview of 3D Data Content, File Formats and Viewers. Retrieved from https://isda.ncsa.illinois.edu/drupal/sites/default/files/NCSA-ISDA-2008-002.pdf.
[47]
Oracle. Oracle 3D. Retrieved from https://docs.oracle.com/database/121/SPATL/three-dimensional-spatial-objects.htm#SPATL468.
[48]
M. Tamer Özsu and Patrick Valduriez. 2011. Principles of Distributed Database Systems. Springer Science & Business Media.
[49]
Jignesh Patel, JieBing Yu, Navin Kabra, Kristin Tufte, Biswadeep Nag, Josef Burger, Nancy Hall, Karthikeyan Ramasamy, Roger Lueder, Curt Ellmann, et al. 1997. Building a scaleable geo-spatial DBMS: Technology, implementation, and evaluation. In ACM SIGMOD Record, Vol. 26. ACM, 336–347.
[50]
Jingliang Peng, Yan Huang, C.-C. Jay Kuo, Ilya Eckstein, and M. Gopi. 2010. Feature oriented progressive lossless mesh coding. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 2029–2038.
[51]
Jingliang Peng, Chang-Su Kim, and C.-C. Jay Kuo. 2005. Technologies for 3D mesh compression: A survey. J. Vis. Commun. Image Repres. 16, 6 (2005), 688–733.
[52]
Jingliang Peng and C.-C. Jay Kuo. 2005. Geometry-guided progressive lossless 3D mesh coding with octree (OT) decomposition. In ACM Transactions on Graphics (TOG), Vol. 24. ACM, 609–616.
[53]
Lucas C. Villa Real, Bruno Silva, Dikran S. Meliksetian, and Kaique Sacchi. 2019. Large-scale 3D geospatial processing made possible. In 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 199–208.
[54]
Nick Roussopoulos, Stephen Kelley, and Frédéric Vincent. 1995. Nearest neighbor queries. In ACM SIGMOD Record, Vol. 24. ACM, 71–79.
[55]
Orit Rozenblatt-Rosen, Aviv Regev, Philipp Oberdoerffer, Tal Nawy, Anna Hupalowska, Jennifer E. Rood, Orr Ashenberg, Ethan Cerami, Robert J. Coffey, Emek Demir, et al. 2020. The Human Tumor Atlas Network: Charting tumor transitions across space and time at single-cell resolution. Cell 181, 2 (2020), 236–249.
[56]
Eric J. Stollnitz, Tony D. DeRose, Anthony D. DeRose, and David H. Salesin. 1996. Wavelets for Computer Graphics: Theory and Applications. Morgan Kaufmann.
[57]
Andrea Tagliasacchi, Ibraheem Alhashim, Matt Olson, and Hao Zhang. 2012. Mean curvature skeletons. In Computer Graphics Forum. Wiley Online Library, 1735–1744.
[58]
Tohru Takahashi. 2011. Pathology of Organ Structure by Analysis and Interpretation of Images (2nd ed.). SciPress, Tokyo.
[59]
Mingjie Tang, Yongyang Yu, Qutaibah M. Malluhi, Mourad Ouzzani, and Walid G. Aref. 2016. LocationSpark: A distributed in-memory data management system for big spatial data. Proc. VLDB Endowm. 9, 13 (2016), 1565–1568.
[60]
David Taniar and Wenny Rahayu. 2013. A taxonomy for nearest neighbour queries in spatial databases. J. Comput. Syst. Sci. 79, 7 (2013), 1017–1039.
[61]
Pierre Terdiman. [n.d.]. OPCODE 3D Collision Detection Library, 2005. http://www.codercorner.com/Opcode.htm.
[62]
Sébastien Valette, Raphaëlle Chaine, and Rémy Prost. 2009. Progressive lossless mesh compression via incremental parametric refinement. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 1301–1310.
[63]
Hoang Vo, Ablimit Aji, and Fusheng Wang. 2014. SATO: A spatial data partitioning framework for scalable query processing. In 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 545–548.
[64]
Hoang Vo, Yanhui Liang, Jun Kong, and Fusheng Wang. 2018. iSPEED: A scalable and distributed in-memory-based spatial query system for large and structurally complex 3D data. Proc. VLDB Endow. 11, 12 (2018), 2078–2081.
[65]
Fusheng Wang, Jun Kong, Jingjing Gao, Lee A. D. Cooper, Tahsin Kurc, Zhengwen Zhou, David Adler, Cristobal Vergara-Niedermayr, Bryan Katigbak, Daniel J. Brat, et al. 2013. A high-performance spatial database-based approach for pathology imaging algorithm evaluation. J. Pathol. Inform. 4, 1 (2013), 5.
[66]
Kaibo Wang, Yin Huai, Rubao Lee, Fusheng Wang, Xiaodong Zhang, and Joel H. Saltz. 2012. Accelerating pathology image data cross-comparison on CPU-GPU hybrid systems. Proc. VLDB Endow. 5, 11 (July 2012), 1543–1554. DOI:DOI:https://doi.org/10.14778/2350229.2350268
[67]
Mei-Tzu Wang. 2016. Nearest neighbor query processing using the network voronoi diagram. Data Knowl. Eng. 103 (2016), 19–43.
[68]
Xunxian Wang, Susan Lindsay, and Richard Baldock. 2010. From spatial-data to 3D models of the developing human brain. Methods 50, 2 (2010), 96–104.
[69]
Dong Xie, Feifei Li, Bin Yao, Gefei Li, Liang Zhou, and Minyi Guo. 2016. Simba: Efficient in-memory spatial analytics. In International Conference on Management of Data. ACM, 1071–1085.
[70]
Simin You, Jianting Zhang, and Le Gruenwald. 2015. Large-scale spatial join query processing in cloud. In 31st IEEE International Conference on Data Engineering Workshops. IEEE, 34–41.
[71]
Jia Yu, Jinxuan Wu, and Mohamed Sarwat. 2015. GeoSpark: A cluster computing framework for processing large-scale spatial data. In 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 70.
[72]
Andi Zang, Shiyu Luo, Xin Chen, and Goce Trajcevski. 2019. Real-time applications using high resolution 3D objects in high definition maps (systems paper). In 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 229–238.
[73]
Shubin Zhang, Jizhong Han, Zhiyong Liu, Kai Wang, and Zhiyong Xu. 2009. SJMR: Parallelizing spatial join with MapReduce on clusters. In IEEE International Conference on Cluster Computing and Workshops. IEEE, 1–8.
Index Terms
- Efficient 3D Spatial Queries for Complex Objects
Recommendations
Efficient Bundled Spatial Range Queries
SIGSPATIAL '19: Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information SystemsEfficiently querying multiple spatial data sets is a growing challenge for scientists. Astronomers query data sets that contain different types of stars (e.g., dwarfs, giants, stragglers) while neuroscientists query different data sets that model ...
Towards Parallel Spatial Query Processing for Big Spatial Data
IPDPSW '12: Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD ForumIn recent years, spatial applications have become more and more important in both scientific research and industry. Spatial query processing is the fundamental functioning component to support spatial applications. However, the state-of-the-art ...
Selectivity Estimation of Complex Spatial Queries
SSTD '01: Proceedings of the 7th International Symposium on Advances in Spatial and Temporal DatabasesSeveral studies have focused on the efficient processing of simple spatial query types such as selections and spatial joins. Little work, however, has been done towards the optimization of queries that process several spatial inputs and combine them ...
Comments
Information & Contributors
Information
Published In
June 2022
253 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 February 2022
Accepted: 01 November 2021
Revised: 01 August 2021
Received: 01 January 2021
Published in TSAS Volume 8, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
- Refereed
Funding Sources
- National Science Foundation
- National Institute of Health
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 635Total Downloads
- Downloads (Last 12 months)122
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format