Simba: Efficient in-memory spatial analytics

D Xie, F Li, B Yao, G Li, L Zhou, M Guo - Proceedings of the 2016 …, 2016 - dl.acm.org
D Xie, F Li, B Yao, G Li, L Zhou, M Guo
Proceedings of the 2016 international conference on management of data, 2016dl.acm.org
Large spatial data becomes ubiquitous. As a result, it is critical to provide fast, scalable, and
high-throughput spatial queries and analytics for numerous applications in location-based
services (LBS). Traditional spatial databases and spatial analytics systems are disk-based
and optimized for IO efficiency. But increasingly, data are stored and processed in memory
to achieve low latency, and CPU time becomes the new bottleneck. We present the Simba
(Spatial In-Memory Big data Analytics) system that offers scalable and efficient in-memory …
Large spatial data becomes ubiquitous. As a result, it is critical to provide fast, scalable, and high-throughput spatial queries and analytics for numerous applications in location-based services (LBS). Traditional spatial databases and spatial analytics systems are disk-based and optimized for IO efficiency. But increasingly, data are stored and processed in memory to achieve low latency, and CPU time becomes the new bottleneck. We present the Simba (Spatial In-Memory Big data Analytics) system that offers scalable and efficient in-memory spatial query processing and analytics for big spatial data. Simba is based on Spark and runs over a cluster of commodity machines. In particular, Simba extends the Spark SQL engine to support rich spatial queries and analytics through both SQL and the DataFrame API. It introduces indexes over RDDs in order to work with big spatial data and complex spatial operations. Lastly, Simba implements an effective query optimizer, which leverages its indexes and novel spatial-aware optimizations, to achieve both low latency and high throughput. Extensive experiments over large data sets demonstrate Simba's superior performance compared against other spatial analytics system.
ACM Digital Library