Abstract
An increasing number of telescopes are being built to provide multi-band data of celestial objects, which is indispensable to astronomical research. The amount of collected observation data, however, have put tremendous pressure on computing systems. Moreover, with the development of the observation equipment, the number of astronomical catalogs that telescopes generated per day keeps increasing rapidly. In this paper, we propose a distributed cone search indexing system (DCSIS) for Multi-Band Astronomical Catalogs among multi-band astronomical catalogs to solve this problem. Major contributions of DCSIS include defining new meta file format for astronomical catalogs, achieving scalability and parallelism for cone search, and the ability to flexibly add data to index system. Evaluations are performed on the Tianhe-1A supercomputer to showcase DCSIS’ scalability for large scale deployment, the results of which show that DCSIS reduce the response time of Multi-band cone search into a tolerant range.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Crab nebula messier 1. http://www.constellation-guide.com/crab-nebula-messier-1/
Brown, P.G.: Overview of sciDB: large scale array storage, processing and analysis. In: ACM SIGMOD International Conference on Management of Data, pp. 963–968 (2010)
Chilingarian, I., Bartunov, O., Richter, J., Sigaev, T.: PostgreSQL: the suitable DBMS solution for astronomy and astrophysics, vol. 314, p. 225 (2004)
Derriere, S., Ochsenbein, F., Egret, D.: On-line access to very large catalogues. In: Astronomical Data Analysis Software and Systems IX, vol. 216, p. 235 (2000)
Djorgovski, S.G., Williams, R.: Virtual observatory: From concept to implementation. Computer Science (2005)
Google: Google protocol buffers homepage. https://developers.google.com/protocol-buffers/
Hong, Z., Yu, C., Xia, R., Xiao, J., Wang, J., Sun, J., Cui, C.: AQUAdex: a highly efficient indexing and retrieving method for astronomical big data of time series images. In: Wang, G., Zomaya, A., Perez, G.M., Li, K. (eds.) ICA3PP 2015. LNCS, vol. 9529, pp. 92–105. Springer, Cham (2015). doi:10.1007/978-3-319-27122-4_7
Hu, L., Nooshabadi, S., Ahmadi, M.: Massively parallel KD-tree construction and nearest neighbor search algorithms. In: IEEE International Symposium on Circuits and Systems, pp. 2752–2755 (2015)
Landais, G., Ochsenbein, F.: The new version of the vizier catalogue service. Mod. Lang. Notes 461(3), 383 (2012)
NASA: Jet propulsion laboratory healpix homepage. http://healpix.jpl.nasa.gov/
Ochsenbein, F.: The vizier system for accessing astronomical data. vol. 145, p. 387 (1998)
Ochsenbein, F., Derriere, S., Nicaisse, S., Schaaff, A.: Clustering the large vizier catalogues, the cocat experience. vol. 314, p. 58 (2004)
Ochsenbein, F., Bauer, P., Marcout, J.: Astronomy and astrophysics the vizier database of astronomical catalogues (2000)
Plante, R., Williams, R., Hanisch, R., Szalay, A.: Simple cone search version 1.03 (2008)
Planthaber, G., Stonebraker, M., Frew, J.: EarthDB: scalable analysis of MODIS data using sciDB. In: ACM Sigspatial International Workshop on Analytics for Big Geospatial Data, pp. 11–19 (2012)
Richter, S., Schuh, S., Dittrich, J.: Towards zero-overhead static and adaptive indexing in hadoop. VLDB J. 23(3), 469–494 (2014)
Smareglia, R., Laurino, O., Knapic, C.: VODance: VO data access layer service creation made easy. vol. 442, p. 575 (2011)
Stonebraker, M., Brown, P., Poliakov, A., Raman, S.: The architecture of sciDB. In: Bayard Cushing, J., French, J., Bowers, S. (eds.) SSDBM 2011. LNCS, vol. 6809, pp. 1–16. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22351-8_1
Xinhua: Xinhua insight: Installation complete on world’s largest radio telescope. http://news.xinhuanet.com/english/2016-07/03/c_135485389.htm
Yang, X.J., Liao, X.K., Lu, K., Hu, Q.F., Song, J.Q., Su, J.S.: The TianHe-1A supercomputer: its hardware and software. J. Comput. Sci. Technol. 26(3), 344–351 (2011)
Zhao, Q., Sun, J., Yu, C., Xiao, J., Cui, C., Zhang, X.: Improved parallel processing function for high-performance large-scale astronomical cross-matching. Trans. Tianjin Univ. 17(1), 62–67 (2011)
Acknowledgements
This work is supported by the Joint Research Fund in Astronomy (U1531111) under cooperative agreement between the National Natural Science Foundation of China (NSFC) and Chinese Academy of Sciences (CAS), the National Natural Science Foundation of China (11573019, 61602336). Special thanks goes to Mr. Zhi Hong for providing writing assistance for the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, C. et al. (2017). KD-Tree and HEALPix-Based Distributed Cone Search Indexing System for Multi-Band Astronomical Catalogs. In: Ibrahim, S., Choo, KK., Yan, Z., Pedrycz, W. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2017. Lecture Notes in Computer Science(), vol 10393. Springer, Cham. https://doi.org/10.1007/978-3-319-65482-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-65482-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65481-2
Online ISBN: 978-3-319-65482-9
eBook Packages: Computer ScienceComputer Science (R0)