Abstract
Many real-world applications need to know the distances between some or all pairs of vertices in a large graph, but the existing distributed single-source shortest path (SSSP) solution can not efficiently handle the problem of such multi-source computation. While, distributed multi-source shortest path (MSSP) has not attracted enough attention. This paper thereby proposes a distributed multi-source shortest path algorithm by scheduling source vertices in different batches and sharing graph traversal operations in the same batch for efficiency (B-MSSP), which effectively solves the performance bottleneck of MSSP on big graphs. To correctly run multi source vertices in the same batch, a message cluster mechanism is designed to effectively manage the communication. A further analysis reveals that the overall performance of B-MSSP is affected by the batch size. Thus, an adaptive cost-benefit model is proposed to quickly and easily select a proper batch size for better performance. Extensive experiments over lots of real graphs validate the efficiency and effectiveness of our proposals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Then, M., et al.: The more the merrier: efficient multi-source graph traversal. Proc. VLDB Endow. 8(4), 449–460 (2014)
Then, M., Günnemann, S., Kemper, A., Neumann, T.: Efficient batched distance, closeness and betweenness centrality computation in unweighted and weighted graphs. Datenbank-Spektrum 17(2), 169–182 (2017)
Kaufmann, M., Then, M., Kemper, A., Neumann, T.: Parallel array-based single-and multi-source breadth first searches on large dense graphs. In: EDBT, pp. 1–12 (2017)
Shen, D.: Lower bounds on rate of convergence of matrix products in all pairs shortest path of social network. arXiv preprint arXiv:2006.13412 (2020)
Kang, S.J., Lee, S.Y., Lee, K.M.: Performance comparison of OpenMP, MPI, and MapReduce in practical problems. In: Advances in Multimedia 2015 (2015)
Pradhan, A., Mahinthakumar, G.: Finding all-pairs shortest path for a large-scale transportation network using parallel Floyd-Warshall and parallel Dijkstra algorithms. J. Comput. Civil Eng. 27(3), 263–273 (2013)
Yang, L., Li, R., Tang, Z.: Research on the single source shortest path algorithm using MapReduce. Microcomput. Inf. 27(12), 97–99 (2011). (in Chinese)
Adoni, W.Y.H., Nahhal, T., Aghezzaf, B., Elbyed, A.: The MapReduce-based approach to improve the shortest path computation in large-scale road networks: the case of A* algorithm. J. Big Data 5(1), 1–24 (2018)
Gao, Y., Yao, L., Yu, J.: Research of query verification algorithm on body sensing data in cloud computing environment. In: Ni, W., Wang, X., Song, W., Li, Y. (eds.) WISA 2019. LNCS, vol. 11817, pp. 176–188. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30952-7_20
Arfat, Y., Suma, S., Mehmood, R., Albeshri, A.: Parallel shortest path big data graph computations of US road network using apache spark: survey, architecture, and evaluation. In: Mehmood, R., See, S., Katib, I., Chlamtac, I. (eds.) Smart Infrastructure and Applications. EICC, pp. 185–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-13705-2_8
Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD, pp. 135–146 (2010)
Liu, H., Huang, H.H., Hu, Y.: IBFS: concurrent breadth-first search on GPUs. In: the 2016 International Conference on Management of Data, pp. 403–416 (2016)
Zhang, Y., et al.: CGraph: a correlations-aware approach for efficient concurrent iterative graph processing. In: 2018 Annual Technical Conference, pp. 441–452 (2018)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (61902366 and 61902365), the Fundamental Research Funds for the Central Universities (202042008), the Project funded by China Postdoctoral Science Foundation (2020T130623, 2019M652474 and 2019M652473), the Projects funded by Postdoctoral Creative Foundation in Shandong Province and Postdoctoral Application&Research Foundation in Qingdao City, the Qingdao Independent Innovation Major Project (20-3-2-12-xx).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X. et al. (2021). An Adaptive Sharing Framework for Efficient Multi-source Shortest Path Computation. In: Xing, C., Fu, X., Zhang, Y., Zhang, G., Borjigin, C. (eds) Web Information Systems and Applications. WISA 2021. Lecture Notes in Computer Science(), vol 12999. Springer, Cham. https://doi.org/10.1007/978-3-030-87571-8_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-87571-8_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87570-1
Online ISBN: 978-3-030-87571-8
eBook Packages: Computer ScienceComputer Science (R0)
