Cited By
View all- Udhayasekar SSrinivasan KKumar PChilukuri B(2024)Label-Setting Algorithm for Multi-Destination K Simple Shortest Paths Problem and ApplicationAlgorithms10.3390/a1708032517:8(325)Online publication date: 25-Jul-2024
PeeK: A Prune-Centric Approach for K Shortest Path Computation
Authors: 
Wang Feng, Shiyang Chen, Hang Liu, Yuede JiAuthors Info & Claims
Wang Feng, Shiyang Chen, Hang Liu, Yuede JiAuthors Info & ClaimsArticle No.: 18, Pages 1 - 14
Abstract
The K shortest path (KSP) algorithm, which finds the top K shortest simple paths from a source to a target vertex, has a wide range of real-world applications, e.g., routing, vulnerability detection, and biology analysis. While the top K shortest simple paths offer invaluable insights, computing them is time-consuming. For example, on a Twitter graph (61.6M vertices and 1.5B edges), the best parallel method needs about 20 minutes to get 128 shortest paths between two vertices. A key observation we made is existing works search K shortest paths from the original graph, while top K shortest paths only cover a meager portion of the original graph, e.g., less than 0.001% on a Twitter graph for K = 128.
This paper introduces PeeK, a pruning-centric approach for KSP computation. First, PeeK applies K upper bound pruning to prune the vertices and edges that will not appear in any of the K shortest paths. Second, PeeK adaptively compacts the graph that, not only removes the deleted vertices or edges but also efficiently computes the downstream task. Furthermore, we design efficient techniques to parallelize and distribute PeeK. We compare PeeK with five algorithms on various graphs. For parallel computation with 32 threads, PeeK achieves 5.1× and 28.8× speedup over the state-of-the-art for K = 8, 128, respectively. More importantly, when K increases, the runtime of PeeK is barely affected. In particular, when K increases from 2 to 128 (64×), the runtime of PeeK only increases 1.1×, while the state-of-the-art method increases 10.3×.
References
[1]
[n. d.]. Block indirect sort algorithm. https://www.boost.org/doc/libs/1_81_0/libs/sort/doc/html/sort/parallel.html#sort.parallel.block_indirect_sort.
[2]
[n. d.]. Distributed Sorting Algorithms. https://brunomaga.github.io/Distributed-Sort.
[3]
[n. d.]. Graph500: reference implementations. https://github.com/graph500/graph500.
[4]
[n. d.]. Texas Advanced Computing Center. https://www.tacc.utexas.edu.
[5]
Deepak Ajwani, Erika Duriakova, Neil Hurley, Ulrich Meyer, and Alexander Schickedanz. 2018. An Empirical Comparison of K-Shortest Simple Path Algorithms on Multicores. In Proceedings of the 47th International Conference on Parallel Processing (ICPP). 1--12.
[6]
Ali Al Zoobi, David Coudert, and Nicolas Nisse. 2020. Space and time trade-off for the k shortest simple paths problem. In SEA 2020-18th International Symposium on Experimental Algorithms, Vol. 160. 13.
[7]
Ali Al Zoobi, David Coudert, and Nicolas Nisse. 2021. Finding the k Shortest Simple Paths: Time and Space trade-offs. Ph. D. Dissertation. Inria; I3S, Université Côte d'Azur.
[8]
M Albulet. 2016. Spacex non-geostationary satellite system: Attachment a technical information to supplement schedules. US Fed. Commun. Comm., Washington, DC, USA, Rep. SAT-OA-20161115-00118 (2016).
[9]
Renzo Angles and Claudio Gutierrez. 2008. Survey of graph database models. ACM Computing Surveys (CSUR) 40, 1 (2008), 1--39.
[10]
Lars Backstrom, Dan Huttenlocher, Jon Kleinberg, and Xiangyang Lan. 2006. Group formation in large social networks: membership, growth, and evolution. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 44--54.
[11]
Scott Beamer, Krste Asanović, and David Patterson. 2015. The GAP benchmark suite. arXiv preprint arXiv:1508.03619 (2015).
[12]
E Blelloch Guy. 1990. Prefix Sums and Their Applications. Technical Report. Tech. rept. CMU-CS-90-190. School of Computer Science, Carnegie Mellon University.
[13]
Mumeng Bo and Meihui Zhang. 2021. Learning Dynamic Coherence with Graph Attention Network for Biomedical Entity Linking. In 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.
[14]
Benjamin Bowman, Craig Laprade, Yuede Ji, and H. Howie Huang. 2020. Detecting Lateral Movement in Enterprise Computer Networks with Unsupervised Graph AI. In Proceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID).
[15]
Aydin Buluç, Jeremy T Fineman, Matteo Frigo, John R Gilbert, and Charles E Leiserson. 2009. Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks. In Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures. 233--244.
[16]
Aydin Buluç and Kamesh Madduri. 2011. Parallel breadth-first search on distributed memory systems. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, seattle, Washington USA, 1--12.
[17]
Jian Cao, Qiang Li, Yuede Ji, Yukun He, and Dong Guo. 2016. Detection of forwarding-based malicious URLs in online social networks. International Journal of Parallel Programming 44, 1 (2016), 163--180.
[18]
Deepayan Chakrabarti, Yiping Zhan, and Christos Faloutsos. 2004. R-MAT: A recursive model for graph mining. In Proceedings of the 2004 SIAM International Conference on Data Mining. SIAM, 442--446.
[19]
Lei Cui, Jiancong Cui, Yuede Ji, Zhiyu Hao, Lun Li, and Zhenquan Ding. 2023. API2Vec: Learning Representations of API Sequences for Malware Detection. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 261--273.
[20]
Alin Deutsch, Nadime Francis, Alastair Green, Keith Hare, Bei Li, Leonid Libkin, Tobias Lindaaker, Victor Marsault, Wim Martens, Jan Michels, et al. 2022. Graph pattern matching in GQL and SQL/PGQ. In Proceedings of the 2022 International Conference on Management of Data. 2246--2258.
[21]
Laxman Dhulipala, Guy E Blelloch, and Julian Shun. 2019. Low-latency graph streaming using compressed purely-functional trees. In Proceedings of the 40th ACM SIGPLAN conference on programming language design and implementation. 918--934.
[22]
Edsger W Dijkstra. 2022. A note on two problems in connexion with graphs. In Edsger Wybe Dijkstra: His Life, Work, and Legacy. 287--290.
[23]
David Eppstein. 1998. Finding the k shortest paths. SIAM Journal on computing 28, 2 (1998), 652--673.
[24]
Bo Feng, Qiang Li, Yuede Ji, Dong Guo, and Xiangyu Meng. 2019. Stopping the cyberattack in the early stage: assessing the security risks of social network users. Security and Communication Networks 2019 (2019).
[25]
Gang Feng. 2014. Finding k Shortest Simple Paths in Directed Graphs: A Node Classification Algorithm. Networks 64, 1 (2014), 6--17.
[26]
Rod Fleck. 2019. Application of Kuiper Systems LLC for Authority to Launch and Operate a Non-Geostationary Satellite Orbit System in Ka-band Frequencies.
[27]
Qiang Fu, Yuede Ji, and H Howie Huang. 2022. TLPGNN: A lightweight two-level parallelism paradigm for graph neural network computation on GPU. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. 122--134.
[28]
Giorgio Gallo and Stefano Pallottino. 1988. Shortest path algorithms. Annals of operations research 13, 1 (1988), 1--79.
[29]
Giacomo Giuliari, Tommaso Ciussani, Adrian Perrig, and Ankit Singla. 2021. {ICARUS}: Attacking low earth orbit satellite networks. In 2021 USENIX Annual Technical Conference (USENIX ATC 21). 317--331.
[30]
Zvi Gotthilf and Moshe Lewenstein. 2009. Improved Algorithms for the K Simple Shortest Paths and the Replacement Paths Problems. Inform. Process. Lett. 109, 7 (2009), 352--355.
[31]
Mark Harris, Shubhabrata Sengupta, and John D Owens. 2007. Parallel prefix sum (scan) with CUDA. GPU gems 3, 39 (2007), 851--876.
[32]
Haoyu He, Yuede Ji, and H Howie Huang. 2022. Illuminati: Towards explaining graph neural networks for cybersecurity analysis. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). IEEE, 74--89.
[33]
Yukun He, Qiang Li, Jian Cao, Yuede Ji, and Dong Guo. 2017. Understanding socialbot behavior on end hosts. International Journal of Distributed Sensor Networks 13, 2 (2017), 1550147717694170.
[34]
John Hershberger, Matthew Maxel, and Subhash Suri. 2007. Finding the k shortest simple paths: A new algorithm and its implementation. ACM Transactions on Algorithms (TALG) 3, 4 (2007), 45--es.
[35]
Daniel S. Hirschberg. 1978. Fast parallel sorting algorithms. Commun. ACM 21, 8 (1978), 657--661.
[36]
Walter Hoffman and Richard Pavley. 1959. A method for the solution of the n th best path problem. Journal of the ACM (JACM) 6, 4 (1959), 506--514.
[37]
Yuede Ji, Lei Cui, and H. Howie Huang. 2021. BugGraph: Differentiating Source-Binary Code Similarity with Graph Triplet-Loss Network. In 16th ACM ASIA Conference on Computer and Communications Security (AsiaCCS).
[38]
Yuede Ji, Lei Cui, and H. Howie Huang. 2021. BugGraph: Differentiating Source-Binary Code Similarity with Graph Triplet-Loss Network. In 16th ACM ASIA Conference on Computer and Communications Security (ASIACCS).
[39]
Yuede Ji, Lei Cui, and H Howie Huang. 2021. Vestige: Identifying Binary Code Provenance for Vulnerability Detection. In International Conference on Applied Cryptography and Network Security (ACNS). Springer, 287--310.
[40]
Yuede Ji, Mohamed Elsabagh, Ryan Johnson, and Angelos Stavrou. 2021. DEFInit: An Analysis of Exposed Android Init Routines. In 30th USENIX Security Symposium (USENIX Security).
[41]
Yuede Ji, Yukun He, Xinyang Jiang, Jian Cao, and Qiang Li. 2016. Combating the evasion mechanisms of social bots. Computers & Security (2016).
[42]
Yuede Ji, Yukun He, Xinyang Jiang, and Qiang Li. 2014. Towards social botnet behavior detecting in the end host. In 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS). IEEE, 320--327.
[43]
Yuede Ji, Hang Liu, and H. Howie Huang. 2018. iSpan: Parallel Identification of Strongly Connected Components with Spanning Trees. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE, 731--742.
[44]
Yuede Ji, Hang Liu, and H. Howie Huang. 2020. SwarmGraph: Analyzing Large-Scale In-Memory Graphs on GPUs. In International Conference on High Performance Computing and Communications (HPCC). IEEE.
[45]
Masahiko Jinno, Bartlomiej Kozicki, Hidehiko Takara, Atsushi Watanabe, Yoshiaki Sone, Takafumi Tanaka, and Akira Hirano. 2010. Distance-adaptive spectrum resource allocation in spectrum-sliced elastic optical path network [topics in optical communications]. IEEE Communications Magazine 48, 8 (2010), 138--145.
[46]
Jérôme Kunegis. 2013. Konect: the koblenz network collection. In Proceedings of the 22nd international conference on world wide web. 1343--1350.
[47]
Denis Kurz and Petra Mutzel. 2016. A sidetrack-based algorithm for finding the k shortest simple paths in a directed graph. 27th International Symposium on Algorithms and Computation (2016).
[48]
Dominique LaSalle, Md Mostofa Ali Patwary, Nadathur Satish, Narayanan Sundaram, Pradeep Dubey, and George Karypis. 2015. Improving graph partitioning for modern graphs and architectures. In Proceedings of the 5th Workshop on Irregular Applications: Architectures and Algorithms. ACM, 14.
[49]
Eugene L Lawler. 1972. A Procedure for Computing the K Best Solutions to Discrete Optimization Problems and Its Application to the Shortest Path Problem. Management Science 18, 7 (1972), 401--405.
[50]
John Lhota and Lei Xie. 2016. Protein-fold recognition using an improved single-source K diverse shortest paths algorithm. Proteins: Structure, Function, and Bioinformatics 84, 4 (2016), 467--472.
[51]
Bi-Qing Li, Tao Huang, Lei Liu, Yu-Dong Cai, and Kuo-Chen Chou. 2012. Identification of colorectal cancer related genes with mRMR and shortest path in protein-protein interaction network. PloS one 7, 4 (2012), e33393.
[52]
Bo Liu, Gao Cong, Yifeng Zeng, Dong Xu, and Yeow Meng Chee. 2013. Influence spreading path and its application to the time constrained social influence maximization problem and beyond. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2013), 1904--1917.
[53]
Rui Liu, Raj Velamur Srinivasan, Kiyana Zolfaghar, Si-Chi Chin, Senjuti Basu Roy, Aftab Hasan, and David Hazel. 2014. Pathway-finder: An interactive recommender system for supporting personalized care pathways. In 2014 IEEE International Conference on Data Mining Workshop. IEEE, 1219--1222.
[54]
Tiantian Liu, Zijin Feng, Huan Li, Hua Lu, Muhammad Aamir Cheema, Hong Cheng, and Jianliang Xu. 2020. Shortest path queries for indoor venues with temporal variations. In 2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 2014--2017.
[55]
Nirmesh Malviya, Samuel Madden, and Arnab Bhattacharya. 2011. A continuous query system for dynamic route planning. In 2011 IEEE 27th International Conference on Data Engineering. IEEE, 792--803.
[56]
Ulrich Meyer and Peter Sanders. 2003. Δ-Stepping: A Parallel Single Source Shortest Path Algorithm. Journal of Algorithms 49, 1 (2003), 114--152.
[57]
Sara Nazari, M Reza Meybodi, M Ali Salehigh, and Sara Taghipour. 2008. An advanced algorithm for finding shortest path in car navigation system. In 2008 First International Conference on Intelligent Networks and Intelligent Systems. IEEE, 671--674.
[58]
Uyen TV Nguyen, Shanika Karunasekera, Lars Kulik, Egemen Tanin, Rui Zhang, Haolan Zhang, Hairuo Xie, and Kotagiri Ramamohanarao. 2015. A randomized path routing algorithm for decentralized route allocation in transportation networks. In Proceedings of the 8th ACM SIGSPATIAL International Workshop on Computational Transportation Science. 15--20.
[59]
Prashant Pandey, Brian Wheatman, Helen Xu, and Aydin Buluc. 2021. Terrace: A Hierarchical Graph Container for Skewed Dynamic Graphs. In Proceedings of the 2021 International Conference on Management of Data. 1372--1385.
[60]
Antonio Sedeño-Noda. 2012. An Efficient Time and Space K Point-to-Point Shortest Simple Paths Algorithm. Appl. Math. Comput. 218, 20 (2012), 10244--10257.
[61]
Robert Sedgewick. 2001. Algorithms in C, part 5: graph algorithms. Pearson Education.
[62]
Yu-Keng Shih and Srinivasan Parthasarathy. 2012. A single source k-shortest paths algorithm to infer regulatory pathways in a gene network. Bioinformatics 28, 12 (2012), i49--i58.
[63]
Avadhesh Pratap Singh and Dhirendra Pratap Singh. 2015. Implementation of K-shortest Path Algorithm in GPU Using CUDA. Procedia Computer Science 48 (2015), 5--13.
[64]
Edgar Solomonik and Laxmikant V Kale. 2010. Highly scalable parallel sorting. In 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS). IEEE, 1--12.
[65]
Mikkel Thorup. 1999. Undirected single-source shortest paths with positive integer weights in linear time. Journal of the ACM (JACM) 46, 3 (1999), 362--394.
[66]
Xin Wan, Lei Wang, Nan Hua, Hanyi Zhang, and Xiaoping Zheng. 2011. Dynamic routing and spectrum assignment in flexible optical path networks. In 2011 Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference. IEEE, 1--3.
[67]
Brian Wheatman and Helen Xu. 2018. Packed compressed sparse row: A dynamic graph representation. In 2018 IEEE High Performance extreme Computing Conference (HPEC). IEEE, 1--7.
[68]
Brian Wheatman and Helen Xu. 2021. A parallel packed memory array to store dynamic graphs. In 2021 Proceedings of the Workshop on Algorithm Engineering and Experiments (ALENEX). SIAM, 31--45.
[69]
Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck. 2014. Modeling and discovering vulnerabilities with code property graphs. In Security and Privacy (SP), 2014 IEEE Symposium on. IEEE, 590--604.
[70]
Jin Y Yen. 1971. Finding the K Shortest Loopless Paths in a Network. Management Science 17, 11 (1971), 712--716.
[71]
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 1234--1241.
[72]
Yunhui Zheng and Xiangyu Zhang. 2013. Path sensitive static analysis of web applications for remote code execution vulnerability detection. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 652--661.
Recommendations
Acceleration of shortest path and constrained shortest path computation
WEA'05: Proceedings of the 4th international conference on Experimental and Efficient AlgorithmsWe study acceleration methods for point-to-point shortest path and constrained shortest path computations in directed graphs, in particular in road and railroad networks. Our acceleration methods are allowed to use a preprocessing of the network data to ...
Generalization of Shortest Path Map
ITNG '10: Proceedings of the 2010 Seventh International Conference on Information Technology: New GenerationsWe consider the problem of constructing shortest path maps in two dimensions under angle constraint. Shortest path maps are used for planning short length paths from a fixed source point s to varying goal points. In the standard shortest path map the ...
A Multiple Pairs Shortest Path Algorithm
The multiple pairs shortest path problem (MPSP) arises in many applications where the shortest paths and distances between only some specific pairs of origin-destination (OD) nodes in a network are desired. The traditional repeated single-source shortest ...
Comments
Information & Contributors
Information
Published In
November 2023
1428 pages
ISBN:9798400701092
DOI:10.1145/3581784
- Chair:
- Dorian Arnold,
- Program Chair:
- Rosa M Badia,
- Program Co-chair:
- Kathryn Mohror
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 November 2023
Check for updates
Qualifiers
- Research-article
Conference
SC '23
Sponsor:
SC '23: International Conference for High Performance Computing, Networking, Storage and Analysis
November 12 - 17, 2023
CO, Denver, USA
Acceptance Rates
Overall Acceptance Rate 1,516 of 6,373 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 233Total Downloads
- Downloads (Last 12 months)233
- Downloads (Last 6 weeks)16
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
Cited By
View all- Udhayasekar SSrinivasan KKumar PChilukuri B(2024)Label-Setting Algorithm for Multi-Destination K Simple Shortest Paths Problem and ApplicationAlgorithms10.3390/a1708032517:8(325)Online publication date: 25-Jul-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in