Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3627535.3638508acmconferencesArticle/Chapter ViewAbstractPublication PagesppoppConference Proceedingsconference-collections
research-article
Open access

Parallel k-Core Decomposition with Batched Updates and Asynchronous Reads

Published: 20 February 2024 Publication History

Abstract

Maintaining a dynamic k-core decomposition is an important problem that identifies dense subgraphs in dynamically changing graphs. Recent work by Liu et al. [SPAA 2022] presents a parallel batch-dynamic algorithm for maintaining an approximate k-core decomposition. In their solution, both reads and updates need to be batched, and therefore each type of operation can incur high latency waiting for the other type to finish. To tackle most real-world workloads, which are dominated by reads, this paper presents a novel hybrid concurrent-parallel dynamic k-core data structure where asynchronous reads can proceed concurrently with batches of updates, leading to significantly lower read latencies. Our approach is based on tracking causal dependencies between updates, so that causally related groups of updates appear atomic to concurrent readers. Our data structure guarantees linearizability and liveness for both reads and updates, and maintains the same approximation guarantees as prior work. Our experimental evaluation on a 30-core machine shows that our approach reduces read latency by orders of magnitude compared to the batch-dynamic algorithm, up to a (4.05 · 105)-factor. Compared to an unsynchronized (non-linearizable) baseline, our read latency overhead is only up to a 3.21-factor greater, while improving accuracy of coreness estimates by up to a factor of 52.7.

References

[1]
Umut A. Acar, Daniel Anderson, Guy E. Blelloch, and Laxman Dhulipala. 2019. Parallel Batch-Dynamic Graph Connectivity. In The 31st ACM Symposium on Parallelism in Algorithms and Architectures. 381--392.
[2]
Umut A. Acar, Daniel Anderson, Guy E. Blelloch, Laxman Dhulipala, and Sam Westrick. 2020. Parallel Batch-Dynamic Trees via Change Propagation. In Annual European Symposium on Algorithms, Vol. 173. 2:1--2:23.
[3]
Yehuda Afek, Hagit Attiya, Danny Dolev, Eli Gafni, Michael Merritt, and Nir Shavit. 1993. Atomic Snapshots of Shared Memory. Journal of the ACM (JACM) 40, 4 (1993), 873--890.
[4]
Kunal Agrawal, Jeremy T. Fineman, Kefu Lu, Brendan Sheridan, Jim Sukha, and Robert Utterback. 2014. Provably Good Scheduling for Parallel Programs That Use Data Structures through Implicit Batching. In Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures. 84--95.
[5]
Vitaly Aksenov, Petr Kuznetsov, and Anatoly Shalyto. 2018. Parallel Combining: Benefits of Explicit Synchronization. In International Conference on Principles of Distributed Systems (OPODIS), Vol. 125. 11:1--11:16.
[6]
Dan Alistarh, Alexander Fedorov, and Nikita Koval. 2019. In Search of the Fastest Concurrent Union-Find Algorithm. In 23rd International Conference on Principles of Distributed Systems, Vol. 153. 15:1--15:16.
[7]
Md. Altaf-Ul-Amin, Yoko Shinbo, Kenji Mihara, Ken Kurokawa, and Shigehiko Kanaya. 2006. Development and implementation of an algorithm for detection of protein complexes in large interaction networks. BMC Bioinform. 7 (2006), 207.
[8]
J. Ignacio Alvarez-Hamelin, Luca Dall'Asta, Alain Barrat, and Alessandro Vespignani. 2005. Large scale networks fingerprinting and visualization using the k-core decomposition. In Advances in Neural Information Processing Systems. 41--50.
[9]
Daniel Anderson and Guy E. Blelloch. 2023. Deterministic and Work-Efficient Parallel Batch-Dynamic Trees in Low Span. CoRR abs/2306.08786 (2023), 20 pages.
[10]
Daniel Anderson, Guy E. Blelloch, Anubhav Baweja, and Umut A. Acar. 2021. Efficient Parallel Self-Adjusting Computation. In 33rd ACM Symposium on Parallelism in Algorithms and Architectures. 59--70.
[11]
Richard Anderson and Ernst W Mayr. 1984. A P-complete problem and approximations to it. Technical Report. Stanford University.
[12]
Sabeur Aridhi, Martin Brugnara, Alberto Montresor, and Yannis Velegrakis. 2016. Distributed k-core decomposition and maintenance in large dynamic graphs. In ACM International Conference on Distributed and Event-based Systems (DEBS). 161--168.
[13]
Gary D. Bader and Christopher W. V. Hogue. 2003. An automated method for finding molecular complexes in large protein interaction networks. BMC Bioinform. 4 (2003), 2.
[14]
Greg Barnes. 1993. A Method for Implementing Lock-Free Shared-Data Structures. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 261--270.
[15]
Sayan Bhattacharya, Monika Henzinger, Danupon Nanongkai, and Charalampos Tsourakakis. 2015. Space- and Time-Efficient Algorithm for Maintaining Dense Subgraphs on One-Pass Dynamic Streams. In ACM Symposium on Theory of Computing (STOC). 173--182.
[16]
Guy E. Blelloch, Daniel Anderson, and Laxman Dhulipala. 2020. Brief Announcement: ParlayLib - A Toolkit for Parallel Algorithms on Shared-Memory Multicore Machines. In ACM Symp. on Parallel Alg. (SPAA). 507--509.
[17]
Francesco Bonchi, Francesco Gullo, Andreas Kaltenbrunner, and Yana Volkovich. 2014. Core decomposition of uncertain graphs. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 1316--1325.
[18]
Nathan Bronson, Zach Amsden, George Cabrera, Prasad Chakka, Peter Dimov, Hui Ding, Jack Ferris, Anthony Giardullo, Sachin Kulkarni, Harry C. Li, Mark Marchukov, Dmitri Petrov, Lovro Puzar, Yee Jiun Song, and Venkateshwaran Venkataramani. 2013. TAO: Facebook's Distributed Data Store for the Social Graph. In USENIX Annual Technical Conference (ATC), Andrew Birrell and Emin Gün Sirer (Eds.). 49--60. https://www.usenix.org/conference/atc13/technical-sessions/presentation/bronson
[19]
Shai Carmi, Shlomo Havlin, Scott Kirkpatrick, Yuval Shavitt, and Eran Shir. 2007. A model of Internet topology using k-shell decomposition. Proceedings of the National Academy of Sciences 104, 27 (2007), 11150--11154.
[20]
Audrey Cheng, Xiao Shi, Aaron N. Kabcenell, Shilpa Lawande, Hamza Qadeer, Jason Chan, Harrison Tin, Ryan Zhao, Peter Bailis, Mahesh Balakrishnan, Nathan Bronson, Natacha Crooks, and Ion Stoica. 2022. TAOBench: An End-to-End Benchmark for Social Networking Workloads. Proceedings of the VLDB Endowment 15, 9 (2022), 1965--1977.
[21]
Deming Chu, Fan Zhang, Xuemin Lin, Wenjie Zhang, Ying Zhang, Yinglong Xia, and Chenyi Zhang. 2020. Finding the Best k in Core Decomposition: A Time and Space Optimal Solution. In IEEE International Conference on Data Engineering (ICDE). 685--696.
[22]
Martino Ciaperoni, Edoardo Galimberti, Francesco Bonchi, Ciro Cattuto, Francesco Gullo, and Alain Barrat. 2020. Relevance of temporal cores for epidemic spread in temporal networks. Scientific Reports 10, 1 (2020), 12529.
[23]
Nachshon Cohen, Rachid Guerraoui, and Igor Zablotchi. 2018. The Inherent Cost of Remembering Consistently. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 259--269.
[24]
Laxman Dhulipala, Guy E. Blelloch, Yan Gu, and Yihan Sun. 2022. PaC-trees: supporting parallel and compressed purely-functional collections. In 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation. 108--121.
[25]
Laxman Dhulipala, Guy E. Blelloch, and Julian Shun. 2017. Julienne: A Framework for Parallel Graph Algorithms using Work-efficient Bucketing. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 293--304.
[26]
Laxman Dhulipala, Guy E. Blelloch, and Julian Shun. 2018. Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 393--404.
[27]
Laxman Dhulipala, Guy E. Blelloch, and Julian Shun. 2019. Low-latency graph streaming using compressed purely-functional trees. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 918--934.
[28]
Laxman Dhulipala, Changwan Hong, and Julian Shun. 2020. ConnectIt: A Framework for Static and Incremental Parallel Graph Connectivity Algorithms. Proc. VLDB Endow. 14, 4 (Dec 2020), 653--667.
[29]
Laxman Dhulipala, Quanquan C. Liu, Julian Shun, and Shangdi Yu. 2021. Parallel Batch-Dynamic k-Clique Counting. In Symposium on Algorithmic Principles of Computer Systems (APOCS). 129--143.
[30]
Yon Dourisboure, Filippo Geraci, and Marco Pellegrini. 2009. Extraction and classification of dense implicit communities in the Web graph. ACM Trans. Web 3, 2 (2009), 7:1--7:36.
[31]
Faith Ellen, Panagiota Fatourou, Eric Ruppert, and Franck van Breugel. 2010. Non-blocking binary search trees. In ACM Symposium on Principles of Distributed Computing (PODC). 131--140.
[32]
Fatemeh Esfahani, Venkatesh Srinivasan, Alex Thomo, and Kui Wu. 2019. Efficient Computation of Probabilistic Core Decomposition at Web-Scale. In International Conference on Extending Database Technology (EDBT). 325--336.
[33]
Hossein Esfandiari, Silvio Lattanzi, and Vahab S. Mirrokni. 2018. Parallel and Streaming Algorithms for K-Core Decomposition. In International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 80). 1396--1405.
[34]
Yixiang Fang, Reynold Cheng, Xiaodong Li, Siqiang Luo, and Jiafeng Hu. 2017. Effective Community Search over Large Spatial Graphs. Proceedings of the VLDB Endowment 10, 6 (2017), 709--720.
[35]
Alexander Fedorov, Nikita Koval, and Dan Alistarh. 2021. A Scalable Concurrent Algorithm for Dynamic Connectivity. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 208--220.
[36]
Paolo Ferragina and Fabrizio Luccio. 1994. Batch Dynamic Algorithms for Two Graph Problems. In International PARLE Conference on Parallel Architectures and Languages Europe, Vol. 817. 713--724.
[37]
Keir Fraser. 2004. Practical lock-freedom. Ph. D. Dissertation. University of Cambridge, UK.
[38]
Kasimir Gabert, Ali Pinar, and Ümit V. Çatalyürek. 2021. Shared-Memory Scalable k-Core Maintenance on Dynamic Graphs and Hypergraphs. In IEEE International Parallel and Distributed Processing Symposium (IPDPS) Workshops. 998--1007.
[39]
Edoardo Galimberti, Francesco Bonchi, Francesco Gullo, and Tommaso Lanciano. 2020. Core Decomposition in Multilayer Networks: Theory, Algorithms, and Applications. ACM Trans. Knowl. Discov. Data 14, 1 (2020), 11:1--11:40.
[40]
Mohsen Ghaffari, Silvio Lattanzi, and Slobodan Mitrovic. 2019. Improved Parallel Algorithms for Density-Based Network Clustering. In International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 97). 2201--2210.
[41]
Christos Giatsidis, Fragkiskos D. Malliaros, Dimitrios M. Thilikos, and Michalis Vazirgiannis. 2014. CoreCluster: A Degeneracy Based Graph Clustering Framework. In AAAI Conference on Artificial Intelligence. 44--50.
[42]
John Healy, Jeannette C. M. Janssen, Evangelos E. Milios, and William Aiello. 2006. Characterization of Graphs Using Degree Cores. In International Workshop on Algorithms and Models for the Web-Graph (WAW), Vol. 4936. 137--148.
[43]
Monika Henzinger, Stefan Neumann, and Andreas Wiese. 2020. Explicit and Implicit Dynamic Coloring of Graphs with Bounded Arboricity. CoRR abs/2002.10142 (2020), 18 pages.
[44]
Maurice Herlihy and Nir Shavit. 2012. The Art of Multiprocessor Programming, Revised Reprint (1st ed.). Morgan Kaufmann Publishers Inc.
[45]
Changwan Hong, Laxman Dhulipala, and Julian Shun. 2020. Exploring the Design Space of Static and Incremental Graph Connectivity Algorithms on GPUs. In Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques. 55--69.
[46]
Qiang-Sheng Hua, Yuliang Shi, Dongxiao Yu, Hai Jin, Jiguo Yu, Zhipeng Cai, Xiuzhen Cheng, and Hanhua Chen. 2020. Faster Parallel Core Maintenance Algorithms in Dynamic Graphs. IEEE Transactions on Parallel and Distributed Systems 31, 6 (2020), 1287--1300.
[47]
Siddhartha V. Jayanti and Robert E. Tarjan. 2021. Concurrent disjoint set union. Distributed Computing 34, 6 (2021), 413--436.
[48]
Hai Jin, Na Wang, Dongxiao Yu, Qiang-Sheng Hua, Xuanhua Shi, and Xia Xie. 2018. Core Maintenance in Dynamic Graphs: A Parallel Approach Based on Matching. IEEE Transactions on Parallel and Distributed Systems 29, 11 (2018), 2416--2428.
[49]
Humayun Kabir and Kamesh Madduri. 2017. Parallel k-Core Decomposition on Multicore Platforms. In IEEE International Parallel and Distributed Processing Symposium Workshops, (IPDPS). 1482--1491.
[50]
Wissam Khaouid, Marina Barsky, S. Venkatesh, and Alex Thomo. 2015. K-Core Decomposition of Large Networks on a Single PC. Proceedings of the VLDB Endowment 9, 1 (2015), 13--23.
[51]
Maksim Kitsak, Lazaros K Gallos, Shlomo Havlin, Fredrik Liljeros, Lev Muchnik, H Eugene Stanley, and Hernán A Makse. 2010. Identification of influential spreaders in complex networks. Nature Physics 6, 11 (2010), 888--893.
[52]
Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon. 2010. What is Twitter, a Social Network or a News Media?. In International Conference on World Wide Web. 591--600.
[53]
Victor E. Lee, Ning Ruan, Ruoming Jin, and Charu C. Aggarwal. 2010. A Survey of Algorithms for Dense Subgraph Discovery. In Managing and Mining Graph Data. Advances in Database Systems, Vol. 40. 303--336.
[54]
Conggai Li, Fan Zhang, Ying Zhang, Lu Qin, Wenjie Zhang, and Xuemin Lin. 2019. Efficient Progressive Minimum k-core Search. Proceedings of the VLDB Endowment 13, 3 (2019), 362--375.
[55]
Rong-Hua Li, Jeffrey Xu Yu, and Rui Mao. 2014. Efficient Core Maintenance in Large Dynamic Graphs. IEEE Trans. Knowl. Data Eng. 26, 10 (2014), 2453--2465.
[56]
Zhe Lin, Fan Zhang, Xuemin Lin, Wenjie Zhang, and Zhihong Tian. 2021. Hierarchical Core Maintenance on Large Dynamic Graphs. Proceedings of the VLDB Endowment 14, 5 (2021), 757--770.
[57]
Quanquan C. Liu, Jessica Shi, Shangdi Yu, Laxman Dhulipala, and Julian Shun. 2022. Parallel Batch-Dynamic Algorithms for k-Core Decomposition and Related Graph Problems. In ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 191--204.
[58]
Quanquan C. Liu, Julian Shun, and Igor Zablotchi. 2024. Parallel k-Core Decomposition with Batched Updates and Asynchronous Reads. arXiv:2401.08015 [cs.DC]
[59]
Ying Liu, Ming Tang, Tao Zhou, and Younghae Do. 2015. Core-like groups result in invalidation of identifying super-spreader by k-shell decomposition. Scientific Reports 5, 1 (2015), 9602.
[60]
Qi Luo, Dongxiao Yu, Feng Li, Zhenhao Dou, Zhipeng Cai, Jiguo Yu, and Xiuzhen Cheng. 2019. Distributed Core Decomposition in Probabilistic Graphs. In International Conference on Computational Data and Social Networks (CSoNet), Vol. 11917. 16--32.
[61]
Fragkiskos D. Malliaros, Maria-Evgenia G. Rossi, and Michalis Vazirgiannis. 2016. Locating influential nodes in complex networks. Scientific Reports 6, 1 (2016), 19307.
[62]
David W. Matula and Leland L. Beck. 1983. Smallest-Last Ordering and clustering and Graph Coloring Algorithms. J. ACM 30, 3 (1983), 417--427.
[63]
Sourav Medya, Tianyi Ma, Arlei Silva, and Ambuj K. Singh. 2020. A Game Theoretic Approach For k-Core Minimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 1922--1924.
[64]
Michael Mitzenmacher, Jakub Pachocki, Richard Peng, Charalampos E. Tsourakakis, and Shen Chen Xu. 2015. Scalable Large Near-Clique Detection in Large-Scale Networks via Sampling. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 815--824.
[65]
Eisha Nathan, E. Jason Riedy, Anita Zakrzewska, and Chunxing Yin. 2017. A New Direction for Streaming Graph Analysis. In IEEE International Conference on Cluster Computing (CLUSTER). 645--646.
[66]
Shaunak Pawagi and Owen Kaser. 1993. Optimal parallel algorithms for multiple updates of minimum spanning trees. Algorithmica 9, 4 (1993), 357--381.
[67]
Ahmet Erdem Sariyüce, Bugra Gedik, Gabriela Jacques-Silva, Kun-Lung Wu, and Ümit V. Çatalyürek. 2013. Streaming Algorithms for k-core Decomposition. Proceedings of the VLDB Endowment 6, 6 (2013), 433--444.
[68]
Ahmet Erdem Sariyüce, Bugra Gedik, Gabriela Jacques-Silva, Kun-Lung Wu, and Ümit V. Çatalyürek. 2016. Incremental k-core decomposition: algorithms and evaluation. Proceedings of the VLDB Endowment 25, 3 (2016), 425--447.
[69]
X. Shen and W. Liang. 1993. A parallel algorithm for multiple edge updates of minimum spanning trees. In Proceedings Seventh International Parallel Processing Symposium. 310--317.
[70]
Bintao Sun, T.-H. Hubert Chan, and Mauro Sozio. 2020. Fully Dynamic Approximate k-Core Decomposition in Hypergraphs. ACM Trans. Knowl. Discov. Data 14, 4 (2020), 39:1--39:21.
[71]
Tom Tseng, Laxman Dhulipala, and Julian Shun. 2022. Parallel Batch-Dynamic Minimum Spanning Forest and the Efficiency of Dynamic Agglomerative Graph Clustering. In Proceedings of the 34th ACM Symposium on Parallelism in Algorithms and Architectures. 233--245.
[72]
Kai Wang, Xin Cao, Xuemin Lin, Wenjie Zhang, and Lu Qin. 2018. Efficient Computing of Radius-Bounded k-Cores. In IEEE International Conference on Data Engineering (ICDE). 233--244.
[73]
Na Wang, Dongxiao Yu, Hai Jin, Chen Qian, Xia Xie, and Qiang-Sheng Hua. 2017. Parallel Algorithm for Core Maintenance in Dynamic Graphs. In IEEE International Conference on Distributed Computing Systems (ICDCS). 2366--2371.
[74]
Dong Wen, Lu Qin, Ying Zhang, Xuemin Lin, and Jeffrey Xu Yu. 2019. I/O Efficient Core Graph Decomposition: Application to Degeneracy Ordering. IEEE Trans. Knowl. Data Eng. 31, 1 (2019), 75--90.
[75]
Jaewon Yang and Jure Leskovec. 2015. Defining and evaluating network communities based on ground-truth. Knowl. Inf. Syst. 42, 1 (2015), 181--213.
[76]
Fan Zhang, Ying Zhang, Lu Qin, Wenjie Zhang, and Xuemin Lin. 2017. When Engagement Meets Similarity: Efficient (k, r)-Core Computation on Social Networks. Proceedings of the VLDB Endowment 10, 10 (2017), 998--1009.
[77]
Haohua Zhang, Hai Zhao, Wei Cai, Jie Liu, and Wanlei Zhou. 2010. Using the k-core decomposition to analyze the static structure of large-scale software systems. J. Supercomput. 53, 2 (2010), 352--369.
[78]
Yikai Zhang, Jeffrey Xu Yu, Ying Zhang, and Lu Qin. 2017. A Fast Order-Based Approach for Core Maintenance. In IEEE International Conference on Data Engineering (ICDE). 337--348.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PPoPP '24: Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming
March 2024
498 pages
ISBN:9798400704352
DOI:10.1145/3627535
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2024

Check for updates

Badges

Author Tags

  1. parallelism
  2. concurrency
  3. k-core decomposition

Qualifiers

  • Research-article

Funding Sources

Conference

PPoPP '24

Acceptance Rates

Overall Acceptance Rate 230 of 1,014 submissions, 23%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 487
    Total Downloads
  • Downloads (Last 12 months)487
  • Downloads (Last 6 weeks)56
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media