research-article

Communication-avoiding parallel minimum cuts and connected components

Authors:

Lukas Gianinazzi,

Alessandro De Palma,

Torsten HoeflerAuthors Info & Claims

PPoPP '18: Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

Pages 219 - 232

https://doi.org/10.1145/3178487.3178504

Published: 10 February 2018 Publication History

Abstract

We present novel scalable parallel algorithms for finding global minimum cuts and connected components, which are important and fundamental problems in graph processing. To take advantage of future massively parallel architectures, our algorithms are communication-avoiding: they reduce the costs of communication across the network and the cache hierarchy. The fundamental technique underlying our work is the randomized sparsification of a graph: removing a fraction of graph edges, deriving a solution for such a sparsified graph, and using the result to obtain a solution for the original input. We design and implement sparsification with O(1) synchronization steps. Our global minimum cut algorithm decreases communication costs and computation compared to the state-of-the-art, while our connected components algorithm incurs few cache misses and synchronization steps. We validate our approach by evaluating MPI implementations of the algorithms on a petascale supercomputer. We also provide an approximate variant of the minimum cut algorithm and show that it approximates the exact solutions well while using a fraction of cores in a fraction of time.

Supplementary Material

Artifacts Available (comm-avoiding-cuts-cc-v0.1.4.zip)

Scripts, Source Files

Download
12.55 MB

References

[1]

James Abello, Adam L. Buchsbaum, and Jeffery Westbrook. 1998. A Functional Approach to External Graph Algorithms. In Proceedings of the 6th Annual European Symposium on Algorithms (ESA '98). Springer-Verlag, London, UK, UK, 332--343. http://dl.acm.org/citation.cfm?id=647908.740141

Digital Library

[2]

Micah Adler, Wolfgang Dittrich, Ben Juurlink, Miroslaw Kutylowski, and Ingo Rieping. 1998. Communication-optimal Parallel Minimum Spanning Tree Algorithms (Extended Abstract). In Proceedings of the Tenth Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA '98). ACM, New York, NY, USA, 27--36.

Digital Library

[3]

Réka Albert and Albert-László Barabási. 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74 (Jan 2002), 47--97. Issue 1.

[4]

Friedhelm Meyer auf der Heide and Gabriel T. Martinez. 1998. Communication-efficient parallel multiway and approximate minimum cut computation. In LATIN'98: Theoretical Informatics. Springer, 316--330.

Digital Library

[5]

Michael Brinkmeier. 2007. A Simple and Fast Min-Cut Algorithm. Theory Comput. Syst. 41, 2 (2007), 369--380.

Digital Library

[6]

Deepayan Chakrabarti, Yiping Zhan, and Christos Faloutsos. 2004. R-MAT: A Recursive Model for Graph Mining. In Proceedings of the Fourth SIAM International Conference on Data Mining, Lake Buena Vista, Florida, USA, April 22--24, 2004, Michael W. Berry, Umeshwar Dayal, Chandrika Kamath, and David B. Skillicorn (Eds.). SIAM, 442--446.

[7]

Ding-Kai Chen, Hong-Men Su, and Pen-Chung Yew. 1990. The Impact of Synchronization and Granularity on Parallel Systems. SIGARCH Comput. Archit. News 18, 2SI (May 1990), 239--248.

Digital Library

[8]

Frank K. H. A. Dehne, Afonso Ferreira, Edson Cáceres, Siang W. Song, and Alessandro Roncato. 2002. Efficient Parallel Graph Algorithms for Coarse-Grained Multicomputers and BSP. Algorithmica 33, 2 (2002), 183--200.

Digital Library

[9]

Paul Erdös and Alfréd Rényi. 1959. On random graphs, I. Publicationes Mathematicae (Debrecen) 6 (1959), 290--297.

[10]

L.R. Ford and Delbert R. Fulkerson. 1962. Flows in networks. Vol. 1962. Princeton Princeton University Press.

[11]

Matteo Frigo, Charles E. Leiserson, Harald Prokop, and Sridhar Ramachandran. 2012. Cache-Oblivious Algorithms. ACM Trans. Algorithms 8, 1 (2012), 4:1--4:22.

Digital Library

[12]

Hillel Gazit. 1986. An Optimal Randomized Parallel Algorithm for Finding Connected Components in a Graph. In 27th Annual Symposium on Foundations of Computer Science, Toronto, Canada, 27--29 October 1986. IEEE Computer Society, 492--501.

Digital Library

[13]

Barbara Geissmann and Lukas Gianinazzi. 2017. Cache Oblivious Minimum Cut. Springer International Publishing, Cham, 285--296.

[14]

Steve Goddard, Subodh Kumar, and Jan F. Prins. 1994. Connected components algorithms for mesh-connected parallel computers. In Parallel Algorithms, Proceedings of a DIMACS Workshop, Brunswick, New Jersey, USA, October 17--18, 1994 (DIMACS Series in Discrete Mathematics and Theoretical Computer Science), Sandeep Nautam Bhatt (Ed.), Vol. 30. DIMACS/AMS, 43--58. http://dimacs.rutgers.edu/Volumes/Vol30.html

[15]

Douglas Gregor and Andrew Lumsdaine. 2005. The parallel BGL: A generic library for distributed graph computations. Parallel Object-Oriented Scientific Computing (POOSC) 2 (2005), 1--18.

[16]

Shay Halperin and Uri Zwick. 1996. Optimal randomized EREW PRAM Algorithms for Finding Spanning Forests and for other Basic Graph Connectivity Problems. In Proceedings of the Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, 28--30 January 1996, Atlanta, Georgia. 438--447. http://dl.acm.org/citation.cfm?id-313852.314099

Digital Library

[17]

Jianxiu Hao and James B. Orlin. 1992. A faster algorithm for finding the minimum cut in a graph. In Proceedings of the third annual ACM-SIAM symposium on Discrete algorithms. Society for Industrial and Applied Mathematics, 165--174.

Digital Library

[18]

Torsten Hoefler and Roberto Belli. 2015. Scientific Benchmarking of Parallel Computing Systems. IEEE/ACM International Conference on High Performance Computing, Networking, Storage and Analysis (SC15).

Digital Library

[19]

Torsten Hoefler, William Gropp, Rajeev Thakur, and Jesper Larsson Träff. 2010. Toward Performance Models of MPI Implementations for Understanding Application Scaling Issues. In Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Stuttgart, Germany, September 12--15, 2010. Proceedings (Lecture Notes in Computer Science), Rainer Keller, Edgar Gabriel, Michael M. Resch, and Jack Dongarra (Eds.), Vol. 6305. Springer, 21--30.

Digital Library

[20]

Mark Hoemmen. 2010. Communication-avoiding Krylov Subspace Methods. Ph.D. Dissertation. Berkeley, CA, USA. Advisor(s) Demmel, James W. AAI3413388.

Digital Library

[21]

Keechul Jung, Kwang In Kim, and Anil K. Jain. 2004. Text information extraction in images and video: a survey. Pattern Recognition 37, 5 (2004), 977 -- 997.

[22]

David R. Karger. 2000. Minimum cuts in near-linear time. J. ACM 47, 1 (2000), 46--76.

Digital Library

[23]

David R. Karger. 2001. A randomized fully polynomial time approximation scheme for the all-terminal network reliability problem. SIAM review 43, 3 (2001), 499--522.

Digital Library

[24]

David R. Karger, Philip N. Klein, and Robert Endre Tarjan. 1995. A Randomized Linear-Time Algorithm to Find Minimum Spanning Trees. J. ACM 42, 2 (1995), 321--328.

Digital Library

[25]

David R. Karger and Clifford Stein. 1996. A new approach to the minimum cut problem. Journal of the ACM (JACM) 43, 4 (1996), 601--640.

Digital Library

[26]

John Kim, Wiliam J Dally, Steve Scott, and Dennis Abts. 2008. Technology-driven, highly-scalable dragonfly topology. In ACM SIGARCH Computer Architecture News, Vol. 36. IEEE Computer Society, 77--88.

Digital Library

[27]

Jean-Yves Le Boudec. 2010. Performance evaluation of computer and communication systems. EPFL Press.

[28]

Andrew Lumsdaine, Douglas P. Gregor, Bruce Hendrickson, and Jonathan W. Berry. 2007. Challenges in Parallel Graph Processing. Parallel Processing Letters 17, 1 (2007), 5--20.

[29]

Hiroshi Nagamochi and Toshihide Ibaraki. 1992. Computing edge-connectivity in multigraphs and capacitated graphs. SIAM Journal on Discrete Mathematics 5, 1 (1992), 54--66.

Digital Library

[30]

Donald Nguyen, Andrew Lenharth, and Keshav Pingali. 2013. A light-weight infrastructure for graph analytics. In ACM SIGOPS 24th Symposium on Operating Systems Principles, SOSP '13, Farmington, PA, USA, November 3--6, 2013, Michael Kaminsky and Mike Dahlin (Eds.). ACM, 456--471.

Digital Library

[31]

Simone F. Oliveira, Karl Fürlinger, and Dieter Kranzlmüller. 2012. Trends in Computation, Communication and Storage and the Consequences for Data-intensive Science. In 14th IEEE International Conference on High Performance Computing and Communication & 9th IEEE International Conference on Embedded Software and Systems, HPCC-ICESS 2012. 572--579.

Digital Library

[32]

Víctor Osma-Ruiz, Juan I. Godino-Llorente, Nicolás Sáenz-Lechón, and Pedro Gómez-Vilda. 2007. An Improved Watershed Algorithm Based on Efficient Computation of Shortest Paths. Pattern Recogn. 40, 3 (March 2007), 1078--1090.

Digital Library

[33]

David A. Patterson. 2004. Latency lags bandwith. Commun. ACM 47, 10 (2004), 71--75.

Digital Library

[34]

Jelena Pjesivac-Grbovic, Thara Angskun, George Bosilca, Graham E. Fagg, Edgar Gabriel, and Jack Dongarra. 2007. Performance analysis of MPI collective operations. Cluster Computing 10, 2 (2007), 127--143.

Digital Library

[35]

Michael Oser Rabin. 1976. Probablistic Algorithms. In Algorithms and Complexity, Joseph F. Traub (Ed.). Academic Press, 21--36.

[36]

John H. Reif. 1993. Synthesis of Parallel Algorithms (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[37]

John H. Reif and Sandeep Sen. 1989. Polling: A New Randomized Sampling Technique for Computational Geometry. In Proceedings of the 21st Annual ACM Symposium on Theory of Computing, May 14--17, 1989, Seattle, Washigton, USA. 394--404.

Digital Library

[38]

John K. Salmon, Mark A. Moraes, Ron O. Dror, and David E. Shaw. 2011. Parallel Random Numbers: As Easy As 1, 2, 3. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC '11). ACM, New York, NY, USA, Article 16, 12 pages.

Digital Library

[39]

Satu Elisa Schaeffer. 2007. Survey: Graph Clustering. Comput. Sci. Rev. 1, 1 (Aug. 2007), 27--64.

Digital Library

[40]

Roded Sharan and Ron Shamir. 2000. CLICK: a clustering algorithm with applications to gene expression analysis. In Proc Int Conf Intell Syst Mol Biol, Vol. 8. 16.

Digital Library

[41]

Jeremy G. Siek, Lie-Quan Lee, and Andrew Lumsdaine. 2001. Boost Graph Library: User Guide and Reference Manual, The. Pearson Education.

[42]

Francesco Silvestri. 2007. On the Limits of Cache-oblivious Matrix Transposition. In Proceedings of the 2Nd International Conference on Trustworthy Global Computing (TGC'06). Springer-Verlag, Berlin, Heidelberg, 233--243. http://dl.acm.org/citation.cfm?id = 1776656.1776677

Digital Library

[43]

Marc Snir. 1998. MPI-the Complete Reference: The MPI core. Vol. 1. MIT press.

Digital Library

[44]

Robert Solovay and Volker Strassen. 1977. A Fast Monte-Carlo Test for Primality. SIAM J. Comput. 6, 1 (1977), 84--85.

Digital Library

[45]

Mechthild Stoer and Frank Wagner 1997. A simple min-cut algorithm. J. ACM 44, 4 (1997), 585--591.

Digital Library

[46]

Jayaram K. Udupa and Venkatramana G. Ajjanagadde. 1990. Boundary and object labelling in three-dimensional images. Computer Vision, Graphics, and Image Processing 51, 3 (1990), 355--369.

Digital Library

[47]

Leslie G. Valiant. 1990. A bridging model for parallel computation. Commun. ACM 33, 8 (1990), 103--111.

Digital Library

[48]

Duncan J. Watts and Steven H Strogatz. 1998. Collective Dynamics of small-world networks. Nature 393 (1998).

[49]

Andrew D. Wilson. 2006. Robust Computer Vision-based Detection of Pinching for One and Two-handed Gesture Input. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST '06). ACM, New York, NY, USA, 255--258.

Digital Library

[50]

Yao Zhu and David F. Gleich. 2016. A parallel min-cut algorithm using iteratively reweighted least squares targeting at problems with floating-point edge weights. Parallel Comput. 59 (2016), 43--59.

Digital Library

Cited By

Besta MGrob RMiglioli CBernold NKwasniewski GGjini GKanakagiri RAshkboos SGianinazzi LDryden NHoefler TZhang ARangwala H(2022)Motif Prediction with Graph Neural NetworksProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539343(35-45)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539343
Besta MCarigiet AJanda KVonarburg-Shmaria ZGianinazzi LHoefler T(2020)High-Performance Parallel Graph Coloring with Strong Guarantees on Work, Depth, and QualitySC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00103(1-17)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00103
Henzinger MNoe ASchulz C(2019)Shared-Memory Exact Minimum Cuts2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2019.00013(13-22)Online publication date: May-2019
https://doi.org/10.1109/IPDPS.2019.00013
Show More Cited By

Index Terms

Communication-avoiding parallel minimum cuts and connected components
1. Theory of computation
  1. Design and analysis of algorithms
    1. Distributed algorithms

Recommendations

Communication-avoiding parallel minimum cuts and connected components
PPoPP '18

We present novel scalable parallel algorithms for finding global minimum cuts and connected components, which are important and fundamental problems in graph processing. To take advantage of future massively parallel architectures, our algorithms are ...
Minimum cuts, girth and a spectral threshold

Let G = (V, E) be a simple, undirected, unweighted, connected graph. A cut (A, A) defined by a subset A of V is called trivial if either A or A is a singleton set. Let µ be the second smallest eigenvalue of the Laplacian matrix of G. The length of the ...
Cuts leaving components of given minimum order

For a connected graph G, the restricted edge-connectivity @l"p(G) is defined as the minimum cardinality of an edge-cut over all edge-cuts S such that each component of G-S contains at least p vertices. In the present paper we introduce the more general ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PPoPP '18: Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

February 2018

442 pages

ISBN:9781450349826

DOI:10.1145/3178487

General Chair:
Andreas Krall
Vienna University of Technology, Austria
,
Program Chair:
Thomas R. Gross
ETH Zürich, Switzerland

ACM SIGPLAN Notices Volume 53, Issue 1
PPoPP '18
January 2018
426 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3200691
Editor:
Matthew Fluet
Rodchester Institude of Technology
Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Available

Author Tags

Qualifiers

Research-article

Conference

PPoPP '18

Sponsor:

PPoPP '18: 23nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

February 24 - 28, 2018

Vienna, Austria

Acceptance Rates

Overall Acceptance Rate 230 of 1,014 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
422
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Besta MGrob RMiglioli CBernold NKwasniewski GGjini GKanakagiri RAshkboos SGianinazzi LDryden NHoefler TZhang ARangwala H(2022)Motif Prediction with Graph Neural NetworksProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539343(35-45)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539343
Besta MCarigiet AJanda KVonarburg-Shmaria ZGianinazzi LHoefler T(2020)High-Performance Parallel Graph Coloring with Strong Guarantees on Work, Depth, and QualitySC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00103(1-17)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00103
Henzinger MNoe ASchulz C(2019)Shared-Memory Exact Minimum Cuts2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS.2019.00013(13-22)Online publication date: May-2019
https://doi.org/10.1109/IPDPS.2019.00013
Henzinger MNoe ASchulz CStrash D(2018)Practical Minimum Cut AlgorithmsACM Journal of Experimental Algorithmics10.1145/327466223(1-22)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3274662
Besta MStanojevic DZivic TSingh JHoerold MHoefler TEvripidou SStenström PO'Boyle M(2018)Log(graph)Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques10.1145/3243176.3243198(1-13)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3243176.3243198
Besta MHoefler T(2024)Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency AnalysisIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3303431(1-20)Online publication date: 2024
https://doi.org/10.1109/TPAMI.2023.3303431
Besta MGerstenberger RPeter EFischer MPodstawski MBarthels CAlonso GHoefler T(2023)Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph QueriesACM Computing Surveys10.1145/360493256:2(1-40)Online publication date: 15-Sep-2023
https://dl.acm.org/doi/10.1145/3604932
Lakhotia KBesta MMonroe LIsham KIff PHoefler TPetrini FWolf FShende SCulhane CAlam SJagode H(2022)PolarFlyProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571900(1-15)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571900
Lakhotia KBesta MMonroe LIsham KIff PHoefler TPetrini F(2022)PolarFly: A Cost-Effective and Flexible Low-Diameter TopologySC22: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41404.2022.00017(1-15)Online publication date: Nov-2022
https://doi.org/10.1109/SC41404.2022.00017
Besta MKanakagiri RKwasniewski GAusavarungnirun RBeránek JKanellopoulos KJanda KVonarburg-Shmaria ZGianinazzi LStefan ILuna JGolinowski JCopik MKapp-Schwoerer LDi Girolamo SBlach NKonieczny MMutlu OHoefler T(2021)SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory SystemsMICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3466752.3480133(282-297)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3466752.3480133
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents