research-article

(Semi-)external algorithms for graph partitioning and clustering

Authors:

Yaroslav Akhremtsev,

Christian SchulzAuthors Info & Claims

ALENEX '15: Proceedings of the Meeting on Algorithm Engineering & Expermiments

Pages 33 - 43

Published: 05 January 2015 Publication History

Abstract

In this paper, we develop semi-external and external memory algorithms for graph partitioning and clustering problems. Graph partitioning and clustering are key tools for processing and analyzing large complex networks. We address both problems in the (semi-)external model by adapting the size-constrained label propagation technique. Our (semi-)external size-constrained label propagation algorithm can be used to compute graph clusterings and is a prerequisite for the (semi-)external graph partitioning algorithm. The algorithm is then used for both the coarsening and the refinement phase of a multilevel algorithm to compute graph partitions. Our algorithm is able to partition and cluster huge complex networks with billions of edges on cheap commodity machines. Experiments demonstrate that the semi-external graph partitioning algorithm is scalable and can compute high quality partitions in time that is comparable to the running time of an efficient internal memory implementation. A parallelization of the algorithm in the semi-external model further reduces running time.

References

[1]

Kyrola, A., Blelloch, G., Guestrin, C.: GraphChi: Large-scale Graph Computation on Just a PC. In: Proc. of the 10th USENIX Symp. on Operating Systems Design and Implementation (OSDI). Volume 8. (2012) 31--46

[2]

Garey, M. R., Johnson, D. S., Stockmeyer, L.: Some Simplified NP-complete Problems. (1974) 47--63

[3]

Bui, T. N., Jones, C.: Finding Good Approximate Vertex and Edge Partitions is NP-Hard. Information Processing Letters 42(3) (1992) 153--159

[4]

Abello, J., Buchsbaum, A. L., Westbrook, J.: A Functional Approach to External Graph Algorithms. Algorithmica 32(3) (2002) 437--458

[5]

Bichot, C., Siarry, P., eds.: Graph Partitioning. Wiley (2011)

[6]

Buluç, A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C.: Recent Advances in Graph Partitioning. Technical Report ArXiv:1311.3144 (2013)

[7]

Diekmann, R., Preis, R., Schlimbach, F., Walshaw, C.: Shape-optimized Mesh Partitioning and Load Balancing for Parallel Adaptive FEM. Parallel Computing 26(12) (2000) 1555--1581

[8]

Abou-Rjeili, A., Karypis, G.: Multilevel Algorithms for Partitioning Power-Law Graphs. In: Proc. of 20th Int. Parallel and Distributed Processing Symp. (2006)

[9]

Meyerhenke, H., Monien, B., Schamberger, S.: Accelerating Shape Optimizing Load Balancing for Parallel FEM Simulations by Algebraic Multigrid. In: Proc. of 20th Int. Parallel and Distributed Processing Symp. (2006)

[10]

Hendrickson, B.: Chaco: Software for Partitioning Graphs. http://www.cs.sandia.gov/~bahendr/chaco.html

[11]

Walshaw, C., Cross, M.: Mesh Partitioning: A Multilevel Balancing and Refinement Algorithm. SIAM J. on Scientific Computing 22(1) (2000) 63--80

[12]

Karypis, G., Kumar, V.: A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs. SIAM J. on Scientific Computing 20(1) (1998) 359--392

[13]

Monien, B., Schamberger, S.: Graph Partitioning with the Party Library: Helpful-Sets in Practice (2004)

[14]

Pellegrini, F.: Scotch Home Page. http://www.labri.fr/pelegrin/scotch

[15]

Raghavan, U. N., Albert, R., Kumara, S.: Near Linear Time Algorithm to Detect Community Structures in Large-Scale Networks. Physical Review E 76(3) (2007)

[16]

Meyerhenke, H., Sanders, P., Schulz, C.: Partitioning Complex Networks via Size-constrained Clustering. In: Proc. of the 13th Int. Symp. on Experimental Algorithms (SEA'14). LNCS, Springer (2014)

[17]

Stanton, I., Kliot, G.: Streaming Graph Partitioning for Large Distributed Graphs. In: Proc. of the 18th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. KDD '12, ACM (2012) 1222--1230

[18]

Sanders, P., Schulz, C.: Think Locally, Act Globally: Highly Balanced Graph Partitioning. In: Proc. of the 12th Int. Symp. on Experimental Algorithms (SEA'13). LNCS, Springer (2013)

[19]

Zeh, N.: I/O-efficient Graph Algorithms. EEF Summer School on Massive Data Sets (2002)

[20]

Sanders, P.: Fast priority queues for cached memory. ACM Journal of Experimental Algorithmics 5 (1999) 312--327

[21]

Arge, L.: The Buffer Tree: A Technique for Designing Batched External Data Structures. Algorithmica 37(1) (2003) 1--24

[22]

Beckmann, A., Bingmann, T., Dementiev, R., Sanders, P., Singler, J.: STXXL Home Page. http://stxxl.sourceforge.net/

[23]

University of Florida. Sparse Matrix Collection.

[24]

Laboratory of Web Algorithms, University of Milano, Datasets

[25]

Holtgrewe, M., Sanders, P., Schulz, C.: Engineering a Scalable High Quality Graph Partitioner. Proc. of the 24th Int. Parallel and Distributed Processing Symp. (2010) 1--12

[26]

Akhremtsev, Y., Sanders, P., Schulz, C.: (Semi-)External Algorithms for Graph Partitioning and Clustering). Technical report, Karlsruhe Institute of Technology (2014)

Cited By

Valejo AFerreira VFabbri ROliveira MLopes A(2020)A Critical Survey of the Multilevel Method in Complex NetworksACM Computing Surveys10.1145/337934753:2(1-35)Online publication date: 17-Apr-2020
https://dl.acm.org/doi/10.1145/3379347
Henzinger MNoe ASchulz CStrash D(2018)Practical Minimum Cut AlgorithmsACM Journal of Experimental Algorithmics10.1145/327466223(1-22)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3274662

Index Terms

(Semi-)external algorithms for graph partitioning and clustering
1. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
      1. Graph algorithms

Index terms have been assigned to the content through auto-classification.

Recommendations

Fast Approximate Graph Partitioning Algorithms

We study graph partitioning problems on graphs with edge capacities and vertex weights. The problems of b-balanced cuts and k-balanced partitions are unified into a new problem called minimum capacity $\rho$-separators. A $\rho$-separator is a subset of ...
Partitioning (hierarchically clustered) complex networks via size-constrained graph clustering

The most commonly used method to tackle the graph partitioning problem in practice is the multilevel metaheuristic. In this paper we introduce size-constrained label propagation (SCLaP) and show how it can be used to instantiate both the coarsening ...
Reusable components for partitioning clustering algorithms

Clustering algorithms are well-established and widely used for solving data-mining tasks. Every clustering algorithm is composed of several solutions for specific sub-problems in the clustering process. These solutions are linked together in a ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ALENEX '15: Proceedings of the Meeting on Algorithm Engineering & Expermiments

January 2015

187 pages

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 05 January 2015

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Valejo AFerreira VFabbri ROliveira MLopes A(2020)A Critical Survey of the Multilevel Method in Complex NetworksACM Computing Surveys10.1145/337934753:2(1-35)Online publication date: 17-Apr-2020
https://dl.acm.org/doi/10.1145/3379347
Henzinger MNoe ASchulz CStrash D(2018)Practical Minimum Cut AlgorithmsACM Journal of Experimental Algorithmics10.1145/327466223(1-22)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3274662

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents