A Framework for Parallelizing Approximate Gaussian Elimination
Authors: 
Yves Baumann, Rasmus KyngAuthors Info & Claims
Yves Baumann, Rasmus KyngAuthors Info & ClaimsPages 195 - 206
Abstract
In a breakthrough result, Spielman and Teng (2004) developed a nearly-linear time solver for Laplacian linear equations, i.e. equations where the coefficient matrix is symmetric with non-negative diagonals and zero row sums. Since the development of the Spielman-Teng solver, there has been substantial progress, simplifying and improving their result, but obtaining a fast practical, parallel Laplacian solver remains an open problem.
We present a framework for obtaining extremely simple, parallel Laplacian linear equation solvers with nearly-linear work and sublinear depth. Our framework allows us to parallelize any Laplacian solver based on repeated single-vertex approximate Gaussian elimination. We demonstrate this by parallelizing both the algorithm of Kyng and Sachdeva (2016) and the practical variant by Gao, Kyng, and Spielman (2023). Our framework is work-efficient in the sense of matching the sequential work of these algorithms.
Our parallelization framework is very simple: We sample a subset of the current low-degree vertices (sparse columns), and in parallel we eliminate all vertices that are isolated in the resulting induced subgraph. This approach can be combined with any parallelizable approximate single-vertex elimination subroutine with sparse output. Given the simplicity of the approach, we believe that using it to parallelize the solver of Gao, Kyng, and Spielman (2023) is the most promising direction for obtaining practical parallel Laplacian solvers.
If we additionally use a parallel spectral sparsification routine, our approach can be modified to work in polylogarithmic depth and nearly-linear work.
References
[1]
Soheil Behnezhad, Mahsa Derakhshan, MohammadTaghi Hajiaghayi, Cliff Stein, and Madhu Sudan. 2019. Fully Dynamic Maximal Independent Set with Polylogarithmic Update Time. arxiv: 1909.03478 [cs.DS]
[2]
Erik Boman, Bruce Hendrickson, and Stephen Vavasis. 2008. Solving Elliptic Finite Element Systems in Near-Linear Time with Support Preconditioners. arxiv: cs/0407022 [cs.NA]
[3]
Martin Burtscher, Sindhu Devale, Sahar Azimi, Jayadharini Jaiganesh, and Evan Powers. 2018. A High-Quality and Fast Maximal Independent Set Implementation for GPUs. ACM Trans. Parallel Comput., Vol. 5, 2, Article 8 (dec 2018), bibinfonumpages27 pages. https://doi.org/10.1145/3291525
[4]
Shiri Chechik and Tianyi Zhang. 2021. Fully Dynamic Maximal Independent Set in Expected Poly-Log Update Time. arxiv: 1909.03445 [cs.DS]
[5]
Chao Chen, Tianyu Liang, and George Biros. 2021. RCHOL: Randomized Cholesky Factorization for Solving SDD Linear Systems. arxiv: 2011.07769 [math.NA]
[6]
Michael B. Cohen, Rasmus Kyng, Gary L. Miller, Jakub W. Pachocki, Richard Peng, Anup B. Rao, and Shen Chen Xu. 2014. Solving SDD Linear Systems in Nearly m Log1/2 n Time. In Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing. 343--352.
[7]
Samuel I. Daitch and Daniel A. Spielman. 2008. Faster Approximate Lossy Generalized Flow via Interior Point Algorithms. arxiv: 0803.0988 [cs.DS]
[8]
Laxman Dhulipala, Guy E. Blelloch, and Julian Shun. 2018. Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable. In Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures (SPAA '18). ACM. https://doi.org/10.1145/3210377.3210414
[9]
Yuan Gao, Rasmus Kyng, and Daniel A. Spielman. 2023. Robust and Practical Solution of Laplacian Equations by Approximate Elimination. arxiv: 2303.00709 [math.NA]
[10]
Lorenz Hübschle-Schneider and Peter Sanders. 2021. Parallel Weighted Random Sampling. arxiv: 1903.00227 [cs.DS]
[11]
Arun Jambulapati and Aaron Sidford. 2021. Ultrasparse Ultrasparsifiers and Faster Laplacian System Solvers. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA ). SIAM, 540--559.
[12]
Jonathan A. Kelner, Lorenzo Orecchia, Aaron Sidford, and Zeyuan Allen Zhu. 2013. A Simple, Combinatorial Algorithm for Solving SDD Systems in Nearly-Linear Time. In Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing. 911--920.
[13]
Ioannis Koutis. 2014. Simple parallel and distributed algorithms for spectral graph sparsification. arxiv: 1402.3851 [cs.DS]
[14]
Ioannis Koutis, Gary L. Miller, and Richard Peng. 2010. Approaching Optimality for Solving SDD Linear Systems. In Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer Science (FOCS '10). IEEE Computer Society, USA, 235--244. https://doi.org/10.1109/FOCS.2010.29
[15]
Ioannis Koutis, Gary L. Miller, and Richard Peng. 2011. A Nearly-m Log n Time Solver for Sdd Linear Systems. In 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science. IEEE, 590--598.
[16]
Rasmus Kyng, Yin Tat Lee, Richard Peng, Sushant Sachdeva, and Daniel A. Spielman. 2016. Sparsified Cholesky and Multigrid Solvers for Connection Laplacians. In Proceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing. 842--850.
[17]
Rasmus Kyng and Sushant Sachdeva. 2016. Approximate Gaussian Elimination for Laplacians-Fast, Sparse, and Simple. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS ). IEEE, 573--582.
[18]
Yin Tat Lee, Richard Peng, and Daniel A. Spielman. 2015. Sparsified Cholesky Solvers for SDD linear systems. arxiv: 1506.08204 [cs.DS]
[19]
Richard Peng and Daniel A. Spielman. 2013. An Efficient Parallel Solver for SDD Linear Systems. arxiv: 1311.3286 [cs.NA]
[20]
Richard Peng and Daniel A. Spielman. 2014. An Efficient Parallel Solver for SDD Linear Systems. In Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing. 333--342.
[21]
Sushant Sachdeva and Yibin Zhao. 2023. A Simple and Efficient Parallel Laplacian Solver. In Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA '23). ACM. https://doi.org/10.1145/3558481.3591101
[22]
Daniel A. Spielman and Shang-Hua Teng. 2004. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the Thirty-Sixth Annual ACM Symposium on Theory of Computing (Chicago, IL, USA) (STOC '04). Association for Computing Machinery, New York, NY, USA, 81--90. https://doi.org/10.1145/1007352.1007372
[23]
Joel A. Tropp. 2011a. Freedman's inequality for matrix martingales. arxiv: 1101.3039 [math.PR]
[24]
Joel A. Tropp. 2011b. User-Friendly Tail Bounds for Sums of Random Matrices. Foundations of Computational Mathematics, Vol. 12, 4 (Aug. 2011), 389--434. https://doi.org/10.1007/s10208-011--9099-z io
Index Terms
- A Framework for Parallelizing Approximate Gaussian Elimination
Recommendations
Fast Connected Components Algorithms for the EREW PRAM
We present fast and efficient parallel algorithms for finding the connected components of an undirected graph. These algorithms run on the exclusive-read, exclusive-write (EREW) PRAM. On a graph with n vertices and m edges, our randomized algorithm ...
A Fast Derandomization Scheme and Its Applications
This paper presents a fast derandomization scheme for the PROFIT/COST problem. Through the applications of this scheme the time complexity of $O(\log^2 n \log \log n)$ for the $\Delta+1$ vertex-coloring problem using $O((m+n)/\log \log n)$ processors on ...
A Parallel Priority Queue with Constant Time Operations
Parallel and distributed data structuresWe present a parallel priority queue that supports the following operations in constant time:parallel insertionof a sequence of elements ordered according to key,parallel decrease keyfor a sequence of elements ordered according to key,deletion of the ...
Comments
Information & Contributors
Information
Published In
June 2024
510 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGACT: ACM Special Interest Group on Algorithms and Computation Theory
- SIGARCH: ACM Special Interest Group on Computer Architecture
- EATCS: European Association for Theoretical Computer Science
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Swiss National Science Foundation
Conference
SPAA '24: 36th ACM Symposium on Parallelism in Algorithms and Architectures
June 17 - 21, 2024
Nantes, France
Acceptance Rates
Overall Acceptance Rate 447 of 1,461 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 67Total Downloads
- Downloads (Last 12 months)67
- Downloads (Last 6 weeks)7
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in