research-article

An efficient parallel solver for SDD linear systems

Authors:

Daniel A. SpielmanAuthors Info & Claims

STOC '14: Proceedings of the forty-sixth annual ACM symposium on Theory of computing

Pages 333 - 342

https://doi.org/10.1145/2591796.2591832

Published: 31 May 2014 Publication History

Abstract

We present the first parallel algorithm for solving systems of linear equations in symmetric, diagonally dominant (SDD) matrices that runs in polylogarithmic time and nearly-linear work. The heart of our algorithm is a construction of a sparse approximate inverse chain for the input matrix: a sequence of sparse matrices whose product approximates its inverse. Whereas other fast algorithms for solving systems of equations in SDD matrices exploit low-stretch spanning trees, our algorithm only requires spectral graph sparsifiers.

Supplementary Material

MP4 File (p333-sidebyside.mp4)

Download
247.32 MB

References

[1]

{Axe94} Owe Axelsson. Iterative Solution Methods. Cambridge University Press, New York, NY, 1994.

[2]

{BGH⁺06} M. Bern, J. Gilbert, B. Hendrickson, N. Nguyen, and S. Toledo. Support-graph preconditioners. SIAM J. Matrix Anal. & Appl, 27(4):930--951, 2006.

Digital Library

[3]

{BGK⁺11} Guy E Blelloch, Anupam Gupta, Ioannis Koutis, Gary L Miller, Richard Peng, and Kanat Tangwongsan. Near linear-work parallel sdd solvers, low-diameter decomposition, and low-stretch subgraphs. In Proceedings of the 23rd ACM symposium on Parallelism in algorithms and architectures, pages 13--22. ACM, 2011.

Digital Library

[4]

{BH01} Erik Boman and B. Hendrickson. On spanning tree preconditioners. Manuscript, Sandia National Lab., 2001.

[5]

{BH03} Erik G. Boman and Bruce Hendrickson. Support theory for preconditioning. SIAM Journal on Matrix Analysis and Applications, 25(3):694--717, 2003.

Digital Library

[6]

{BHM01} W. L. Briggs, V. E. Henson, and S. F. McCormick. A Multigrid Tutorial, 2nd Edition. SIAM, 2001.

Digital Library

[7]

{BHV08} Erik G. Boman, Bruce Hendrickson, and Stephen A. Vavasis. Solving elliptic finite element systems in near-linear time with support preconditioners. SIAM J. on Numerical Analysis, 46(6):3264--3284, 2008.

Digital Library

[8]

{BSS12} Joshua Batson, Daniel A Spielman, and Nikhil Srivastava. Twice-Ramanujan sparsifiers. SIAM Journal on Computing, 41(6):1704--1721, 2012.

Digital Library

[9]

{CKM⁺11} Paul Christiano, Jonathan A. Kelner, Aleksander Madry, Daniel A. Spielman, and Shang-Hua Teng. Electrical ows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In Proceedings of the 43rd annual ACM symposium on Theory of computing, STOC '11, pages 273--282, New York, NY, USA, 2011. ACM.

Digital Library

[10]

{DS08} Samuel I. Daitch and Daniel A. Spielman. Faster approximate lossy generalized flow via interior point algorithms. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing, pages 451--460, 2008.

Digital Library

[11]

{Jos97} Anil Joshi. Topics in Optimization and Sparse Linear Systems. PhD thesis, UIUC, 1997.

Digital Library

[12]

{KL13} Jonathan A Kelner and Alex Levin. Spectral sparsification in the semi-streaming setting. Theory of Computing Systems, 53(2):243--262, 2013.

[13]

{KM07} Ioannis Koutis and Gary L. Miller. A linear work, o(n^1/6) time, parallel algorithm for solving planar Laplacians. In Proceedings of the 18th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1002--1011, 2007.

Digital Library

[14]

{KM09} J. A. Kelner and A. Madry. Faster generation of random spanning trees. In Foundations of Computer Science, 2009. FOCS'09. 50th Annual IEEE Symposium on, pages 13--21. IEEE, 2009. This result was substantially improved as a result of an observation by James Propp. He will be added as a coauthor on the journal version.

Digital Library

[15]

{KMP10} I. Koutis, G. L. Miller, and R. Peng. Approaching optimality for solving sdd linear systems. In Foundations of Computer Science (FOCS), 2010 51st Annual IEEE Symposium on, pages 235--244, 2010.

Digital Library

[16]

{KMP11} I. Koutis, G. L. Miller, and R. Peng. A nearly-mlogn time solver for sdd linear systems. In Foundations of Computer Science (FOCS), 2011 52nd Annual IEEE Symposium on, pages 590--598, 2011.

Digital Library

[17]

{KMP12} Jonathan A. Kelner, Gary L. Miller, and Richard Peng. Faster approximate multicommodity flow using quadratically coupled ows. In Proceedings of the 44th symposium on Theory of Computing, STOC '12, pages 1--18, New York, NY, USA, 2012. ACM.

Digital Library

[18]

{KMT11} Ioannis Koutis, Gary L Miller, and David Tolliver. Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. Computer Vision and Image Understanding, 115(12):1638--1646, 2011.

Digital Library

[19]

{KOSZ13} Jonathan A Kelner, Lorenzo Orecchia, Aaron Sidford, and Zeyuan Allen Zhu. A simple, combinatorial algorithm for solving sdd systems in nearly-linear time. In Proceedings of the 45th annual ACM symposium on Symposium on theory of computing, pages 911--920. ACM, 2013.

Digital Library

[20]

{Kou14} Ioannis Koutis. A simple parallel algorithm for spectral sparsification. arXiv preprint arXiv:1402.3851, 2014.

[21]

{LKP12} Alex Levin, Ioannis Koutis, and Richard Peng. Improved spectral sparsification and numerical algorithms for sdd matrices. In Proceedings of the 29th Symposium on Theoretical Aspects of Computer Science (STACS), 2012. to appear.

[22]

{LS13} Yin Tat Lee and Aaron Sidford. Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems. In Proceedings of the 54th Annual IEEE Symposium on Foundations of Computer Science, 2013.

Digital Library

[23]

{Mad13} Aleksander Madry. Navigating central path with electrical ows: from ows to matchings, and back. In Proceedings of the 54th Annual IEEE Symposium on Foundations of Computer Science, 2013.

Digital Library

[24]

{OSV12} Lorenzo Orecchia, Sushant Sachdeva, and Nisheeth K. Vishnoi. Approximating the exponential, the lanczos method and an Õ(m)-time spectral algorithm for balanced separator. In Proceedings of The Fourty-Fourth Annual ACM Symposium On The Theory Of Computing (STOC '12), 2012. to appear.

Digital Library

[25]

{OV11} Lorenzo Orecchia and Nisheeth K Vishnoi. Towards an sdp-based approach to spectral methods: A nearly-linear-time algorithm for graph partitioning and decomposition. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms, pages 532--545. SIAM, 2011.

Digital Library

[26]

{Rei98} John Reif. Efficient approximate solution of sparse linear systems. Computers and Mathematics with Applications, 36(9):37--58, 1998.

[27]

{Rud99} M. Rudelson. Random vectors in the isotropic position,. Journal of Functional Analysis, 164(1):60--72, 1999.

[28]

{RV07} Mark Rudelson and Roman Vershynin. Sampling from large matrices: An approach through geometric functional analysis. Journal of the ACM, 54(4):21, 2007.

Digital Library

[29]

{Saa03} Y. Saad. Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2nd edition, 2003.

[30]

{SS08} Daniel A. Spielman and Nikhil Srivastava. Graph sparsification by effective resistances. In Proceedings of the 40th annual ACM Symposium on Theory of Computing, pages 563--568, 2008.

Digital Library

[31]

{ST11} Daniel A. Spielman and Shang-Hua Teng. Spectral sparsification of graphs. SIAM Journal on Computing, 40(4):981--1025, 2011.

Digital Library

[32]

{ST13} Daniel A. Spielman and Shang-Hua Teng. A local clustering algorithm for massive graphs and its application to nearly linear time graph partitioning. SIAM Journal on Computing, 42(1):1--26, 2013.

[33]

{ST14} Daniel A. Spielman and Shang-Hua Teng. Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. SIAM Journal on Matrix Analysis and Applications, 2014. to appear.

[34]

{Vai90} Pravin M. Vaidya. Solving linear equations with symmetric diagonally dominant matrices by constructing good preconditioners. Unpublished manuscript UIUC 1990. A talk based on the manuscript was presented at the IMA Workshop on Graph Theory and Sparse Matrix Computation, October 1991, Minneapolis., 1990.

[35]

{ZBL⁺03} Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In Adv. in Neural Inf. Proc. Sys. 16, pages 321--328, 2003.

[36]

{ZGL03} Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. Semi-supervised learning using Gaussian fields and harmonic functions. In Proc. 20th Int. Conf. on Mach. Learn., 2003.

[37]

{ZS04} Dengyong Zhou and Bernhard Schölkopf. A regularization framework for learning from graph data. In ICML Workshop on Statistical Relational Learning and Its Connections to Other Fields, pages 132--137, 2004.

Cited By

Tian YHow J(2024)Spectral Sparsification for Communication-Efficient Collaborative Rotation and Translation EstimationIEEE Transactions on Robotics10.1109/TRO.2023.332763540(257-276)Online publication date: 2024
https://doi.org/10.1109/TRO.2023.3327635
Dabeer SBagchi ANarain R(2024)GPU-LSolve: An Efficient GPU-Based Laplacian Solver for Million-Scale Graphs2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW63119.2024.00158(890-899)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPSW63119.2024.00158
Jambulapati ALiu YSidford ASaha BServedio R(2023)Chaining, Group Leverage Score Overestimates, and Fast Spectral Hypergraph SparsificationProceedings of the 55th Annual ACM Symposium on Theory of Computing10.1145/3564246.3585136(196-206)Online publication date: 2-Jun-2023
https://dl.acm.org/doi/10.1145/3564246.3585136
Show More Cited By

Index Terms

An efficient parallel solver for SDD linear systems
1. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computations on matrices

Recommendations

Sparsified Cholesky and multigrid solvers for connection laplacians
STOC '16: Proceedings of the forty-eighth annual ACM symposium on Theory of Computing

We introduce the sparsified Cholesky and sparsified multigrid algorithms for solving systems of linear equations. These algorithms accelerate Gaussian elimination by sparsifying the nonzero matrix entries created by the elimination process. We use ...
Solving SDD linear systems in nearly mlog^1/2n time
STOC '14: Proceedings of the forty-sixth annual ACM symposium on Theory of computing

We show an algorithm for solving symmetric diagonally dominant (SDD) linear systems with m non-zero entries to a relative error of ε in O(m log^1/2 n log^c n log(1/ε)) time. Our approach follows the recursive preconditioning framework, which aims to ...
A Simple and Efficient Parallel Laplacian Solver
SPAA '23: Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures

A symmetric matrix is called a Laplacian if it has nonpositive off-diagonal entries and zero row sums. Since the seminal work of Spielman and Teng (2004) on solving Laplacian linear systems in nearly linear time, several algorithms have been designed for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC '14: Proceedings of the forty-sixth annual ACM symposium on Theory of computing

May 2014

984 pages

ISBN:9781450327107

DOI:10.1145/2591796

Program Chair:
David Shmoys
Cornell University

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

STOC '14

Sponsor:

SIGACT

STOC '14: Symposium on Theory of Computing

May 31 - June 3, 2014

New York, New York

Acceptance Rates

STOC '14 Paper Acceptance Rate 91 of 319 submissions, 29%;

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
412
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tian YHow J(2024)Spectral Sparsification for Communication-Efficient Collaborative Rotation and Translation EstimationIEEE Transactions on Robotics10.1109/TRO.2023.332763540(257-276)Online publication date: 2024
https://doi.org/10.1109/TRO.2023.3327635
Dabeer SBagchi ANarain R(2024)GPU-LSolve: An Efficient GPU-Based Laplacian Solver for Million-Scale Graphs2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)10.1109/IPDPSW63119.2024.00158(890-899)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPSW63119.2024.00158
Jambulapati ALiu YSidford ASaha BServedio R(2023)Chaining, Group Leverage Score Overestimates, and Fast Spectral Hypergraph SparsificationProceedings of the 55th Annual ACM Symposium on Theory of Computing10.1145/3564246.3585136(196-206)Online publication date: 2-Jun-2023
https://dl.acm.org/doi/10.1145/3564246.3585136
Ahmadinejad APeebles JPyne ESidford AVadhan S(2023)Singular Value Approximation and Sparsifying Random Walks on Directed Graphs2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00054(846-854)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00054
van Den Brand JZhang D(2023)Faster High Accuracy Multi-Commodity Flow from Single-Commodity Techniques2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS)10.1109/FOCS57990.2023.00036(493-502)Online publication date: 6-Nov-2023
https://doi.org/10.1109/FOCS57990.2023.00036
Henzinger MJin BPeng RWilliamson D(2023)A Combinatorial Cut-Toggling Algorithm for Solving Laplacian Linear SystemsAlgorithmica10.1007/s00453-023-01154-885:12(3680-3716)Online publication date: 1-Aug-2023
https://doi.org/10.1007/s00453-023-01154-8
Anagnostides ILenzen CHaeupler BZuzic GGouleakis T(2023)Almost universally optimal distributed Laplacian solvers via low-congestion shortcutsDistributed Computing10.1007/s00446-023-00454-036:4(475-499)Online publication date: 31-Jul-2023
https://doi.org/10.1007/s00446-023-00454-0
de Vos T(2023)Minimum Cost Flow in the CONGEST ModelStructural Information and Communication Complexity10.1007/978-3-031-32733-9_18(406-426)Online publication date: 25-May-2023
https://doi.org/10.1007/978-3-031-32733-9_18
Peng RSong ZLeonardi SGupta A(2022)Sparsified block elimination for directed laplaciansProceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3519935.3520053(557-567)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519935.3520053
Zhu ZChoi GLui L(2022)Parallelizable Global Quasi-Conformal Parameterization of Multiply Connected Surfaces via Partial WeldingSIAM Journal on Imaging Sciences10.1137/21M146632315:4(1765-1807)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1137/21M1466323
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents