research-article

Open access

Bidiagonal SVD Computation via an Associated Tridiagonal Eigenproblem

Authors:

Paulo B. VasconcelosAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 46, Issue 2

Article No.: 14, Pages 1 - 25

https://doi.org/10.1145/3361746

Published: 19 May 2020 Publication History

All formats PDF

Abstract

The Singular Value Decomposition (SVD) is widely used in numerical analysis and scientific computing applications, including dimensionality reduction, data compression and clustering, and computation of pseudo-inverses. In many cases, a crucial part of the SVD of a general matrix is to find the SVD of an associated bidiagonal matrix. This article discusses an algorithm to compute the SVD of a bidiagonal matrix through the eigenpairs of an associated symmetric tridiagonal matrix. The algorithm enables the computation of only a subset of singular values and corresponding vectors, with potential performance gains. The article focuses on a sequential version of the algorithm, and discusses special cases and implementation details. The implementation, called BDSVDX, has been included in the LAPACK library. We use a large set of bidiagonal matrices to assess the accuracy of the implementation, both in single and double precision, as well as to identify potential shortcomings. The results show that BDSVDX can be up to three orders of magnitude faster than existing algorithms, which are limited to the computation of a full SVD. We also show comparisons of an implementation that uses BDSVDX as a building block for the computation of the SVD of general matrices.

References

[1]

M. Abramowitz. 1974. Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables. Dover Publications, Inc., New York, NY.

Digital Library

[2]

E. Anderson, Z. Bai, C. Bischof, L. S. Blackford, J. Demmel, J. J. Dongarra, J. Du Croz, S. Hammarling, A. Greenbaum, A. McKenney, and D. Sorensen. 1999. LAPACK Users’ Guide (3rd ed.). Society for Industrial and Applied Mathematics, Philadelphia, PA.

[3]

T. Auckenthaler, V. Blum, H. J. Bungartz, T. Huckle, R. Johanni, L. Krämer, B. Lang, H. Lederer, and P. R. Willems. 2011. Parallel solution of partial symmetric eigenvalue problems from electronic structure calculations. Parallel Comput. 37 (2011), 783--794.

Digital Library

[4]

Å. Björck. 1996. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, Philadelphia, PA.

[5]

L. S. Blackford, J. Choi, A. Cleary, E. D’Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley. 1997. ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia, PA.

[6]

J. Demmel. 1997. Applied Numerical Linear Algebra. Vol. 56. Society for Industrial and Applied Mathematics, Philadelphia, PA.

Digital Library

[7]

J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst. 2000. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide. Society for Industrial and Applied Mathematics, Philadelphia, PA. Bai, Zhaojun (Ed.).

[8]

J. Demmel and W. Kahan. 1990. Accurate singular values of bidiagonal matrices. SIAM J. Sci. and Stat. Comput. 11 (1990), 873--912.

Digital Library

[9]

J. Demmel, O. Marques, C. Voemel, and B. Parlett. 2008. Performance and accuracy of LAPACK’s symmetric tridiagonal eigensolvers. SIAM J. Sci. Comput. 30 (2008), 1508--1526.

Digital Library

[10]

I. Dhillon. 1997. A New O(N²) Algorithm for the Symmetric Tridiagonal Eigenvalue/Eigenvector Problem. Ph.D. Dissertation. University of California, Berkeley.

[11]

I. Dhillon, G. Fann, and B. Parlett. 1997. Application of a new algorithm for the symmetric eigenproblem to computational quantum chemistry. In Proceedings of the 8th SIAM Conference on Parallel Processing for Scientific Computing. SIAM, Philadelphia. http://www.cs.utexas.edu/ inderjit/public_papers/fannchem1.ps.

[12]

I. Dhillon and B. Parlett. 2004. Multiple representations to compute orthogonal eigenvectors of symmetric tridiagonal matrices. Linear Algebra Appl. 387 (2004), 1--28.

[13]

I. Dhillon, B. Parlett, and C. Voemel. 2006. The design and implementation of the MRRR algorithm. ACM Trans. Math. Softw. 32 (2006), 533--560.

Digital Library

[14]

J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, S. Tomov, and I. Yamazaki. 2018. The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale. SIAM Rev. 60, 4 (2018), 808--865.

Digital Library

[15]

K. Fan and A. Hoffman. 1955. Some metric inequalities in the space of matrices. Proc. Amer. Math. Soc. 6, 1 (1955), 111--116.

[16]

G. Golub and W. Kahan. 1965. Calculating the singular values and pseudo-inverse of a matrix. J. Society for Ind. Appl. Math. Ser. B Numer. Anal. 2, 2 (1965), 205--224.

[17]

B. Grosser and B. Lang. 2003. An O(n²) algorithm for the bidiagonal SVD. Linear Algebra Appl. 358, 1 (2003), 45--70.

[18]

B. Grosser and B. Lang. 2005. On symmetric eigenproblems induced by the bidiagonal SVD. SIAM. J. Matrix Anal. and Appl. 26, 3 (2005), 599--620.

Digital Library

[19]

Nicholas J. Higham and Pythagoras Papadimitriou. 1993. Parallel Singular Value Decomposition via the Polar Decomposition. University of Manchester, Department of Mathematics.

[20]

Intel. 2017a. Intel(R) Fortran Compiler 2018 Update 1 for Linux.

[21]

Intel. 2017b. Intel(R) Math Kernel Library 2018 Update 1 for Linux.

[22]

LAPACK. 2018. LAPACK—Linear Algebra PACKage. https://github.com/Reference-LAPACK.

[23]

S. Li, M. Gu, L. Cheng, X. Chi, and M. Sun. 2014. An accelerated divide-and-conquer algorithm for the bidiagonal SVD problem. SIAM. J. Matrix Anal. and Appl. 35 (2014), 1038--1057.

[24]

O. Marques, J. Demmel, C. Voemel, and B. Parlett. 2008. A testing infrastructure for symmetric tridiagonal eigensolvers. ACM Trans. Math. Softw. 35 (2008), 8:1--8:13.

Digital Library

[25]

O. Marques and P. B. Vasconcelos. 2016. Computing the bidiagonal SVD through an associated tridiagonal eigenproblem. In 12th International Conference on High Performance Computing for Computational Science (VECPAR 2016) Porto, Portugal, June 28-30, 2016, Revised Selected Papers. Springer, Heildeberg, Germany, 64--74.

[26]

MatrixMarket. 2018. Matrix Market. Retrieved from http://math.nist.gov/MatrixMarket.

[27]

A. M. Matsekh. 2005. The Godunov-inverse iteration: A fast and accurate solution to the symmetric tridiagonal eigenvalue problem. Appl. Numer. Math. 54, 2 (2005), 208--221.

Digital Library

[28]

Yuji Nakatsukasa, Zhaojun Bai, and François Gygi. 2010. Optimizing Halley’s iteration for computing the matrix polar decomposition. SIAM J. Matrix Anal. Appl. 31, 5 (2010), 2700--2720.

Digital Library

[29]

Yuji Nakatsukasa and Nicholas J. Higham. 2013. Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the SVD. SIAM J. Sci. Comput. 35, 3 (2013), A1325--A1349.

Digital Library

[30]

F. Olver, D. Lozier, R. Boisvert, and C. Clark. 2010. NIST Handbook of Mathematical Functions Hardback and CD-ROM. Cambridge University Press, New York.

[31]

B. Parlett. 1995. The new QD algorithms. Acta Numer. 4 (1995), 459--491.

[32]

B. Parlett. 1998. The Symmetric Eigenvalue Problem. Vol. 20. Society for Industrial and Applied Mathematics, Philadelphia, PA.

[33]

B. Parlett and C. Vömel. 2009. The spectrum of a glued matrix. SIAM J. Matrix Anal. Appl. 31, 1 (2009), 114--132.

Digital Library

[34]

ScaLAPACK. 2012. ScaLAPACK, version 2.0.2. Retrieved from http://www.netlib.org/scalapack.

[35]

TAU. 2018. TAU Performance System. Retrieved from https://www.cs.uoregon.edu/research/tau.

[36]

C. Voemel. 2010. ScaLAPACK’s MRRR algorithm. ACM Trans. Math. Softw. 37 (2010), 1--35.

Digital Library

[37]

J. Vogel, J. Xia, S. Cauley, and V. Balakrishnan. 2016. Superfast divide-and-conquer method and perturbation analysis for structured eigenvalue solutions. SIAM J. Sci. Comput. 38, 3 (2016), A1358--A1382.

Digital Library

[38]

P. Willems. 2010. On MR³-type Algorithms for the Tridiagonal Symmetric Eigenproblem and the Bidiagonal SVD. Ph.D. Dissertation. University of Wuppertal.

[39]

P. Willems and B. Lang. 2013. A framework for the MR³ algorithm: Theory and implementation. SIAM J. Sci. Comput. 35 (2013), 740--766.

[40]

P. Willems, B. Lang, and C. Voemel. 2006. Computing the bidiagonal SVD using multiple relatively robust representations. SIAM. J. Matrix Anal. and Appl. 28 (2006), 907--926.

Digital Library

Cited By

Keyes DLtaief HNakatsukasa YSukkari DMohror KArnold DBadia R(2023)High-Performance SVD Partial Spectrum ComputationProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607109(1-12)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607109
Chu WZhao YYuan H(2022)A Novel Divisional Bisection Method for the Symmetric Tridiagonal Eigenvalue ProblemMathematics10.3390/math1015278210:15(2782)Online publication date: 5-Aug-2022
https://doi.org/10.3390/math10152782
Tănăsescu ACarabaş MPop FPopescu P(2021)Scalability of k-Tridiagonal Matrix Singular Value DecompositionMathematics10.3390/math92331239:23(3123)Online publication date: 3-Dec-2021
https://doi.org/10.3390/math9233123
Show More Cited By

Index Terms

Bidiagonal SVD Computation via an Associated Tridiagonal Eigenproblem
1. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computations on matrices
  2. Mathematical software
    1. Solvers

Recommendations

The design and implementation of the MRRR algorithm

In the 1990's, Dhillon and Parlett devised the algorithm of multiple relatively robust representations (MRRR) for computing numerically orthogonal eigenvectors of a symmetric tridiagonal matrix T with O(n²) cost. While previous publications related to ...
Residual Bounds on Approximate Solutions for the Unitary Eigenproblem

Let $A$ be an $n \times n$ unitary matrix, and let the columns of an $n \times l \, (l < n)$ matrix $\tilde{X}_1$ form an orthonormal basis for an approximate eigenspace $\tilde{\cal X}_1$ of $A$. Then there are two problems: How near is $\tilde{\cal X}...
Algorithm 880: A testing infrastructure for symmetric tridiagonal eigensolvers

LAPACK is often mentioned as a positive example of a software library that encapsulates complex, robust, and widely used numerical algorithms for a wide range of applications. At installation time, the user has the option of running a (limited) number ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 46, Issue 2

June 2020

274 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/3401021

Editors:
Zhaojun Bai
University of California at Davis, USA
,
Wolfgang Bangerth
Colorado State University, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 May 2020

Accepted: 01 September 2019

Revised: 01 January 2019

Received: 01 April 2018

Published in TOMS Volume 46, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation
Fundação para a Ciência e a Tecnologia (FCT) through the sabbatical fellowship

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
1,459
Total Downloads

Downloads (Last 12 months)406
Downloads (Last 6 weeks)42

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Keyes DLtaief HNakatsukasa YSukkari DMohror KArnold DBadia R(2023)High-Performance SVD Partial Spectrum ComputationProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607109(1-12)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607109
Chu WZhao YYuan H(2022)A Novel Divisional Bisection Method for the Symmetric Tridiagonal Eigenvalue ProblemMathematics10.3390/math1015278210:15(2782)Online publication date: 5-Aug-2022
https://doi.org/10.3390/math10152782
Tănăsescu ACarabaş MPop FPopescu P(2021)Scalability of k-Tridiagonal Matrix Singular Value DecompositionMathematics10.3390/math92331239:23(3123)Online publication date: 3-Dec-2021
https://doi.org/10.3390/math9233123
Yu JHe YYan QKang X(2021)SpecView: Malware Spectrum Visualization Framework With Singular Spectrum TransformationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2021.312472516(5093-5107)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1109/TIFS.2021.3124725
Isinkaye F(2021)Matrix Factorization in Recommender Systems: Algorithms, Applications, and Peculiar ChallengesIETE Journal of Research10.1080/03772063.2021.199735769:9(6087-6100)Online publication date: 21-Nov-2021
https://doi.org/10.1080/03772063.2021.1997357

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents