research-article

A scalable randomized least squares solver for dense overdetermined systems

Authors:

Georgios Kollias,

Christopher Carothers,

Petros DrineasAuthors Info & Claims

ScalA '15: Proceedings of the 6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems

Article No.: 3, Pages 1 - 8

https://doi.org/10.1145/2832080.2832083

Published: 15 November 2015 Publication History

Abstract

We present a fast randomized least-squares solver for distributed-memory platforms. Our solver is based on the Blendenpik algorithm, but employs a batchwise randomized unitary transformation scheme. The batchwise transformation enables our algorithm to scale the distributed memory vanilla implementation of Blendenpik by up to ×3 and provides up to ×7.5 speedup over a state-of-the-art scalable least-squares solver based on the classic QR based algorithm. Experimental evaluations on terabyte scale matrices demonstrate excellent speedups on up to 16384 cores on a Blue Gene/Q supercomputer.

References

[1]

H. Avron, P. Maymounkov, and S. Toledo. Blendenpik: Supercharging lapack's least-squares solver. SIAM J. Scientific Computing, 32(3):1217--1236, 2010.

Digital Library

[2]

K. L. Clarkson and D. P. Woodruff. Low rank approximation and regression in input sparsity time. CoRR, abs/1207.6365, 2012.

[3]

L. Dagum and R. Menon. OpenMP: An industry-standard API for shared-memory programming. IEEE Comput. Sci. Eng., 5(1):46--55, Jan. 1998.

Digital Library

[4]

T. A. Davis and Y. Hu. The university of florida sparse matrix collection. ACM Trans. Math. Softw., 38(1):1:1--1:25, Dec. 2011.

Digital Library

[5]

J. Demmel, L. Grigori, M. Hoemmen, and J. Langou. Communication-optimal parallel and sequential QR and LU factorizations. ArXiv e-prints, Aug. 2008.

[6]

J. Demmel and K. Yelick. Communication avoiding (CA) and other innovative algorithms. The Berkeley Par Lab: Progress in the Parallel Computing Landscape, pages 243--250.

[7]

P. Drineas, M. W. Mahoney, S. Muthukrishnan, and T. Sarlós. Faster least squares approximation. Numer. Math., 117(2):219--249, Feb. 2011.

Digital Library

[8]

M. P. Forum. MPI: A message-passing interface standard. Technical report, Knoxville, TN, USA, 1994.

Digital Library

[9]

M. Frigo and S. G. Johnson. FFTW: An adaptive software architecture for the FFT. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, volume 3, pages 1381--1384, Seattle, Washington, 1998.

[10]

K. A. Gallivan, R. J. Plemmons, and A. H. Sameh. Parallel Algorithms for Dense Linear Algebra Computations. SIAM Review, 32(1):54--135, 1990.

Digital Library

[11]

J. R. Hammond, A. Schäfer, and R. Latham. To INT_MAX... and beyond!: Exploring large-count support in MPI. In Proceedings of the 2014 Workshop on Exascale MPI, ExaMPI '14, pages 1--8, Piscataway, NJ, USA, 2014. IEEE Press.

Digital Library

[12]

C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO'04), Palo Alto, California, Mar 2004.

Digital Library

[13]

X. Meng, M. A. Saunders, and M. W. Mahoney. LSRN: A parallel iterative solver for strongly over- or under-determined systems. CoRR, abs/1109.5981, 2011.

[14]

C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw., 8(1):43--71, Mar. 1982.

Digital Library

[15]

J. Poulson, B. Marker, R. A. van de Geijn, J. R. Hammond, and N. A. Romero. Elemental: A new framework for distributed memory dense matrix computations. ACM Trans. Math. Softw., 39(2):13:1--13:24, Feb. 2013.

Digital Library

[16]

V. Rokhlin and M. Tygert. A fast randomized algorithm for overdetermined linear least-squares regression. Proc. Natl. Acad. Sci. USA, 105(36):13212--13217, 2008.

[17]

J. Yang, X. Meng, and M. W. Mahoney. Implementing randomized matrix algorithms in parallel and distributed environments. CoRR, abs/1502.03032, 2015.

Cited By

Hervert-Escobar LAlexandrov V(2022)Territorial design optimization for business sales planJournal of Computational and Applied Mathematics10.1016/j.cam.2018.02.010340:C(501-507)Online publication date: 13-Apr-2022
https://dl.acm.org/doi/10.1016/j.cam.2018.02.010
Iyer CCarothers CDrineas P(2016)Randomized sketching for large-scale sparse ridge regression problemsProceedings of the 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems10.5555/3019094.3019103(65-72)Online publication date: 13-Nov-2016
https://dl.acm.org/doi/10.5555/3019094.3019103
Iyer CCarothers CDrineas P(2016)Randomized Sketching for Large-Scale Sparse Ridge Regression Problems2016 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScalA)10.1109/ScalA.2016.013(65-72)Online publication date: Nov-2016
https://doi.org/10.1109/ScalA.2016.013

Index Terms

A scalable randomized least squares solver for dense overdetermined systems

Recommendations

Blendenpik: Supercharging LAPACK's Least-Squares Solver

Several innovative random-sampling and random-mixing techniques for solving problems in linear algebra have been proposed in the last decade, but they have not yet made a significant impact on numerical linear algebra. We show that by using a high-...
Superfast and Stable Structured Solvers for Toeplitz Least Squares via Randomized Sampling

We present some superfast ($O((m+n)\log^{2}(m+n))$ complexity) and stable structured direct solvers for $m\times n$ Toeplitz least squares problems. Based on the displacement equation, a Toeplitz matrix $T$ is first transformed into a Cauchy-like matrix $\...
Extra-Precise Iterative Refinement for Overdetermined Least Squares Problems

We present the algorithm, error bounds, and numerical results for extra-precise iterative refinement applied to overdetermined linear least squares (LLS) problems. We apply our linear system refinement algorithm to Björck’s augmented linear system ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ScalA '15: Proceedings of the 6th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems

November 2015

53 pages

ISBN:9781450340113

DOI:10.1145/2832080

Conference Chairs:
Vassil Alexandrov
ICREA-Barcelona Supercomputing Center, Spain
,
Al Geist
Oak Ridge National Laboratory
,
Jack Dongarra
University of Tennessee, Knoxville

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing
SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS\DATC: IEEE Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SC15

Sponsor:

SIGHPC
SIGARCH
IEEE-CS\DATC

SC15: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 15, 2015

Texas, Austin

Acceptance Rates

ScalA '15 Paper Acceptance Rate 6 of 10 submissions, 60%;

Overall Acceptance Rate 12 of 20 submissions, 60%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
101
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hervert-Escobar LAlexandrov V(2022)Territorial design optimization for business sales planJournal of Computational and Applied Mathematics10.1016/j.cam.2018.02.010340:C(501-507)Online publication date: 13-Apr-2022
https://dl.acm.org/doi/10.1016/j.cam.2018.02.010
Iyer CCarothers CDrineas P(2016)Randomized sketching for large-scale sparse ridge regression problemsProceedings of the 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems10.5555/3019094.3019103(65-72)Online publication date: 13-Nov-2016
https://dl.acm.org/doi/10.5555/3019094.3019103
Iyer CCarothers CDrineas P(2016)Randomized Sketching for Large-Scale Sparse Ridge Regression Problems2016 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScalA)10.1109/ScalA.2016.013(65-72)Online publication date: Nov-2016
https://doi.org/10.1109/ScalA.2016.013

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents