research-article

Minimization Techniques for Piecewise Differentiable Functions: : The $l_1$ Solution to an Overdetermined Linear System

Authors:

Richard H. Bartels,

Andrew R. Conn,

James W. SinclairAuthors Info & Claims

SIAM Journal on Numerical Analysis, Volume 15, Issue 2

Pages 224 - 241

https://doi.org/10.1137/0715015

Published: 01 April 1978 Publication History

Abstract

A new algorithm is presented for computing a vector x which satisfies a given m by $n(m > n \geqq 2)$ linear system in the sense that the $l_1 $ norm is minimized. That is, if A is a matrix having m columns $a_1, \cdots,a_m $ each of length n, and b is a vector with components $\beta _1, \cdots,\beta _m $, then x is selected so that \[\phi (x) = ||A^T x - b||_1 = \sum _{i = 1}^m {\left|a_i^T x - \beta _i \right|} \] is as small as possible. Such solutions are of interest for the “robust” fitting of a linear model to data.

The function $\phi $ is directly minimized in a finite number of steps using techniques borrowed from Conn’s approach toward minimizing piecewise differentiable functions. In these techniques if x is any point and $A_\mathcal {Z} $ stands for the submatrix consisting of those columns $a_j$ from A for which the corresponding residuals $a_j^T x - \beta _j $ are zero, then the discontinuities in the gradient of $\phi $ at x are handled by making use of the projector onto the null space of $A_\mathcal {Z}^T $.

Attention has been paid both to numerical stability and efficiency in maintaining and updating a factorization of $A_\mathcal{Z} $ from which the necessary projector is obtainable.

The algorithm compares favorably with the best so far reported for the linear $l_1$ problem, and it can easily be extended to handle linear constraints.

References

[1]

I. Barrodale, F. D. K. Roberts, An improved algorithm for discrete $l\sb{1}$ linear approximation, SIAM J. Numer. Anal., 10 (1973), 839–848

Digital Library

[2]

I. Barrodale, F. D. K. Roberts, Algorithm 478, Solution of an overdetermined system of equations in the $l_{1}$ norm, Comm. ACM, 17 (1974), 319–320

Digital Library

[3]

Richard H. Bartels, Conk Andrew R., James W. Sinclair, A FORTRAN program for solving overdetermined systems of linear equations in the l1 sense, Tech. Rep., 236, Mathematical Sciences Dept., Johns Hopkins Univ., Baltimore, MD, 1976

[4]

Jon F. Claerbout, Francis Muir, Robust modeling with erratic data, Geophysics, 38 (1973), 826–844

[5]

A. R. Conn, Linear programming via a nondifferentiable penalty function, SIAM J. Numer. Anal., 13 (1976), 145–154

Digital Library

[6]

J. W. Daniel, W. B. Gragg, L. Kaufman, G. W. Stewart, Reorthogonalization and stable algorithms for updating the Gram-Schmidt $QR$ factorization, Math. Comp., 30 (1976), 772–795

[7]

George E. Forsythe, Cleve B. Moler, Computer solution of linear algebraic systems, Prentice-Hall Inc., Englewood Cliffs, N.J., 1967xi+148

[8]

W. Morven Gentleman, Least squares computations by Givens transformations without square roots, J. Inst. Math. Appl., 12 (1973), 329–336

[9]

P. E. Gill, G. H. Golub, W. Murray, M. A. Saunders, Methods for modifying matrix factorizations, Math. Comp., 28 (1974), 505–535

[10]

Gill Philip E., Walter Murray, Two methods for the solution of linearly constrained and unconstrained optimization, Tech. Rep., NAC 25, National Physical Laboratory, Teddington, Middlesex, England, 1972

[11]

Philip E. Gill, Walter Murray, Michael A. Saunders, Methods for computing and modifying the $LDV$ factors of a matrix, Math. Comp., 29 (1975), 1051–1077

[12]

Donald Goldfarb, Matrix factorizations in optimization of non-linear functions subject to linear constraints, Math. Programming, 10 (1976), 1–31

Digital Library

[13]

G. Hadley, Linear programming, Addison-Wesley Series in Industrial Management, Addison-Wesley Publishing Co., Inc., Reading, Mass.-London, 1962xii+520

[14]

Charles L. Lawson, Richard J. Hanson, Solving least squares problems, Prentice-Hall Inc., Englewood Cliffs, N.J., 1974xii+340

[15]

Christoph Witzgall, On discrete l1 approximation, working paper

Cited By

Watson G(1998)On Computing the Least Quantile of Squares EstimateSIAM Journal on Scientific Computing10.1137/S106482759528376819:4(1125-1138)Online publication date: 1-Jul-1998
https://dl.acm.org/doi/10.1137/S1064827595283768
Barrodale IRoberts F(1978)An Efficient Algorithm for Discrete $l_1$ Linear Approximation with Linear ConstraintsSIAM Journal on Numerical Analysis10.1137/071504015:3(603-611)Online publication date: 1-Jun-1978
https://dl.acm.org/doi/10.1137/0715040
Bartels RConn ACharalambous C(1978)On Cline’s Direct Method for Solving Overdetermined Linear Systems in the $L_\infty $ SenseSIAM Journal on Numerical Analysis10.1137/071501715:2(255-270)Online publication date: 1-Apr-1978
https://dl.acm.org/doi/10.1137/0715017

Index Terms

Minimization Techniques for Piecewise Differentiable Functions: The $l_1$ Solution to an Overdetermined Linear System

Index terms have been assigned to the content through auto-classification.

Recommendations

Optimal Composition Ordering Problems for Piecewise Linear Functions

In this paper, we introduce maximum composition ordering problems. The input is n real functions $$f_1,\dots ,f_n:\mathbb {R}\rightarrow \mathbb {R}$$f1,ź,fn:RźR and a constant $$c\in \mathbb {R}$$cźR. We consider two settings: total and partial ...
Solving the equality-constrained minimization problem of polynomial functions
Abstract
The purpose of this paper is to solve the equality-constrained minimization problem of polynomial functions. Let $${\mathbb {R}}$$ be the field of real numbers, and $${\mathbb {R}}[x_1,\ldots ,x_n]$$ the ring of polynomials over $${\mathbb {R}}$$ ...
Problems and Techniques

The Problems and Techniques section in this issue contains three papers on very different topics: achieving reliability through optimization, improving the solution of systems of linear equations by preconditioning, and determining asymptotic ...

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Numerical Analysis

SIAM Journal on Numerical Analysis Volume 15, Issue 2

Apr 1978

213 pages

ISSN:0036-1429

DOI:10.1137/sjnaam.1978.15.issue-2

Issue’s Table of Contents

Copyright © 1978 Society for Industrial and Applied Mathematics.

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 April 1978

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Watson G(1998)On Computing the Least Quantile of Squares EstimateSIAM Journal on Scientific Computing10.1137/S106482759528376819:4(1125-1138)Online publication date: 1-Jul-1998
https://dl.acm.org/doi/10.1137/S1064827595283768
Barrodale IRoberts F(1978)An Efficient Algorithm for Discrete $l_1$ Linear Approximation with Linear ConstraintsSIAM Journal on Numerical Analysis10.1137/071504015:3(603-611)Online publication date: 1-Jun-1978
https://dl.acm.org/doi/10.1137/0715040
Bartels RConn ACharalambous C(1978)On Cline’s Direct Method for Solving Overdetermined Linear Systems in the $L_\infty $ SenseSIAM Journal on Numerical Analysis10.1137/071501715:2(255-270)Online publication date: 1-Apr-1978
https://dl.acm.org/doi/10.1137/0715017

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents