research-article

Robust principal component analysis?

Authors:

Emmanuel J. Candès,

John WrightAuthors Info & Claims

Journal of the ACM (JACM), Volume 58, Issue 3

Article No.: 11, Pages 1 - 37

https://doi.org/10.1145/1970392.1970395

Published: 09 June 2011 Publication History

Abstract

This article is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low-rank component and a sparse component. Can we recover each component individually? We prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the ℓ₁ norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.

References

[1]

Basri, R., and Jacobs, D. 2003. Lambertian reflectance and linear subspaces. IEEE Trans. Patt. Anal. Mach. Intel. 25, 2, 218--233.

Digital Library

[2]

Beck, A., and Teboulle, M. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 1, 183--202.

Digital Library

[3]

Becker, S., Bobin, J., and Candès, E. J. 2011. NESTA: A fast and accuract first-order method for sparse recovery. SIAM J. Imag. Sci. 4, 1, 1--39.

Digital Library

[4]

Belkin, M., and Niyogi, P. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 6, 1373--1396.

Digital Library

[5]

Bertsekas, D. 1982. Constrained Optimization and Lagrange Multiplier Method. Academic Press.

[6]

Cai, J., Cand&esgrave;, E. J., and Shen, Z. 2010. A singular value thresholding algorithm for matrix completion. SIAM J. Optimiz. 20, 4, 1956--1982.

Digital Library

[7]

Candès, E. J., and Plan, Y. 2010. Matrix completion with noise. Proc. IEEE 98, 6, 925--936.

[8]

Candès, E. J., and Recht, B. 2009. Exact matrix completion via convex optimzation. Found. Comput. Math. 9, 717--772.

Digital Library

[9]

Candès, E. J., Romberg, J., and Tao, T. 2006. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 52, 2, 489--509.

Digital Library

[10]

Candès, E. J., and Tao, T. 2010. The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56, 5, 2053--2080.

Digital Library

[11]

Cevher, V., Sankaranarayanan, A., Duarte, M., Reddy, D., Baraniuk, R., and Chellappa, R. 2009. Compressive sensing for background subtraction. In Proceedings of the European Conference on Computer Vision (ECCV).

Digital Library

[12]

Chandrasekaran, V., Sanghavi, S., Parrilo, P., and Willsky, A. 2009. Rank-sparsity incoherence for matrix decomposition. Siam J. Optim., to appear http://arxiv.org/abs/0906.2220.

[13]

Chen, S., Donoho, D., and Saunders, M. 2001. Atomic decomposition by basis pursuit. SIAM Rev. 43, 1, 129--159.

Digital Library

[14]

Dewester, S., Dumains, S., Landauer, T., Furnas, G., and Harshman, R. 1990. Indexing by latent semantic analysis. J. Soc. Inf. Sci. 41, 6, 391--407.

[15]

Eckart, C., and Young, G. 1936. The approximation of one matrix by another of lower rank. Psychometrika 1, 211--218.

[16]

Fazel, M., Hindi, H., and Boyd, S. 2003. Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. In Proceedings of the American Control Conference 2156--2162.

[17]

Fischler, M., and Bolles, R. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 381--385.

Digital Library

[18]

Georghiades, A., Belhumeur, P., and Kriegman, D. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Patt. Anal. Mach. Intel. 23, 6.

Digital Library

[19]

Gnanadesikan, R., and Kettenring, J. 1972. Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics 28, 81--124.

[20]

Goldfarb, D., and Ma, S. 2009. Convergence of fixed point continuation algorithms for matrix rank minimization. http://arxiv.org/abs/0906.3499.

[21]

Grant, M., and Boyd, S. 2009. CVX: Matlab software for disciplined convex programming (web page and software). http://stanford.edu/~boyd/cvx.

[22]

Gross, D. 2011. Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57, 3, 1548--1566.

Digital Library

[23]

Gross, D., Liu, Y.-K., Flammia, S. T., Becker, S., and Eisert, J. 2010. Quantum state tomography via compressed sensing. Phys. Rev. Lett. 105, 15.

[24]

Hey, T., Tansley, S., and Tolle, K. 2009. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research.

[25]

Hotelling, H. 1933. Analysis of a complex of statistical variables into principal components. J. Educat. Psych. 24, 417--441.

[26]

Huber, P. 1981. Robust Statistics. Wiley.

[27]

Jolliffe, I. 1986. Principal Component Analysis. Springer-Verlag.

[28]

Ke, Q., and Kanade, T. 2005. Robust &ell;<sup>1</sup>-norm factorization in the presence of outliers and missing data. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.

Digital Library

[29]

Keshavan, R. H., Montanari, A., and Oh, S. 2010. Matrix completion from a few entries. IEEE Trans. Inf. Theory 56, 6, 2980--2998.

Digital Library

[30]

Kontogiorgis, S., and Meyer, R. 1989. A variable-penalty alternating direction method for convex optimization. Math. Prog. 83, 29--53.

Digital Library

[31]

Ledoux, M. 2001. The Concentration of Measure Phenomenon. American Mathematical Society.

[32]

Li, L., Huang, W., Gu, I., and Tian, Q. 2004. Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans. Image Proc. 13, 11, 1459--1472.

Digital Library

[33]

Lin, Z., Chen, M., and Ma, Y. 2009a. The augmented Lagrange multiplier method for exact recovery of a corrupted low-rank matrices. http://arxiv.org/abs/1009.5055.

[34]

Lin, Z., Ganesh, A., Wright, J., Wu, L., Chen, M., and Ma, Y. 2009b. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. In Proceedings of the Symposium on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[35]

Lions, P., and Mercier, B. 1979. Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 6, 964--979.

Digital Library

[36]

Liu, Z., and Vandenberge, L. 2009. Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31, 3, 1235--1256.

Digital Library

[37]

Ma, S., Goldfarb, D., and Chen, L. 2009. Fixed point and Bregman iterative methods for matrix rank minimization. Math. Prog. Ser. A. DOI 10.1007/s10107-009-0306-5.

Digital Library

[38]

Nesterov, Y. 1983. A method of solving a convex programming problem with convergence rate O(1/k<sup>2</sup>). Soviet Math. Dokl. 27, 2, 372--376.

[39]

Nesterov, Y. 2005. Smooth minimization of non-smooth functions. Math. Prog. 103, 1.

Digital Library

[40]

Nesterov, Y. 2007. Gradient methods for minimizing composite objective functions. Tech. rep. - CORE - Universite Catholique de Louvain.

[41]

Netflix, Inc. The Netflix prize. http://www.netflixprize.com/.

[42]

Osher, S., Burger, M., Goldfarb, D., Xu, J., and Yin, W. 2005. An iterative regularization method for total variation-based image restoration. Multi. Model. Simul. 4, 460--489.

[43]

Papadimitriou, C., Raghavan, P., Tamaki, H., and Vempala, S. 2000. Latent semantic indexing, a probabilistic analysis. J. Comput. Syst. Sci. 61, 2, 217--235.

Digital Library

[44]

Recht, B. 2009. A simpler approach to matrix completion. CoRR abs/0910.0651.

[45]

Recht, B., Fazel, M., and Parillo, P. 2010. Guaranteed minimum rank solution of matrix equations via nuclear norm minimization. SIAM Rev. 52, 471--501.

Digital Library

[46]

Stauffer, C., and Grimson, E. 1999. Adaptive background mixture models for real-time tracking. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.

[47]

Tenenbaum, J., de Silva, V., and Langford, J. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500, 2319--2323.

[48]

Toh, K. C., and Yun, S. 2010. An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6, 615--640.

[49]

Torre, F. D. L., and Black, M. 2003. A framework for robust subspace learning. Int. J. Comput. Vis. 54, 117--142.

Digital Library

[50]

Vershynin, R. 2011. Introduction to the non-asymptotic analysis of random matrices. http://www-personal.umich.edu/&tilde;romanv/papers/non-asymptotic-rmt-plain.pdf.

[51]

Yin, W., Hale, E., and Zhang, Y. 2008a. Fixed-point continuation for &ell;<sup>1</sup>-minimization: Methodology and convergence. SIAM J. Optimiz. 19, 3, 1107--1130.

Digital Library

[52]

Yin, W., Osher, S., Goldfarb, D., and Darbon, J. 2008b. Bregman iterative algorithms for &ell;₁-minimization with applications to compressed sensing. SIAM J. Imag. Sci. 1, 1, 143--168.

Digital Library

[53]

Yuan, X., and Yang, J. 2009. Sparse and low-rank matrix decomposition via alternating direction methods. http://www.optimization-online.org/08_HTML/2009/11/2447.html.

[54]

Zhou, Z., Wagner, A., Mobahi, H., Wright, J., and Ma, Y. 2009. Face recognition with contiguous occlusion using Markov random fields. In Proceedings of the International Conference on Computer Vision (ICCV).

Cited By

Theodoridis S(2026)Dimensionality reduction and latent variable modelingMachine Learning10.1016/B978-0-44-329238-5.00026-3(1103-1179)Online publication date: 2026
https://doi.org/10.1016/B978-0-44-329238-5.00026-3
Liu PZhai YZhu HYe ZHe QTang ZTang P(2025)An Underwater Polarization Imaging Technique Based on the Construction and Decomposition of the Low-Rank and Sparse Matrix in Stokes Space for Polarization State ImagingSensors10.3390/s2503070425:3(704)Online publication date: 24-Jan-2025
https://doi.org/10.3390/s25030704
Zou SWang XYuan TZeng KLi GXie X(2025)Underwater moving target detection using online robust principal component analysis and multimodal anomaly detectionThe Journal of the Acoustical Society of America10.1121/10.0034831157:1(122-136)Online publication date: 10-Jan-2025
https://doi.org/10.1121/10.0034831
Show More Cited By

Index Terms

Robust principal component analysis?
1. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Regression analysis
        Robust regression

Recommendations

Robust principal component analysis via capped norms
KDD '13: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining

In many applications such as image and video processing, the data matrix often possesses simultaneously a low-rank structure capturing the global information and a sparse component capturing the local information. How to accurately extract the low-rank ...
Robust Q-mode principal component analysis in L1

We propose principal component analysis (PCA) of a data set based on the L"1-norm. We distinguish between Q-mode and R-mode analyses. The Q-mode L"1 principal components are sequentially calculated by an enumeration procedure. We show that the Q-mode L"...
Robust block tensor principal component analysis
Highlights
- The robust block tensor principle component analysis is proposed to extract the low-rank and sparse components in block tensors for a good analysis scale.
Abstract
Robust tensor principal component analysis based on tensor singular value decomposition (t-SVD) is a very effective tool to extract the low rank and sparse components in multi-way signals. In this paper, instead of the tensor nuclear ...

Reviews

Reviewer: Jan De Beule

Consider a matrix M , such that M = L 0 + S 0, where L 0 is supposed to have low rank, and S 0 is supposed to be sparse. An interesting question, also motivated by numerous applications described in the paper, is whether one can efficiently reconstruct L 0 and S 0 if only M is given. The authors describe a procedure, called principal component pursuit (PCP), that recovers L 0 and S 0 exactly, under certain assumptions. One theorem clearly states the procedure, together with its necessary assumptions. The paper contains not only a mathematical description of PCP, but also a section describing numerical experiments using real-life data. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image Journal of the ACM

Journal of the ACM Volume 58, Issue 3

May 2011

122 pages

ISSN:0004-5411

EISSN:1557-735X

DOI:10.1145/1970392

Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2011

Accepted: 01 February 2011

Revised: 01 February 2011

Received: 01 January 2010

Published in JACM Volume 58, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4,773
Total Citations
View Citations
23,253
Total Downloads

Downloads (Last 12 months)1,454
Downloads (Last 6 weeks)139

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Theodoridis S(2026)Dimensionality reduction and latent variable modelingMachine Learning10.1016/B978-0-44-329238-5.00026-3(1103-1179)Online publication date: 2026
https://doi.org/10.1016/B978-0-44-329238-5.00026-3
Liu PZhai YZhu HYe ZHe QTang ZTang P(2025)An Underwater Polarization Imaging Technique Based on the Construction and Decomposition of the Low-Rank and Sparse Matrix in Stokes Space for Polarization State ImagingSensors10.3390/s2503070425:3(704)Online publication date: 24-Jan-2025
https://doi.org/10.3390/s25030704
Zou SWang XYuan TZeng KLi GXie X(2025)Underwater moving target detection using online robust principal component analysis and multimodal anomaly detectionThe Journal of the Acoustical Society of America10.1121/10.0034831157:1(122-136)Online publication date: 10-Jan-2025
https://doi.org/10.1121/10.0034831
Kou ZWang JJia YLiu BGeng X(2025)Instance-Dependent Inaccurate Label Distribution LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332987036:1(1425-1437)Online publication date: Jan-2025
https://doi.org/10.1109/TNNLS.2023.3329870
He ZMa JYuan X(2025)Improved Turbo Message Passing for Compressive Robust Principal Component Analysis: Algorithm Design and Asymptotic AnalysisIEEE Transactions on Information Theory10.1109/TIT.2024.350947671:2(1323-1361)Online publication date: Feb-2025
https://doi.org/10.1109/TIT.2024.3509476
Luo WLee YJian XHao T(2025)A New Method for GPR Clutter Suppression Based on Stationary Graph Signals ProcessingIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.351504163(1-12)Online publication date: 2025
https://doi.org/10.1109/TGRS.2024.3515041
Ghosh MHalder RChandra J(2025)SpaTeD: Sparsity-Aware Tensor Decomposition-Based Representation Learning Framework for Phishing Scams DetectionIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.346255212:1(320-334)Online publication date: Feb-2025
https://doi.org/10.1109/TCSS.2024.3462552
Mesquita A(2025)Robust and Bayesian Subspace IdentificationIEEE Transactions on Automatic Control10.1109/TAC.2024.346556070:2(1395-1401)Online publication date: Feb-2025
https://doi.org/10.1109/TAC.2024.3465560
Zhang XYang GSu HChen YXue ZDu Q(2025)Spectral-Spatial Ensemble Low-Rank Domain Adaptation for Hyperspectral Image ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.350225318(1329-1344)Online publication date: 2025
https://doi.org/10.1109/JSTARS.2024.3502253
Lu DAn WWang HLing QCao DLi MLin Z(2025)Infrared Moving Small Target Detection Based on Spatial–Temporal Feature Fusion Tensor ModelIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing10.1109/JSTARS.2024.349122118(78-99)Online publication date: 2025
https://doi.org/10.1109/JSTARS.2024.3491221
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents