research-article

Robust nonnegative matrix factorization using L21-norm

Authors:

Heng HuangAuthors Info & Claims

CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Pages 673 - 682

https://doi.org/10.1145/2063576.2063676

Published: 24 October 2011 Publication History

Abstract

Nonnegative matrix factorization (NMF) is widely used in data mining and machine learning fields. However, many data contain noises and outliers. Thus a robust version of NMF is needed. In this paper, we propose a robust formulation of NMF using L21 norm loss function. We also derive a computational algorithm with rigorous convergence analysis. Our robust NMF approach, (1) can handle noises and outliers; (2) provides very efficient and elegant updating rules; (3) incurs almost the same computational cost as standard NMF, thus potentially to be used in more real world application tasks. Experiments on 10 datasets show that the robust NMF provides more faithful basis factors and consistently better clustering results as compared to standard NMF.

References

[1]

J.-P. Brunet, P. Tamayo, T. Golub, and J. Mesirov. Metagenes and molecular pattern discovery using matrix factorization. Proc. Nat'l Academy of Sciences USA, 102(12):4164--4169, 2004.

[2]

D. Cai, X. He, X. Wu, and J. Han. Non-negative matrix factorization on manifold. In ICDM, pages 63--72, 2008.

Digital Library

[3]

M. Cooper and J. Foote. Summarizing video using non-negative similarity matrix factorization. In Proc. IEEE Workshop on Multimedia Signal Processing, pages 25--28, 2002.

[4]

I. Dhillon and S. Sra. Generalized nonnegative matrix approximations with Bregman divergences. In Advances in Neural Information Processing Systems 17, Cambridge, MA, 2005. MIT Press.

[5]

C. Ding, X. He, and H. Simon. On the equivalence of nonnegative matrix factorization and spectral clustering. Proc. SIAM Data Mining Conf, 2005.

[6]

C. Ding, T. Li, and M. Jordan. Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Analysis and Machine Intelligence, 2010. (LBNL Tech Report 60428, 2006).

Digital Library

[7]

C. Ding, T. Li, and W. Peng. Nonnegative matrix factorization and probabilistic latent semantic indexing: Equivalence, chi-square statistic, and a hybrid method. Proc. National Conf. Artificial Intelligence, 2006.

Digital Library

[8]

C. Ding, T. Li, W. Peng, and H. Park. Orthogonal nonnegative matrix tri-factorizations for clustering. In Proceedings of ACM SIGKDD, pages 126--135, 2006.

Digital Library

[9]

C. Ding, D. Zhou, X. He, and H. Zha. R1-pca: Rotational invariant l1-norm principal component analysis for robust subspace factorization (pdf file). Proc. Int'l Conf. Machine Learning (ICML), June 2006.

Digital Library

[10]

A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. PAMI, 23:643--660, 2001.

Digital Library

[11]

Q. Gu and J. Zhou. Co-clustering on manifolds. In KDD, pages 359--368, 2009.

Digital Library

[12]

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pages 2278--2324, 1998.

[13]

D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In NIPS, 2000.

Digital Library

[14]

S. Li, X. Hou, H. Zhang, and Q. Cheng. Learning spatially localized, parts-based representation. In CVPR, pages 207--212, 2001.

[15]

T. Li, C. Ding, and M. I. Jordan. Solving consensus and semi-supervised clustering problems using nonnegative matrix factorization. In ICDM, pages 577--582, 2007.

Digital Library

[16]

F. Nie, C. H. Q. Ding, D. Luo, and H. Huang. Improved minmax cut graph clustering with nonnegative relaxation. In ECML/PKDD, pages 451--466, 2010.

Digital Library

[17]

F. Nie, H. Huang, X. Cai, and C. Ding. Efficient and robust feature selection via joint l2,1-norms minimization. NIPS, 2010.

Digital Library

[18]

F. Nie, D. Xu, I. W. Tsang, and C. Zhang. Spectral embedded clustering. In IJCAI, pages 1181--1186, 2009.

Digital Library

[19]

P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5:111--126, 1994.

[20]

V. P. Pauca, F. Shahnaz, M. Berry, and R. Plemmons. Text mining using non-negative matrix factorization. In Proc. SIAM Int'l conf on Data Mining, pages 452--456, 2004.

[21]

F. Sha, L. K. Saul, and D. D. Lee. Multiplicative updates for nonnegative quadratic programming in support vector machines. In Advances in Neural Information Processing Systems 15. MIT Press, Cambridge, MA, 2003.

[22]

N. Srebro, J. Rennie, and T. Jaakkola. Maximum margin matrix factorization. In Advances in Neural Information Processing Systems, Cambridge, MA, 2005. MIT Press.

[23]

F. Wang, T. Li, and C. Zhang. Semi-supervised clustering via matrix factorization. In SDM, pages 1--12, 2008.

[24]

N. Z. Weixiang Liu and Q. You. Nonnegative matrix factorization and its applications in pattern recognition. Chinese Science Bulletin, 51(1):7--18, 2006.

[25]

Y.-L. Xie, P. Hopke, and P. Paatero. Positive matrix factorization applied to a curve resolution problem. Journal of Chemometrics, 12(6):357--364, 1999.

Cited By

Shen CHe YQin J(2024)Robust Multi-Dimensional Time Series ForecastingEntropy10.3390/e2601009226:1(92)Online publication date: 22-Jan-2024
https://doi.org/10.3390/e26010092
Qian YZou QZhao MLiu YGuo FDing Y(2024)scRNMF: An imputation method for single-cell RNA-seq data by robust and non-negative matrix factorizationPLOS Computational Biology10.1371/journal.pcbi.101233920:8(e1012339)Online publication date: 8-Aug-2024
https://doi.org/10.1371/journal.pcbi.1012339
张钰(2024)Construction of Machine Learning Disease Prediction Model Based on Macro-Genomic AnalysisAdvances in Applied Mathematics10.12677/AAM.2024.13102313:01(199-207)Online publication date: 2024
https://doi.org/10.12677/AAM.2024.131023
Show More Cited By

Index Terms

Robust nonnegative matrix factorization using L21-norm
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
    2. Machine learning approaches
      1. Markov decision processes
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Markov decision processes

Recommendations

Robust Capped Norm Nonnegative Matrix Factorization: Capped Norm NMF
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

As an important matrix factorization model, Nonnegative Matrix Factorization (NMF) has been widely used in information retrieval and data mining research. Standard Nonnegative Matrix Factorization is known to use the Frobenius norm to calculate the ...
Manifold NMF with L21 norm for clustering

Nonnegative matrix factorization has been widely used in data mining and machine learning fields as a clustering algorithm. The standard nonnegative matrix factorization algorithm utilizes the sum of squared error to measure the quality of factorization,...
Elastic nonnegative matrix factorization
Highlights
- ENMF introduces an elastic loss that takes advantage of both Frobenius norm and ℓ2, 1 norm when noise distribution is uncertain, therefore ENMF is far more insensitive to noise and outliers.
- ENMF takes the geometric information of the ...
Abstract
Nonnegative matrix factorization (NMF) plays a vital role in data mining and machine learning fields. Standard NMF utilizes the Frobenius norm while robust NMF uses the robust ℓ_2,1-norm to measure the quality of factorization, given the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

October 2011

2712 pages

ISBN:9781450307178

DOI:10.1145/2063576

Editors:
Bettina Berendt,
Arjen de Vries,
Wenfei Fan,
Craig Macdonald
University of Glasgow, UK
,
Iadh Ounis
University of Glasgow, UK
,
Ian Ruthven
University of Strathclyde, UK

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '11

Sponsor:

CIKM '11: International Conference on Information and Knowledge Management

October 24 - 28, 2011

Glasgow, Scotland, UK

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

196
Total Citations
View Citations
4,377
Total Downloads

Downloads (Last 12 months)392
Downloads (Last 6 weeks)34

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shen CHe YQin J(2024)Robust Multi-Dimensional Time Series ForecastingEntropy10.3390/e2601009226:1(92)Online publication date: 22-Jan-2024
https://doi.org/10.3390/e26010092
Qian YZou QZhao MLiu YGuo FDing Y(2024)scRNMF: An imputation method for single-cell RNA-seq data by robust and non-negative matrix factorizationPLOS Computational Biology10.1371/journal.pcbi.101233920:8(e1012339)Online publication date: 8-Aug-2024
https://doi.org/10.1371/journal.pcbi.1012339
张钰(2024)Construction of Machine Learning Disease Prediction Model Based on Macro-Genomic AnalysisAdvances in Applied Mathematics10.12677/AAM.2024.13102313:01(199-207)Online publication date: 2024
https://doi.org/10.12677/AAM.2024.131023
Saha SImtiaz H(2024)Privacy-Preserving Non-Negative Matrix Factorization with OutliersACM Transactions on Knowledge Discovery from Data10.1145/363296118:3(1-26)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3632961
Kong QSun JZhang YXu Z(2024)Probabilistic Matrix Factorization for Data With Attributes Based on Finite Mixture ModelingIEEE Transactions on Cybernetics10.1109/TCYB.2022.319644454:2(1154-1166)Online publication date: Feb-2024
https://doi.org/10.1109/TCYB.2022.3196444
Wang RSong PLi SJi LZheng W(2024)Common Latent Embedding Space for Cross-Domain Facial Expression RecognitionIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.327699011:2(2046-2056)Online publication date: Apr-2024
https://doi.org/10.1109/TCSS.2023.3276990
Yang XChe HLeung MWen S(2024)Self-paced regularized adaptive multi-view unsupervised feature selectionNeural Networks10.1016/j.neunet.2024.106295175(106295)Online publication date: Jul-2024
https://doi.org/10.1016/j.neunet.2024.106295
Li JKang PSun WJiang Z(2024)Local residual preserving non-negative matrix factorization for multi-view clusteringNeurocomputing10.1016/j.neucom.2024.128054600(128054)Online publication date: Oct-2024
https://doi.org/10.1016/j.neucom.2024.128054
Yu PJia X(2024)Exploiting indirect linear correlation for label distribution learningNeurocomputing10.1016/j.neucom.2024.128022599(128022)Online publication date: Sep-2024
https://doi.org/10.1016/j.neucom.2024.128022
Hu XXiong DChai L(2024)Robust sparse concept factorization with graph regularization for subspace learningDigital Signal Processing10.1016/j.dsp.2024.104527150:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104527
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents