article

Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian

Authors:

Xiao-Hua ShiAuthors Info & Claims

Neural Processing Letters, Volume 42, Issue 1

Pages 167 - 185

https://doi.org/10.1007/s11063-014-9350-0

Published: 01 August 2015 Publication History

Abstract

Non-negative matrix factorization (NMF) is a very effective method for high dimensional data analysis, which has been widely used in information retrieval, computer vision, and pattern recognition. NMF aims to find two non-negative matrices whose product approximates the original matrix well. It can capture the underlying structure of data in the low dimensional data space using its parts-based representations. However, NMF is actually an unsupervised method without making use of prior information of data. In this paper, we propose a novel pairwise constrained non-negative matrix factorization with graph Laplacian method, which not only utilizes the local structure of the data by graph Laplacian, but also incorporates pairwise constraints generated among all labeled data into NMF framework. More specifically, we expect that data points which have the same class label will have very similar representations in the low dimensional space as much as possible, while data points with different class labels will have dissimilar representations as much as possible. Consequently, all data points are represented with more discriminating power in the lower dimensional space. We compare our approach with other typical methods and experimental results for image clustering show that this novel algorithm achieves the state-of-the-art performance.

References

[1]

Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373---1396

Digital Library

[2]

Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548---1560

Digital Library

[3]

Chapelle O, Schölkopf B, Zien A et al (2006) Semi-supervised learning, vol 2. MIT Press, Cambridge

[4]

Das Gupta M, Xiao J (2011) Non-negative matrix factorization as a feature selection tool for maximum margin classifiers. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2841---2848

Digital Library

[5]

Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc Ser B (Methodological) 39:1---38

[6]

Grira N, Crucianu M, Boujemaa N (2005) Semi-supervised fuzzy clustering with pairwise-constrained competitive agglomeration. In: The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ'05. IEEE, pp 867---872

[7]

Guillamet D, Vitria J (2002) Classifying faces with nonnegative matrix factorization. In: Proceedings of 5th Catalan Conference for Artificial Intelligence

[8]

He R, Zheng W, Hu B, Kong X (2011) Nonnegative sparse coding for discriminative semi-supervised learning. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2849---2856

[9]

He Y, Lu H, Xie S (2013) Semi-supervised non-negative matrix factorization for image clustering with graph laplacian. Multim Tools Appl.

[10]

Kim J, Park H (2008) Sparse nonnegative matrix factorization for clustering. CSE Technical Reports, Georgia Institute of Technology, pp 1---16

[11]

Lee D, Seung H (2001) Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst 13:556---562

[12]

Lee D, Seung H et al (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788---791

[13]

Li T, Ding C (2006) The relationships among various nonnegative matrix factorization methods for clustering. In: 6th International Conference on Data Mining, 2006. ICDM'06. IEEE, pp 362---371

[14]

Liu H, Wu Z (2010) Non-negative matrix factorization with constraints. In: 24th AAAI Conference on Artificial Intelligence.

[15]

Liu H, Wu Z, Cai D, Huang T (2011) Constrained non-negative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 99:1---1

[16]

Liu H, Wu Z, Li X, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299---1311

Digital Library

[17]

Lovász L, Plummer M (1986) Matching theory. Elsevier Science Ltd., Amsterdam, p 121

[18]

Niyogi X (2004) Locality preserving projections. In: Advances in neural information processing systems 16: proceedings of the 2003 conference, vol. 16. The MIT Press, p 153

[19]

Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE, pp 1---8

[20]

Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323

[21]

Saul L, Pereira F (1997) Aggregate and mixed-order markov models for statistical language processing. In: Proceedings of the second conference on empirical methods in natural language processing. Association for Computational Linguistics, Somerset, New Jersey, pp 81---89

[22]

Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22nd international conference on Machine learning. ACM, pp 792---799

[23]

Wang F, Li T, Zhang C (2008) Semi-supervised clustering via matrix factorization. In: Proceedings of The 8th SIAM Conference on Data Mining

[24]

Welling M (2005) Fisher linear discriminant analysis. Technical report, vol 3. Department of Computer Science, University of Toronto

[25]

Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1---3):37---52

[26]

Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210---227

Digital Library

[27]

Wu M, Scholkopf B (2007) A local learning approach for clustering. Adv Neural Inf Process Syst 19:1529

[28]

Xu W, Gong Y (2004) Document clustering by concept factorization. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 202---209

[29]

Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval. ACM, pp 267---273

[30]

Yang L, Jin R, Sukthankar R (2008) Semi-supervised learning with weakly-related unlabeled data: Towards better text categorization. In: 22nd annual conference on neural information processing systems, Citeseer.

[31]

Yang Y, Hu B (2007) Pairwise constraints-guided non-negative matrix factorization for document clustering. In: IEEE/WIC/ACM international conference on web intelligence. IEEE, pp 250---256

Digital Library

[32]

Yang Y, Shen HT, Nie F, Ji R, Zhou X (2011) Nonnegative spectral clustering with discriminative regularization. In: AAAI

Digital Library

[33]

Ye J, Zhao Z, Wu M (2007) Discriminative k-means for clustering. Adv Neural Inf Process Syst 20:1649---1656

[34]

Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition? In: 2011 IEEE international conference on Computer Vision (ICCV). IEEE, pp 471---478

Digital Library

[35]

Zhang Y, Yeung D (2008) Semi-supervised discriminant analysis using robust path-based similarity. In: IEEE conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE, pp 1---8

[36]

Zhang Z, Wang J, Zha H (2005) Adaptive manifold learning. IEEE Trans Pattern Anal Mach Intell 99:1---1

[37]

Zhang Z, Zha H, Zhang M (2008) Spectral methods for semi-supervised manifold learning. In: IEEE conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE, pp 1---6

Cited By

Ge ZYang Y(2024)A simple multi-constraint fusion based semi-supervised non-negative matrix decomposition for image clusteringNeurocomputing10.1016/j.neucom.2024.128432609:COnline publication date: 7-Dec-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128432
Li YLiu YWei JZu BWang H(2022)General Community Detection in Attributed Networks with Consistent-Module Constrained Nonnegative Matrix FactorizationWireless Communications & Mobile Computing10.1155/2022/82361572022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/8236157
Lu HZhao QSang XLu J(2020)Community Detection in Complex Networks Using Nonnegative Matrix Factorization and Density-Based Clustering AlgorithmNeural Processing Letters10.1007/s11063-019-10170-151:2(1731-1748)Online publication date: 7-Jan-2020
https://dl.acm.org/doi/10.1007/s11063-019-10170-1
Show More Cited By

Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian

Non-negative matrix factorization (NMF) plays an important role in multivariate data analysis, and has been widely applied in information retrieval, computer vision, and pattern recognition. NMF is an effective method to capture the underlying structure ...
Community Detection in Social Network with Pairwisely Constrained Symmetric Non-Negative Matrix Factorization
ASONAM '15: Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015

Non-negative Matrix Factorization (NMF) aims to find two non-negative matrices whose product approximates the original matrix well, and is widely used in clustering condition with good physical interpretability and universal applicability. Detecting ...
Graph regularized discriminative non-negative matrix factorization for face recognition

Non-negative matrix factorization (NMF) has been widely employed in computer vision and pattern recognition fields since the learned bases can be interpreted as a natural parts-based representation of the input space, which is consistent with the ...

Comments

Information & Contributors

Information

Published In

cover image Neural Processing Letters

Neural Processing Letters Volume 42, Issue 1

August 2015

249 pages

ISSN:1370-4621

Issue’s Table of Contents

Copyright © Copyright © 2015 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 August 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ge ZYang Y(2024)A simple multi-constraint fusion based semi-supervised non-negative matrix decomposition for image clusteringNeurocomputing10.1016/j.neucom.2024.128432609:COnline publication date: 7-Dec-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128432
Li YLiu YWei JZu BWang H(2022)General Community Detection in Attributed Networks with Consistent-Module Constrained Nonnegative Matrix FactorizationWireless Communications & Mobile Computing10.1155/2022/82361572022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/8236157
Lu HZhao QSang XLu J(2020)Community Detection in Complex Networks Using Nonnegative Matrix Factorization and Density-Based Clustering AlgorithmNeural Processing Letters10.1007/s11063-019-10170-151:2(1731-1748)Online publication date: 7-Jan-2020
https://dl.acm.org/doi/10.1007/s11063-019-10170-1
Zhang KZhang L(2018)Supervised Dictionary Learning with Smooth Shrinkage for Image DenoisingNeural Processing Letters10.1007/s11063-017-9665-847:2(535-548)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s11063-017-9665-8

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents