research-article

Dual-graph regularized concept factorization for multi-view clustering

Authors:

Shaokai LiAuthors Info & Claims

Volume 223, Issue C

https://doi.org/10.1016/j.eswa.2023.119949

Published: 01 August 2023 Publication History

Abstract

Matrix factorization is an important technology that obtains the latent representation of data by mining the potential structure of data. As two popular matrix factorization techniques, concept factorization (CF) and non-negative matrix factorization (NMF) have achieved excellent results in multi-view clustering tasks. Compared with multi-view NMF, multi-view CF not only removes the non-negative constraint but also utilizes the idea of the kernel to learn the latent representation of data. However, both of them ignore the local geometric structure in the nonlinear low-dimensional manifold. Furthermore, most of the existing CF-based methods are designed for single-view tasks, which cannot be directly applied to multi-view clustering tasks. To tackle the above shortcomings, we present a new multi-view clustering algorithm, called dual-graph regularized concept factorization for multi-view clustering (MVDGCF). Specifically, we first extend conventional single-view CF to a multi-view version, which can explore the complementary information of multi-view data more effectively. Then we develop a novel dual-graph regularization strategy, which can simultaneously capture the local structure information of the data space and feature space, respectively. Moreover, an adaptive weight vector is introduced to balance the importance of different views. Finally, extensive experiments are carried out on seven datasets. The results show that our method is superior to several popular multi-view clustering methods.

Highlights

•

We present a new multi-view concept factorization (CF) method for clustering.

•

We elegantly extend the single-view CF to a multi-view version.

•

We design a dual-graph strategy to mine the structural information of data.

•

We develop an efficient optimization algorithm.

References

[1]

Baek S., Yoon G., Song J., Yoon S.M., Deep self-representative subspace clustering network, Pattern Recognition 118 (2021).

[2]

Belkin M., Niyogi P., Laplacian eigenmaps and spectral techniques for embedding and clustering, Advances in Neural Information Processing Systems 14 (2001).

[3]

Belkin M., Niyogi P., Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation 15 (6) (2003) 1373–1396.

Digital Library

[4]

Belkin M., Niyogi P., Sindhwani V., Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, Journal of Machine Learning Research 7 (11) (2006).

[5]

Bickel S., Scheffer T., Multi-view clustering, in: ICDM (vol. 4), Citeseer, 2004, pp. 19–26.

[6]

Cai D., Litekmeans: the fastest matlab implementation of kmeans, 2011, Software Available at: http://www.zjucadcg.cn/dengcai/Data/Clustering.html.

[7]

Cai D., He X., Han J., Locally consistent concept factorization for document clustering, IEEE Transactions on Knowledge and Data Engineering 23 (6) (2010) 902–913.

[8]

Cai D., He X., Han J., Huang T.S., Graph regularized nonnegative matrix factorization for data representation, IEEE Transactions on Pattern Analysis and Machine Intelligence 33 (8) (2010) 1548–1560.

Digital Library

[9]

Chang S., Hu J., Li T., Wang H., Peng B., Multi-view clustering via deep concept factorization, Knowledge-Based Systems 217 (2021).

[10]

Chaudhuri, K., Kakade, S. M., Livescu, K., & Sridharan, K. (2009). Multi-view clustering via canonical correlation analysis. In Proceedings of the 26th annual international conference on machine learning (pp. 129–136).

[11]

Chen M.-S., Lin J.-Q., Li X.-L., Liu B.-Y., Wang C.-D., Huang D., Lai J.-H., Representation learning in multi-view clustering: A literature review, Data Science and Engineering (2022) 1–17.

[12]

Chen Y., Wang S., Zheng F., Cen Y., Graph-regularized least squares regression for multi-view subspace clustering, Knowledge-Based Systems 194 (2020).

[13]

Fu L., Lin P., Vasilakos A.V., Wang S., An overview of recent multi-view clustering, Neurocomputing 402 (2020) 148–161.

[14]

Gu, Q., & Zhou, J. (2009). Co-clustering on manifolds. In Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 359–368).

[15]

Han Y., Wu F., Tao D., Shao J., Zhuang Y., Jiang J., Sparse unsupervised dimensionality reduction for multiple view data, IEEE Transactions on Circuits and Systems for Video Technology 22 (10) (2012) 1485–1496.

[16]

He Y., Lu H., Huang L., Xie S., Pairwise constrained concept factorization for data representation, Neural Networks 52 (2014) 1–17.

[17]

Ji P., Zhang T., Li H., Salzmann M., Reid I., Deep subspace clustering networks, Advances in Neural Information Processing Systems 30 (2017).

[18]

Jillani R., Hussain S.F., Kalva H., Multi-view clustering for fast intra mode decision in HEVC, in: 2020 IEEE international conference on consumer electronics, IEEE, 2020, pp. 1–4.

[19]

Korn F., Pagel B.-U., Faloutsos C., On the” dimensionality curse” and the” self-similarity blessing”, IEEE Transactions on Knowledge and Data Engineering 13 (1) (2001) 96–111.

[20]

Kumar A., Daumé H., A co-training approach for multi-view spectral clustering, in: Proceedings of the 28th international conference on machine learning, Citeseer, 2011, pp. 393–400.

[21]

Lee D.D., Seung H.S., Learning the parts of objects by non-negative matrix factorization, Nature 401 (6755) (1999) 788–791.

[22]

Lee D.D., Seung H.S., Learning the parts of objects by non-negative matrix factorization, Nature 401 (6755) (1999) 788–791.

[23]

Li P., Bu J., Chen C., He Z., Cai D., Relational multimanifold coclustering, IEEE Transactions on Cybernetics 43 (6) (2013) 1871–1881.

[24]

Li P., Bu J., Yang Y., Ji R., Chen C., Cai D., Discriminative orthogonal nonnegative matrix factorization with flexibility for data representation, Expert Systems with Applications 41 (4) (2014) 1283–1293.

[25]

Li P., Chen C., Bu J., Clustering analysis using manifold kernel concept factorization, Neurocomputing 87 (2012) 120–131.

[26]

Li N., Leng C., Cheng I., Basu A., Jiao L., Dual-graph global and local concept factorization for data clustering, IEEE Transactions on Neural Networks and Learning Systems (2022).

[27]

Li J., Zhou G., Qiu Y., Wang Y., Zhang Y., Xie S., Deep graph regularized non-negative matrix factorization for multi-view clustering, Neurocomputing 390 (2020) 108–116.

[28]

Liang N., Yang Z., Li Z., Xie S., Su C.-Y., Semi-supervised multi-view clustering with graph-regularized partially shared non-negative matrix factorization, Knowledge-Based Systems 190 (2020).

Digital Library

[29]

Liu, J., Liu, X., Yang, Y., Liu, L., Wang, S., Liang, W., & Shi, J. (2021). One-pass multi-view clustering for large-scale data. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 12344–12353).

[30]

Liu X., Song P., Sheng C., Zhang W., Robust multi-view non-negative matrix factorization for clustering, Digital Signal Processing 123 (2022).

[31]

Liu J., Wang C., Gao J., Han J., Multi-view clustering via joint nonnegative matrix factorization, Proceedings of the 2013 SIAM international conference on data mining, SIAM, 2013, pp. 252–260.

[32]

Liu M., Yang Z., Li L., Li Z., Xie S., Auto-weighted collective matrix factorization with graph dual regularization for multi-view clustering, Knowledge-Based Systems (2022).

[33]

Liu, H., Yang, Z., & Wu, Z. (2011). Locality-constrained concept factorization. In Twenty-second international joint conference on artificial intelligence (pp. 1378–1383).

[34]

Liu H., Yang G., Wu Z., Cai D., Constrained concept factorization for image representation, IEEE Transactions on Cybernetics 44 (7) (2013) 1214–1224.

[35]

Lu M., Zhang L., Li F., Adaptively local consistent concept factorization for multi-view clustering, Soft Computing 26 (3) (2022) 1043–1055.

[36]

Nie, F., Cai, G., & Li, X. (2017). Multi-view clustering and semi-supervised classification with adaptive neighbours. In Thirty-First AAAI conference on artificial intelligence (pp. 2408–2414).

[37]

Ou W., Long F., Tan Y., Yu S., Wang P., Co-regularized multiview nonnegative matrix factorization with correlation constraint for representation learning, Multimedia Tools and Applications 77 (10) (2018) 12955–12978.

[38]

Roweis S.T., Saul L.K., Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326.

[39]

Seung D., Lee L., Algorithms for non-negative matrix factorization, Advances in Neural Information Processing Systems 13 (2001) 556–562.

[40]

Sha F., Lin Y., Saul L.K., Lee D.D., Multiplicative updates for nonnegative quadratic programming, Neural Computation 19 (8) (2007) 2004–2031.

[41]

Shang F., Jiao L., Wang F., Graph dual regularization non-negative matrix factorization for co-clustering, Pattern Recognition 45 (6) (2012) 2237–2250.

Digital Library

[42]

Sun, M., Zhang, P., Wang, S., Zhou, S., Tu, W., Liu, X., Zhu, E., & Wang, C. (2021). Scalable multi-view subspace clustering with unified anchors. In Proceedings of the 29th ACM international conference on multimedia (pp. 3528–3536).

[43]

Tan J., Shi Y., Yang Z., Wen C., Lin L., Unsupervised multi-view clustering by squeezing hybrid knowledge from cross view and each view, IEEE Transactions on Multimedia 23 (2020) 2943–2956.

[44]

Tenenbaum J.B., Silva V.d., Langford J.C., A global geometric framework for nonlinear dimensionality reduction, Science 290 (5500) (2000) 2319–2323.

[45]

Wang D., Han S., Wang Q., He L., Tian Y., Gao X., Pseudo-label guided collective matrix factorization for multiview clustering, IEEE Transactions on Cybernetics (2021).

[46]

Wang Z., Kong X., Fu H., Li M., Zhang Y., Feature extraction via multi-view non-negative matrix factorization with local graph regularization, in: 2015 IEEE international conference on image processing, IEEE, 2015, pp. 3500–3504.

[47]

Wang D., Li T., Deng P., Wang H., Zhang P., Dual graph-regularized sparse concept factorization for clustering, Information Sciences 607 (2022) 1074–1088.

Digital Library

[48]

Wang R., Nie F., Wang Z., Hu H., Li X., Parameter-free weighted multi-view projected clustering with structured graph learning, IEEE Transactions on Knowledge and Data Engineering 32 (10) (2019) 2014–2025.

[49]

Wang H., Yang Y., Li T., Multi-view clustering via concept factorization with local manifold regularization, in: 2016 IEEE 16th international conference on data mining, IEEE, 2016, pp. 1245–1250.

[50]

Wang H., Yang Y., Liu B., GMC: Graph-based multi-view clustering, IEEE Transactions on Knowledge and Data Engineering 32 (6) (2019) 1116–1129.

[51]

Wang H., Yang Y., Liu B., Fujita H., A study of graph-based system for multi-view clustering, Knowledge-Based Systems 163 (2019) 1009–1019.

[52]

Wang H., Yang Y., Zhang X., Peng B., Parallel multi-view concept clustering in distributed computing, Neural Computing and Applications 32 (10) (2020) 5621–5631.

[53]

Wen J., Yan K., Zhang Z., Xu Y., Wang J., Fei L., Zhang B., Adaptive graph completion based incomplete multi-view clustering, IEEE Transactions on Multimedia 23 (2020) 2493–2504.

[54]

Wen J., Zhang B., Xu Y., Yang J., Han N., Adaptive weighted nonnegative low-rank representation, Pattern Recognition 81 (2018) 326–340.

[55]

Xia T., Tao D., Mei T., Zhang Y., Multiview spectral embedding, IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 40 (6) (2010) 1438–1446.

Digital Library

[56]

Xin X., Wang J., Xie R., Zhou S., Huang W., Zheng N., Semi-supervised person re-identification using multi-view clustering, Pattern Recognition 88 (2019) 285–297.

Digital Library

[57]

Xu, W., & Gong, Y. (2004). Document clustering by concept factorization. In Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval (pp. 202–209).

[58]

Xu, W., Liu, X., & Gong, Y. (2003). Document clustering based on non-negative matrix factorization. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval (pp. 267–273).

[59]

Xu C., Tao D., Xu C., A survey on multi-view learning, 2013, arXiv preprint arXiv:1304.5634.

[60]

Yan K., Fang X., Xu Y., Liu B., Protein fold recognition based on multi-view modeling, Bioinformatics 35 (17) (2019) 2982–2990.

[61]

Yaslan Y., Cataltepe Z., Co-training with relevant random subspaces, Neurocomputing 73 (10–12) (2010) 1652–1661.

[62]

Ye J., Jin Z., Dual-graph regularized concept factorization for clustering, Neurocomputing 138 (2014) 120–130.

[63]

Ye J., Jin Z., Graph-regularized local coordinate concept factorization for image representation, Neural Processing Letters 46 (2017) 427–449.

[64]

Yu J., Hong R., Wang M., You J., Image clustering based on sparse patch alignment framework, Pattern Recognition 47 (11) (2014) 3512–3519.

[65]

Yu H., Zhang T., Chen J., Guo C., Lian Y., Web items recommendation based on multi-view clustering, in: 2018 IEEE 42nd annual computer software and applications conference (vol. 1), IEEE, 2018, pp. 420–425.

[66]

Zhan K., Shi J., Wang J., Tian F., Graph-regularized concept factorization for multi-view document clustering, Journal of Visual Communication and Image Representation 48 (2017) 411–418.

[67]

Zhan K., Shi J., Wang J., Wang H., Xie Y., Adaptive structure concept factorization for multiview clustering, Neural Computation 30 (4) (2018) 1080–1103.

[68]

Zhang Z., Wang J., Zha H., Adaptive manifold learning, IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (2) (2011) 253–265.

[69]

Zhang Y.-F., Xu C., Lu H., Huang Y.-M., Character identification in feature-length films using global face-name matching, IEEE Transactions on Multimedia 11 (7) (2009) 1276–1288.

[70]

Zhang Z., Zhang Y., Li S., Liu G., Zeng D., Yan S., Wang M., Flexible auto-weighted local-coordinate concept factorization: A robust framework for unsupervised clustering, IEEE Transactions on Knowledge and Data Engineering (2019).

[71]

Zhang Y., Zhang Z., Wang Y., Zhang Z., Zhang L., Yan S., Wang M., Dual-constrained deep semi-supervised coupled factorization network with enriched prior, International Journal of Computer Vision 129 (12) (2021) 3233–3254.

[72]

Zhang Z., Zhang Y., Xu M., Zhang L., Yang Y., Yan S., A survey on concept factorization: From shallow to deep representation learning, Information Processing & Management 58 (3) (2021).

Digital Library

[73]

Zong L., Zhang X., Zhao L., Yu H., Zhao Q., Multi-view clustering via multi-manifold regularized non-negative matrix factorization, Neural Networks 88 (2017) 74–89.

Cited By

Haris MYusoff YMohd Zain AKhattak AHussain S(2024)Breaking down multi-view clusteringEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107857132:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107857
Hu XXiong DChai L(2024)Robust multi-view clustering via structure regularization concept factorizationDigital Signal Processing10.1016/j.dsp.2024.104713155:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104713
Hu XXiong DChai L(2024)Robust sparse concept factorization with graph regularization for subspace learningDigital Signal Processing10.1016/j.dsp.2024.104527150:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104527
Show More Cited By

Index Terms

Dual-graph regularized concept factorization for multi-view clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

Robust Dual-Graph Regularized Deep Matrix Factorization for Multi-view Clustering
Abstract
The matrix factorization approaches have been widely applied for multi-view clustering since they can effectively explore complementary information contained in the multi-view data. However, some prior knowledge hidden in multi-view data cannot be ...
Adaptively local consistent concept factorization for multi-view clustering
Abstract
Many real-world datasets consist of multiple views of data items. The rough method of combining multiple views directly through feature concatenation cannot uncover the optimal latent structure shared by multiple views, which would benefit many ...
Fast multi-view clustering via correntropy-based orthogonal concept factorization
Abstract
Owing to its ability to handle negative data and promising clustering performance, concept factorization (CF), an improved version of non-negative matrix factorization, has been incorporated into multi-view clustering recently. Nevertheless, ...

Comments

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 223, Issue C

Aug 2023

1341 pages

ISSN:0957-4174

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 August 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Haris MYusoff YMohd Zain AKhattak AHussain S(2024)Breaking down multi-view clusteringEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107857132:COnline publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107857
Hu XXiong DChai L(2024)Robust multi-view clustering via structure regularization concept factorizationDigital Signal Processing10.1016/j.dsp.2024.104713155:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104713
Hu XXiong DChai L(2024)Robust sparse concept factorization with graph regularization for subspace learningDigital Signal Processing10.1016/j.dsp.2024.104527150:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.dsp.2024.104527
Hu XXiong DChai L(2024)Concept factorization with adaptive graph learning on Stiefel manifoldApplied Intelligence10.1007/s10489-024-05606-854:17-18(8224-8240)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10489-024-05606-8

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents