research-article

Incomplete multi-view clustering via kernelized graph learning

Authors:

Tianrui LiAuthors Info & Claims

Volume 625, Issue C

Pages 1 - 19

https://doi.org/10.1016/j.ins.2023.01.013

Published: 01 May 2023 Publication History

Highlights

•

Our algorithm works on incomplete multi-view data with arbitrary missing patterns.

•

Global structure and nonlinear relationship are learned by kernelized self-expression.

•

Coefficients of kernel combination and weights of view are learned automatically.

•

Our model integrates the related subtasks into a unified framework.

Abstract

A fundamental assumption underpinning the recent progress in multi-view clustering is the full observation of all views, which rarely holds for real-world data as they often suffer from the absence of some instances in individual views. Such incompleteness generally disables the traditional multi-view clustering models in practical applications. This paper proposes a Kernelized Graph-based Incomplete Multi-view Clustering (KGIMC) algorithm to overcome this limitation. The key novelty of our model is that its subtasks, e.g. similarity learning, clustering analysis, and kernel completion, are optimized in a mutual reinforcement manner to achieve an overall optimal clustering result as follows: 1) Similarity learning is directed by clustering analysis to construct a graph with as many connected components just as the number of clusters. 2) The well-constructed similarity graph in the last iteration is employed to guide the process of kernel completion. 3) The updated kernels are in turn used to conduct subsequent similarity learning. We extend our model into multi-kernel settings to alleviate the difficulty of kernel selection. We provide an alternating iterative algorithm to solve KGIMC with convergence guaranteed and complexity analyzed. Extensive experiments are conducted on several popular datasets, and the results demonstrate that KGIMC outperforms the state-of-the-art approaches in general.

References

[1]

Y. Yang, H. Wang, Multi-view Clustering: A Survey, Big Data Mining Anal. 1 (2018) 83–107.

[2]

J. Zhao, X. Xie, X. Xu, S. Sun, Multi-view learning overview: Recent progress and new challenges, Inform Fusion 38 (2017) 43–54.

Digital Library

[3]

A. Saxena, M. Prasad, A. Gupta, N. Bharill, O.P. Patel, A. Tiwari, M.J. Er, W. Ding, C.-T. Lin, A review of clustering techniques and developments, Neurocomputing 267 (2017) 664–681.

Digital Library

[4]

X. Zhang, H. Gao, G. Li, J. Zhao, J. Huo, J. Yin, Y. Liu, L. Zheng, Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition, Inform Sci. 432 (2018) 463–478.

[5]

S. Huang, Z. Xu, I.W. Tsang, Z. Kang, Auto-weighted Multi-view Co-clustering with Bipartite Graphs, Inform Sci. 512 (2020) 18–30.

Digital Library

[6]

H. Wang, Y. Yang, B. Liu, GMC: Graph-Based Multi-View Clustering, IEEE Trans. Knowl. Data En. 32 (2020) 1116–1129.

[7]

S. Bhadra, S. Kaski, J. Rousu, Multi-view kernel completion, Mach. Learn. 106 (2017) 713–739.

[8]

W. Shao, X. Shi, P.S. Yu, Clustering on Multiple Incomplete Datasets via Collective Kernel Learning, in: Proceedings of IEEE International Conference on Data Mining, 2013, pp. 1181-1186.

[9]

W. Zhou, H. Wang, Y. Yang, Consensus Graph Learning for Incomplete Multi-view Clustering, in: Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2019, pp. 529-540.

[10]

H. Wang, L. Zong, B. Liu, Y. Yang, W. Zhou, Spectral Perturbation Meets Incomplete Multi-view Data, in: Proceedings of the International Joint Conference on Artificial Intelligence, 2019, pp. 3677-3683.

[11]

J. Wen, Y. Xu, H. Liu, Incomplete Multiview Spectral Clustering With Adaptive Graph Learning, IEEE Trans. Syst. Man Cy-S. 50 (2020) 1418–1429.

[12]

W. Shao, L. He, P.S. Yu, Multiple incomplete views clustering via weighted nonnegative matrix factorization with L_2,1 regularization, in: Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases, 2015, pp. 318-334.

[13]

S. Li, Y. Jiang, Z. Zhou, Partial multi-view clustering, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2014, pp. 1968-1974.

[14]

H. Zhao, H. Liu, Y. Fu, Incomplete multi-modal visual data grouping, in: Proceedings of International Joint Conference On Artificial Intelligence, 2016, pp. 2392-2398.

[15]

J. Ye, J. Guo, Anchors Bring Ease: An Embarrassingly Simple Approach to Partial Multi-view Clustering, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 118-125.

[16]

A. Trivedi, P. Rai, H. Daumé III, S.L. DuVall, Multiview clustering with incomplete views, in: Proceedings of the NIPS Workshop, 2010.

[17]

M. Hu, S. Chen, Doubly Aligned Incomplete Multi-view Clustering, in: Proceedings of the International Joint Conference on Artificial Intelligence, 2018, pp. 2262-2268.

[18]

X. Liu, M. Li, L. Wang, Y. Dou, J. Yin, E. Zhu, Multiple Kernel k-Means with Incomplete Kernels, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2017, pp. 2259-2265.

[19]

X. Zhu, X. Liu, M. Li, E. Zhu, L. Liu, Z. Cai, J. Yin, W. Gao, Localized Incomplete Multiple Kernel k-means, in: Proceedings of the International Joint Conference on Artificial Intelligence, 2018, pp. 3271-3277.

[20]

Z. Kang, Y. Lu, Y. Su, C. Li, Z. Xu, Similarity Learning via Kernel Preserving Embedding, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 4057-4064.

[21]

G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, Robust Recovery of Subspace Structures by Low-Rank Representation, IEEE Trans. Pattern Anal. 35 (2013) 171–184.

[22]

S. Mehrkanoon, J.A.K. Suykens, Regularized Semipaired Kernel CCA for Domain Adaptation, IEEE Trans. Neur. Net. Lear 29 (2018) 3199–3213.

[23]

S. Huang, Z. Kang, I.W. Tsang, Z. Xu, Auto-weighted multi-view clustering via kernelized graph learning, Pattern Recogn. 88 (2019) 174–184.

[24]

F. Nie, L. Tian, X. Li, Multiview Clustering via Adaptively Weighted Procrustes, in: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018, pp. 2022-2030.

[25]

U. Von Luxburg, A tutorial on spectral clustering, Stat. Comput. 17 (2007) 395–416.

Digital Library

[26]

K. Fan, On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: I*, P. Natl. Acad. Sci. USA 35 (1949) 652–655.

[27]

M. Gonen, E. Alpaydin, Multiple Kernel Learning Algorithms, J. Mach. Learn. Res. 12 (2011) 2211–2268.

Digital Library

[28]

H. Zeng, Y. Cheung, Feature Selection and Kernel Learning for Local Learning-Based Clustering, IEEE Trans. Pattern. Anal. 33 (2011) 1532–1547.

[29]

E. Han, N. Ghadimi, Model identification of proton-exchange membrane fuel cells based on a hybrid convolutional neural network and extreme learning machine optimized by improved honey badger algorithm, Sustainable Energy Technol. Assess. 52 (2022).

[30]

W. Jiang, X. Wang, H. Huang, D. Zhang, N. Ghadimi, Optimal economic scheduling of microgrids considering renewable energy sources based on energy hub model using demand response and improved water wave optimization algorithm, J. Storage Mater. 55 (2022).

[31]

Z. Kang, G. Shi, S. Huang, W. Chen, X. Pu, J.T. Zhou, Z. Xu, Multi-graph fusion for multi-view spectral clustering, Knowl.-Based Syst. 189 (2020).

Digital Library

[32]

S.P. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 2014.

Digital Library

[33]

C. Cortes, M. Mohri, A. Rostamizadeh, Algorithms for Learning Kernels Based on Centered Alignment, J. Mach. Learn. Res. 13 (2012) 795–828.

[34]

X. Zhang, Q. You, Clusterability Analysis and Incremental Sampling for Nyström Extension Based Spectral Clustering, in: Proceedings of the IEEE International Conference on Data Mining, 2011, pp. 942-951.

[35]

Z. Hu, F. Nie, W. Chang, S. Hao, R. Wang, X. Li, Multi-view spectral clustering via sparse graph learning, Neurocomputing 384 (2020) 1–10.

Digital Library

[36]

D. Greene, P. Cunningham, A Matrix Factorization Approach for Integrating Multiple Data Views, in: Proceedings of European Conference On Machine Learning, 2009, pp. 423-438.

[37]

K. Lang, NewsWeeder: learning to filter netnews, in: Proceedings of International Conference On Machine Learning, 1995, pp. 331-339.

[38]

F.S. Samaria, A.C. Harter, Parameterisation of a stochastic model for human face identification, in: Proceedings of the IEEE Workshop on Applications of Computer Vision, 1994, pp. 138-142.

[39]

M.F. Duarte, Y.H. Hu, Vehicle classification in distributed sensor networks, J. Parallel Distributed Comput. 64 (2004) 826–838.

[40]

X. Cai, F. Nie, H. Huang, Multi-view K-means clustering on big data, in: Proceedings of the International Joint Conference On Artificial Intelligence, 2013, pp. 2598-2604.

[41]

A.Y. Ng, M.I. Jordan, Y. Weiss, On Spectral Clustering: Analysis and an algorithm, in: Proceedings of the Advances in Neural Information Processing Systems, 2001, pp. 849-856.

[42]

X. Liu, M. Li, C. Tang, J. Xia, J. Xiong, L. Liu, M. Kloft, E. Zhu, Efficient and Effective Regularized Incomplete Multi-view Clustering, IEEE Trans. Pattern Anal. 43 (2020) 2634–2646.

[43]

Y. Lin, Y. Gou, Z. Liu, B. Li, J. Lv, X. Peng, COMPLETER: Incomplete Multi-View Clustering via Contrastive Prediction, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 11174-11183.

[44]

E. Amigó, J. Gonzalo, J. Artiles, F. Verdejo, A comparison of extrinsic clustering evaluation metrics based on formal constraints, Inf. Retr. 12 (2009) 461–486.

Digital Library

[45]

V.D.M. Laurens, G. Hinton, Visualizing Data using t-SNE, J. Mach. Learn. Res. 9 (2008) 2579–2605.

Cited By

Tu WGuan RZhou SMa CPeng XCai ZLiu ZCheng JLiu XWooldridge MDy JNatarajan S(2024)Attribute-missing graph clustering networkProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i14.29464(15392-15401)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i14.29464
Liu SZhang JWen YYang XWang SZhang YZhu ETang CZhao LLiu XWooldridge MDy JNatarajan S(2024)Sample-level cross-view similarity learning for incomplete multi-view clusteringProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i12.29310(14017-14025)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i12.29310
Zhou LDu GLü KWang LDu J(2024)A Survey and an Empirical Evaluation of Multi-View Clustering ApproachesACM Computing Surveys10.1145/364510856:7(1-38)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1145/3645108
Show More Cited By

Index Terms

Incomplete multi-view clustering via kernelized graph learning
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
2. Information systems
  1. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

Consensus Graph Learning for Incomplete Multi-view Clustering
Advances in Knowledge Discovery and Data Mining
Abstract
Multi-view data clustering is a fundamental task in current machine learning, known as multi-view clustering. Existing multi-view clustering methods mostly assume that each data instance is sampled in all views. However, in real-world applications,...
Balance guided incomplete multi-view spectral clustering
Abstract
There is a large volume of incomplete multi-view data in the real-world. How to partition these incomplete multi-view data is an urgent realistic problem since almost all of the conventional multi-view clustering methods are inapplicable to cases ...
Incomplete Multi-View Clustering with Regularized Hierarchical Graph
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

In this article, we propose a novel and effective incomplete multi-view clustering (IMVC) framework, referred to as incomplete multi-view clustering with regularized hierarchical graph (IMVC_RHG). Different from the existing graph learning-based IMVC ...

Comments

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 625, Issue C

May 2023

780 pages

ISSN:0020-0255

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 May 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tu WGuan RZhou SMa CPeng XCai ZLiu ZCheng JLiu XWooldridge MDy JNatarajan S(2024)Attribute-missing graph clustering networkProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i14.29464(15392-15401)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i14.29464
Liu SZhang JWen YYang XWang SZhang YZhu ETang CZhao LLiu XWooldridge MDy JNatarajan S(2024)Sample-level cross-view similarity learning for incomplete multi-view clusteringProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i12.29310(14017-14025)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i12.29310
Zhou LDu GLü KWang LDu J(2024)A Survey and an Empirical Evaluation of Multi-View Clustering ApproachesACM Computing Surveys10.1145/364510856:7(1-38)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1145/3645108
Khan GKhan JAnwar TAl-Huda ZDiallo BAhmad N(2024)Complementary incomplete weighted concept factorization methods for multi-view clusteringKnowledge and Information Systems10.1007/s10115-024-02197-166:12(7469-7494)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10115-024-02197-1
Cai YZhang ZGhamisi PRasti BLiu XCai Z(2023)Transformer-based contrastive prototypical clustering for multimodal remote sensing dataInformation Sciences: an International Journal10.1016/j.ins.2023.119655649:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.ins.2023.119655

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents