Abstract
Multiple kernel k-means clustering (MKKC) is proposed to efficiently incorporate multiple base kernels to generate an optimal kernel. However, many existing MKKC methods all involve two stages: learning a clustering indicator matrix and performing clustering on it. This cannot ensure the ultimate clustering results are optimal because the optimal values of two steps are not equivalent to those of the original problem. To address this issue, in this paper, we propose a novel method named multiple kernel k-means clustering with block diagonal property (MKKC-BD). It is the first time to find the relationship between an indicator matrix and Laplacian matrix of the graph theory and get a block diagonal (BD) representation of the indicator matrix. By imposing the BD constraint on the indicator matrix, the BD property of the indicator matrix is ensured. Further, the explicit clustering results are generated directly from the unified framework integrating the three processes of learning an optimal kernel, an indicator matrix and clustering results, which shows the clustering task is executed just by one step. In addition, a simple kernel weight strategy is used in this framework to obtain the optimal kernel, where the value of each kernel weight directly reveals the relationship of each base kernel and the optimal kernel. Finally, by extensive experiments on ten data sets and comparison of clustering results with eight state-of-the-art multiple kernel clustering methods, it is concluded that MKKC-BD is effective. Our code is available at https://github.com/mathchen-git/MKKC-BD.
Similar content being viewed by others
Data availability
The datasets used in the experiment are available in the corresponding website at http://featureselection.asu.edu/data sets.php (i.e., AR), https://jundongl.github.io/scikit-feature/datasets.html (i.e., ORL, COIL20, USPS, YALE), https://archive-beta.ics.uci.edu/ml/datasets (i.e., MNIST, MSRA25, SEGMENT), http://www.cs.nyu.edu/ roweis/data.html (i.e.,BA),and http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html (i.e., TR11), where a subset of MNIST is used in our experiment because of the limit of the memory space. This subset is also used in [31].
References
Hartigan JA (1975) Clustering Algorithm. Wiley, New York
MacQueen J (1967) Some methods for classification and analysis of multi-variate observations. In: Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability,Vol 1, pp 281-297
Andrew YN, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14:849–856
Nie FP, Wang XQ, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: SIGKDD, pp 977-986
Girolami MA (2002) Mercer kernel-based clustering in feature space. IEEE Trans Neural Netw 13(3):780–784
Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134
Du L, Zhou P, Shi L et al (2015) Robust multiple kernel \(k\)-means using \(\ell _{2,1}\)-norm. In: IJCAI, pp 3476-3482
Zhou SH, Zhu E, Liu XW et al (2020) Subspace segmentation-based robust multiple kernel clustering. Inf Fus 53:145–154
Lu JT, Lu YH, Wang R, Nie FP et al (2022) Multiple kernel \(k\)-means clustering with simultaneous spectral rotation. In: ICASSP, pp 4143-4147
Liu XW, Zhou SH, Liu L et al (2021) Localized simple multiple kernel \(k\)-means. In: ICCV, pp 9273-9281
Zhu XZ, Liu XW, Li MM et al (2018) Localized incomplete multiple kernel \(k\)-means. In: IJCAI, pp 3271-3277
Liu XW, Dou Y, Yin JP et al (2016) Multiple kernel \(k\)-means clustering with matrix-induced regularization. In: AAAI, pp 1888-1894
Yao YQ, Li Y, Jiang BB et al (2021) Multiple kernel \(k\)-means clustering by selecting representative kernels. IEEE Trans Neural Netw Learn Syst 32(11):4983–4996
Liu XW, Zhou SH, Wang YQ et al (2017) Optimal neighborhood kernel clustering with multiple kernels. In: AAAI, pp 2266-2272
Zhou SH, Liu XW, Li MM et al (2020) Multiple kernel clustering with neighbor-kernel subspace segmentation. IEEE Trans Neural Netw Learn Syst 31(4):1351–1362
Ren ZW, Li HR, Yang C et al (2020) Multiple kernel subspace clustering with local structural graph and low-rank consensus kernel learning. Knowl Based Syst 188:1–9
Ren ZW, Sun QS (2021) Simultaneous global and local graph structure preserving for multiple kernel clustering. IEEE Trans Neural Netw Learn Syst 32(5):1839–1851
Liu JY, Liu XW, Xiong J et al (2022) Optimal neighborhood multiple kernel clustering with adaptive local kernels. IEEE Trans Knowl Data Eng 34(6):2872–2885
Wang R, Lu JT, Lu YH et al (2021) Discrete multiple kernel \(k\)-means. In: IJCAI, pp 3111-3117
Lu CY, Feng JS, Lin ZC et al (2019) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501
Elhamifar E, Vidal R (2013) Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781
Liu GC, Lin ZC, Yan SC et al (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184
Feng JS, Lin ZC, Xu H et al (2014) Robust subspace segmentation with block-diagonal prior. In: CVPR, pp 3818-3825
Kang Z, Lu X, Yi JF et al (2018) Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In: IJCAI, pp 2312-2318
Dattorro J (2010) Convex optimization and euclidean distance geometry http://meboo.convexoptimization.com/Meboo.html
Nie FP, Zhang R, Li XL (2017) A generalized power iteration method for solving quadratic problem on the Stiefel manifold. Sci China Inf Sci 60(11):112101:1-112101:10
Schölkopf B, Smola AJ, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319
Liu XW, Zhu E, Liu JY et al (2020) Simplemkkm: Simple multiple kernel k-means. CoRR, vol. abs/2005.04975 https://arxiv.org/abs/2005.04975
Sun MJ, Wang SW, Zhang P et al (2022) Projective multiple kernel subspace clustering. IEEE Trans Multim 24:2567–2579
Zhan K, Nie FP, Wang J et al (2019) Multiview consensus graph clustering. IEEE Trans Image Process 28(3):1261–1270
Shi ZQ, Liu JL (2023) Noise-tolerant clustering via joint doubly stochastic matrix regularization and dual sparse coding. Expert Syst Appl 222:119814
Acknowledgments
This work is partially supported by the Research Fund of Guangxi Key Lab of Multi-source Information Mining and Security (Grant No: MIMS22-03, MIMS21-M-01), the National Natural Science Foundation of China (Grant No: 61862009).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest or personal relationships related to the work in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, C., Wei, J. & Li, Z. Multiple kernel k-means clustering with block diagonal property. Pattern Anal Applic 26, 1515–1526 (2023). https://doi.org/10.1007/s10044-023-01183-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-023-01183-7