Cited By
View all- Bowen DHaiquan WYuxuan LZhao JMa YRunhe H(2024)Fair and Robust Federated Learning via Decentralized and Adaptive Aggregation based on BlockchainACM Transactions on Sensor Networks10.1145/3673656Online publication date: 17-Jun-2024
Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering
Authors: 
Xiaojia Zhao, Tingting Xu, Qiangqiang Shen, Youfa Liu, Yongyong Chen, Jingyong SuAuthors Info & Claims
Xiaojia Zhao, Tingting Xu, Qiangqiang Shen, Youfa Liu, Yongyong Chen, Jingyong SuAuthors Info & ClaimsArticle No.: 21, Pages 1 - 21
Abstract
Ensemble clustering (EC), utilizing multiple basic partitions (BPs) to yield a robust consensus clustering, has shown promising clustering performance. Nevertheless, most current algorithms suffer from two challenging hurdles: (1) a surge of EC-based methods only focus on pair-wise sample correlation while fully ignoring the high-order correlations of diverse views. (2) they deal directly with the co-association (CA) matrices generated from BPs, which are inevitably corrupted by noise and thus degrade the clustering performance. To address these issues, we propose a novel Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering (DC-RMEC) method, which preserves the high-order inter-view correlation and the high-order correlation of original data simultaneously. Specifically, DC-RMEC constructs a hypergraph from BPs to fuse high-level complementary information from different algorithms and incorporates multiple CA-based representations into a low-rank tensor to discover the high-order relevance underlying CA matrices, such that double high-order correlation of multi-view features could be dexterously uncovered. Moreover, a marginalized denoiser is invoked to gain robust view-specific CA matrices. Furthermore, we develop a unified framework to jointly optimize the representation tensor and the result matrix. An effective iterative optimization algorithm is designed to optimize our DC-RMEC model by resorting to the alternating direction method of multipliers. Extensive experiments on seven real-world multi-view datasets have demonstrated the superiority of DC-RMEC compared with several state-of-the-art multi-view ensemble clustering methods.
References
[1]
Maria Brbić and Ivica Kopriva. 2018. Multi-view low-rank sparse subspace clustering. Pattern Recognition 73 (2018), 247–258.
[2]
J. Douglas Carroll and Jih-Jie Chang. 1970. Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika 35, 3 (1970), 283–319.
[3]
Chih-Fan Chen, Chia-Po Wei, and Yu-Chiang Frank Wang. 2012. Low-rank matrix recovery with structural incoherence for robust face recognition. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2618–2625.
[4]
Minmin Chen, Zhixiang Xu, Kilian Weinberger, and Fei Sha. 2012. Marginalized denoising autoencoders for domain adaptation. International Conference on Machine Learning (2012), 767–774.
[5]
Yongyong Chen, Xiaolin Xiao, Chong Peng, Guangming Lu, and Yicong Zhou. 2022. Low-rank tensor graph learning for multi-view subspace clustering. IEEE Transactions on Circuits and Systems for Video Technology 32, 1 (2022), 92–104.
[6]
Dana Cristofor and Dan A. Simovici. 2002. Finding median partitions using information-theoretical-based genetic algorithms. Journal of Universal Computer Science 8, 2 (2002), 153–172.
[7]
Ehsan Elhamifar and René Vidal. 2013. Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 11 (2013), 2765–2781.
[8]
Xiaozhao Fang, Na Han, Wai Keung Wong, Shaohua Teng, Jigang Wu, Shengli Xie, and Xuelong Li. 2018. Flexible affinity matrix learning for unsupervised and semisupervised classification. IEEE Transactions on Neural Networks and Learning Systems 30, 4 (2018), 1133–1149.
[9]
Li Fei-Fei. 2004. Learning generative visual models from few training examples. In Proceedings of the Workshop on Generative-Model Based Vision, IEEE Proc. CVPR, 2004.
[10]
Xiaoli Zhang Fern and Carla E Brodley. 2004. Solving cluster ensemble problems by bipartite graph partitioning. In Proceedings of the 21st International Conference on Machine Learning. 36.
[11]
Jobin Francis and Sudhish N. George. 2020. A unified tensor framework for clustering and simultaneous reconstruction of incomplete imaging data. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 3 (2020), 1–24.
[12]
Lucas Franek and Xiaoyi Jiang. 2014. Ensemble clustering by means of clustering embedding in vector spaces. Pattern Recognition 47, 2 (2014), 833–842.
[13]
Ana L. N. Fred and Anil K. Jain. 2005. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 6 (2005), 835–850.
[14]
Dong Huang, Jianhuang Lai, and Chang-Dong Wang. 2016. Ensemble clustering using factor graph. Pattern Recognition 50 (2016), 131–142.
[15]
Dong Huang, Jian-Huang Lai, and Chang-Dong Wang. 2015. Robust ensemble clustering using probability trajectories. IEEE Transactions on Knowledge and Data Engineering 28, 5 (2015), 1312–1326.
[16]
Dong Huang, Chang-Dong Wang, and Jian-Huang Lai. 2017. Locally weighted ensemble clustering. IEEE Transactions on Cybernetics 48, 5 (2017), 1460–1473.
[17]
Dong Huang, Chang-Dong Wang, and Jian-Huang Lai. 2023. Fast Multi-view clustering via ensembles: Towards scalability, superiority, and simplicity. IEEE Transactions on Knowledge and Data Engineering (2023).
[18]
Dong Huang, Chang-Dong Wang, Hongxing Peng, Jianhuang Lai, and Chee-Keong Kwoh. 2018. Enhanced ensemble clustering via fast propagation of cluster-wise similarities. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51, 1 (2018), 508–520.
[19]
Natthakan Iam-On, Tossapon Boongoen, Simon Garrett, and Chris Price. 2011. A link-based approach to the cluster ensemble problem. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (2011), 2396–2409.
[20]
Hong Jia and Zhimin Li. 2021. Spectral ensemble clustering with LDA-based co-training for multi-view data analysis. In Proceedings of the 2021 17th International Conference on Computational Intelligence and Security (CIS). IEEE, 367–371.
[21]
Misha E. Kilmer, Karen Braman, Ning Hao, and Randy C. Hoover. 2013. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM Journal on Matrix Analysis and Applications 34, 1 (2013), 148–172.
[22]
Abhishek Kumar, Piyush Rai, and Hal Daume. 2011. Co-regularized multi-view spectral clustering. In Proceedings of the Advances in Neural Information Processing Systems.
[23]
Martin Lades, Jan C. Vorbruggen, Joachim Buhmann, Jörg Lange, Christoph Von Der Malsburg, Rolf P. Wurtz, and Wolfgang Konen. 1993. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42, 3 (1993), 300–311.
[24]
Fabien Lauer and Christoph Schnörr. 2009. Spectral clustering of linear subspaces for motion segmentation. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE, 678–685.
[25]
Hongmin Li, Xiucai Ye, Akira Imakura, and Tetsuya Sakurai. 2020. Ensemble learning for spectral clustering. In Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 1094–1099.
[26]
Yeqing Li, Feiping Nie, Heng Huang, and Junzhou Huang. 2015. Large-scale multi-view spectral clustering via bipartite graph. In Proceedings of the 29th AAAI Conference on Artificial Intelligence.
[27]
Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. 2012. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1 (2012), 171–184.
[28]
Guangcan Liu, Zhouchen Lin, and Yong Yu. 2010. Robust subspace segmentation by low-rank representation. In Proceedings of the 27th International Conference on Machine Learning. 8.
[29]
Hongfu Liu and Yun Fu. 2015. Clustering with partition level side information. In Proceedings of the 2015 IEEE International Conference on Data Mining. IEEE, 877–882.
[30]
Hongfu Liu and Yun Fu. 2018. Consensus guided multi-view clustering. ACM Transactions on Knowledge Discovery from Data 12, 4 (2018), 1–21.
[31]
Hongfu Liu, Tongliang Liu, Junjie Wu, Dacheng Tao, and Yun Fu. 2015. Spectral ensemble clustering. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 715–724.
[32]
Hongfu Liu, Ming Shao, Sheng Li, and Yun Fu. 2016. Infinite ensemble for image clustering. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1745–1754.
[33]
Hongfu Liu, Ming Shao, Sheng Li, and Yun Fu. 2018. Infinite ensemble clustering. Data Mining and Knowledge Discovery 32, 2 (2018), 385–416.
[34]
Hongfu Liu, Junjie Wu, Tongliang Liu, Dacheng Tao, and Yun Fu. 2017. Spectral ensemble clustering via weighted k-means: Theoretical and practical evidence. IEEE Transactions on Knowledge and Data Engineering 29, 5 (2017), 1129–1143.
[35]
Andrew Ng, Michael Jordan, and Yair Weiss. 2001. On spectral clustering: Analysis and an algorithm. In Proceedings of the Advances in Neural Information Processing Systems.
[36]
Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In Proceedings of the International Joint Conference on Artificial Intelligence.1881–1887.
[37]
Dimitrios Rafailidis and Petros Daras. 2012. The TFC model: Tensor factorization and tag clustering for item recommendation in social tagging systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 43, 3 (2012), 673–688.
[38]
Shankar Rao, Roberto Tron, Rene Vidal, and Yi Ma. 2009. Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 10 (2009), 1832–1845.
[39]
Alexander Strehl and Joydeep Ghosh. 2002. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research 3, Dec (2002), 583–617.
[40]
Mengjing Sun, Pei Zhang, Siwei Wang, Sihang Zhou, Wenxuan Tu, Xinwang Liu, En Zhu, and Changjian Wang. 2021. Scalable multi-view subspace clustering with unified anchors. In Proceedings of the ACM International Conference on Multimedia. 3528–3536.
[41]
Panagiotis Symeonidis. 2015. ClustHOSVD: Item recommendation by combining semantically enhanced tag clustering with tensor HOSVD. IEEE Transactions on Systems, Man, and Cybernetics: Systems 46, 9 (2015), 1240–1251.
[42]
Zhiqiang Tao, Jun Li, Huazhu Fu, Yu Kong, and Yun Fu. 2021. From ensemble clustering to subspace clustering: Cluster structure encoding. IEEE Transactions on Neural Networks and Learning Systems 34, 5 (2021), 2670–2681.
[43]
Zhiqiang Tao, Hongfu Liu, Jun Li, Zhaowen Wang, and Yun Fu. 2019. Adversarial graph embedding for ensemble clustering. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization.
[44]
Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2017. From ensemble clustering to multi-view clustering. In Proceedings of the International Joint Conference on Artificial Intelligence.
[45]
Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2019. Marginalized multiview ensemble clustering. IEEE Transactions on Neural Networks and Learning Systems 31, 2 (2019), 600–611.
[46]
Zhiqiang Tao, Hongfu Liu, Sheng Li, Zhengming Ding, and Yun Fu. 2019. Robust spectral ensemble clustering via rank minimization. ACM Transactions on Knowledge Discovery from Data 13, 1 (2019), 1–25.
[47]
Zhiqiang Tao, Hongfu Liu, Sheng Li, and Yun Fu. 2016. Robust spectral ensemble clustering. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management. 367–376.
[48]
Ledyard R Tucker. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 3 (1966), 279–311.
[49]
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, 12 (2010), 3371–3408.
[50]
Hao Wang, Yan Yang, and Bing Liu. 2019. GMC: Graph-based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering 32, 6 (2019), 1116–1129.
[51]
Siwei Wang, Xinwang Liu, Xinzhong Zhu, Pei Zhang, Yi Zhang, Feng Gao, and En Zhu. 2021. Fast parameter-free multi-view subspace clustering with consensus anchor guidance. IEEE Transactions on Image Processing 31 (2021), 556–568.
[52]
Xiaoyu Wang, Tony X. Han, and Shuicheng Yan. 2009. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE, 32–39.
[53]
Yang Wang, Xuemin Lin, Lin Wu, Qing Zhang, and Wenjie Zhang. 2016. Shifting multi-hypergraphs via collaborative probabilistic voting. Knowledge and Information Systems 46, 3 (2016), 515–536.
[54]
Yang Wang, Zhang Wenjie, Lin Wu, Xuemin Lin, Meng Fang, and Shirui Pan. 2016. Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering. In Proceedings of the International Joint Conference on Artificial Intelligence 2016. Association for the Advancement of Artificial Intelligence (AAAI), 2153–2159.
[55]
Yang Wang and Lin Wu. 2018. Beyond low-rank representations: Orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Networks 103 (2018), 1–8.
[56]
Yang Wang, Lin Wu, Xuemin Lin, and Junbin Gao. 2018. Multi-view spectral clustering via structured low-rank matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 29, 10 (2018), 4833–4843.
[57]
Junjie Wu, Hongfu Liu, Hui Xiong, Jie Cao, and Jian Chen. 2014. K-means-based consensus clustering: A unified view. IEEE Transactions on Knowledge and Data Engineering 27, 1 (2014), 155–169.
[58]
Junjie Wu, Hui Xiong, and Jian Chen. 2009. Adapting the right measures for k-means clustering. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 877–886.
[59]
Rongkai Xia, Yan Pan, Lei Du, and Jian Yin. 2014. Robust multi-view spectral clustering via low-rank and sparse decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence.
[60]
Yuan Xie, Dacheng Tao, Wensheng Zhang, Yan Liu, Lei Zhang, and Yanyun Qu. 2018. On unifying multi-view self-representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision 126, 11 (2018), 1157–1179.
[61]
Bo Yuan, Xinbo Gao, Zhenxing Niu, and Qi Tian. 2019. Discovering latent topics by Gaussian latent Dirichlet allocation and spectral clustering. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 1 (2019), 1–18.
[62]
Zhao Zhang, Jiahuan Ren, Sheng Li, Richang Hong, Zhengjun Zha, and Meng Wang. 2019. Robust subspace discovery by block-diagonal adaptive locality-constrained representation. In Proceedings of the 27th ACM International Conference on Multimedia. 1569–1577.
[63]
Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. 2006. Learning with hypergraphs: Clustering, classification, and embedding. In Proceedings of the Advances in Neural Information Processing Systems.
[64]
Peng Zhou, Liang Du, Hanmo Wang, Lei Shi, and Yi-Dong Shen. 2015. Learning a robust consensus matrix for clustering ensemble via kullback-Leibler divergence minimization. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.
Index Terms
- Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering
Recommendations
Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor
As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association ...
Multi-view clustering with dual tensors
AbstractMulti-view clustering methods based on tensor have achieved favorable performance thanks to the powerful capacity of capturing the high-order correlation hidden in multi-view data. However, many existing works only pay attention to exploring the ...
NOODLE: Joint Cross-View Discrepancy Discovery and High-Order Correlation Detection for Multi-View Subspace Clustering
Benefiting from the effective exploration of the valuable topological pair-wise relationship of data points across multiple views, multi-view subspace clustering (MVSC) has received increasing attention in recent years. However, we observe that existing ...
Comments
Information & Contributors
Information
Published In
January 2024
639 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3613542
- Editor:
- Abdulmotaleb El Saddik
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 September 2023
Online AM: 03 August 2023
Accepted: 28 July 2023
Revised: 02 July 2023
Received: 20 September 2022
Published in TOMM Volume 20, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- Guangdong Natural Science Foundation
- Shenzhen Science and Technology Program
- Humanities and Social Sciences Foundation of the Ministry of Education of China
- Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 330Total Downloads
- Downloads (Last 12 months)330
- Downloads (Last 6 weeks)27
Reflects downloads up to 27 Jul 2024
Other Metrics
Citations
Cited By
View all- Bowen DHaiquan WYuxuan LZhao JMa YRunhe H(2024)Fair and Robust Federated Learning via Decentralized and Adaptive Aggregation based on BlockchainACM Transactions on Sensor Networks10.1145/3673656Online publication date: 17-Jun-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in