research-article

Public Access

Multi-View Fusion with Extreme Learning Machine for Clustering

Authors:

Yongshan Zhang,

Philip S. YuAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 10, Issue 5

Article No.: 53, Pages 1 - 23

https://doi.org/10.1145/3340268

Published: 10 October 2019 Publication History

All formats PDF

Abstract

Unlabeled, multi-view data presents a considerable challenge in many real-world data analysis tasks. These data are worth exploring because they often contain complementary information that improves the quality of the analysis results. Clustering with multi-view data is a particularly challenging problem as revealing the complex data structures between many feature spaces demands discriminative features that are specific to the task and, when too few of these features are present, performance suffers. Extreme learning machines (ELMs) are an emerging form of learning model that have shown an outstanding representation ability and superior performance in a range of different learning tasks. Motivated by the promise of this advancement, we have developed a novel multi-view fusion clustering framework based on an ELM, called MVEC. MVEC learns the embeddings from each view of the data via the ELM network, then constructs a single unified embedding according to the correlations and dependencies between each embedding and automatically weighting the contribution of each. This process exposes the underlying clustering structures embedded within multi-view data with a high degree of accuracy. A simple yet efficient solution is also provided to solve the optimization problem within MVEC. Experiments and comparisons on eight different benchmarks from different domains confirm MVEC’s clustering accuracy.

References

[1]

Steffen Bickel and Tobias Scheffer. 2004. Multi-view clustering. In Proceedings of the 2004 International Conference on Data Mining, Vol. 4. 19--26.

[2]

Maria Brbić and Ivica Kopriva. 2018. Multi-view low-rank sparse subspace clustering. Pattern Recognition 73 (2018), 247--258.

[3]

Xiao Cai, Feiping Nie, and Heng Huang. 2013. Multi-view K-means clustering on big data. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2598--2604.

Digital Library

[4]

Xiao Cai, Feiping Nie, Heng Huang, and Farhad Kamangar. 2011. Heterogeneous image feature integration via multi-modal spectral clustering. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. 1977--1984.

Digital Library

[5]

Jiuwen Cao, Zhiping Lin, Guang-Bin Huang, and Nan Liu. 2012. Voting based extreme learning machine. Information Sciences 185, 1 (2012), 66--77.

Digital Library

[6]

Kamalika Chaudhuri, Sham M. Kakade, Karen Livescu, and Karthik Sridharan. 2009. Multi-view clustering via canonical correlation analysis. In Proceedings of the 26th International Conference on Machine Learning. 129--136.

Digital Library

[7]

Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. 2015. Multi-view subspace clustering. In Proceedings of the 2015 IEEE International Conference on Computer Vision. 4238--4246.

Digital Library

[8]

Derek Greene. 2009. A matrix factorization approach for integrating multiple data views. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases. 423--438.

[9]

Saima Hassan, Mojtaba Ahmadieh Khanesar, Jafreezal Jaafar, and Abbas Khosravi. 2017. Comparative analysis of three approaches of antecedent part generation for an IT2 TSK FLS. Applied Soft Computing 51 (2017), 130--144.

Digital Library

[10]

Lifang He, Chun-Ta Lu, Hao Ding, Shen Wang, Linlin Shen, Philip S. Yu, and Ann B. Ragin. 2017. Multi-way multi-level kernel modeling for neuroimaging classification. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 356--364.

[11]

Qing He, Xin Jin, Changying Du, Fuzhen Zhuang, and Zhongzhi Shi. 2014. Clustering in extreme learning machine feature space. Neurocomputing 128 (2014), 88--95.

Digital Library

[12]

Chenping Hou, Feiping Nie, Tao Hong, and Dongyun Yi. 2017. Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Transactions on Knowledge and Data Engineering 29, 9 (2017), 1998--2011.

Digital Library

[13]

Gao Huang, Tianchi Liu, Yan Yang, Zhiping Lin, Shiji Song, and Cheng Wu. 2015. Discriminative clustering via extreme learning machine. Neural Networks 70 (2015), 1--8.

Digital Library

[14]

Gao Huang, Shiji Song, Jatinder ND Gupta, and Cheng Wu. 2014. Semi-supervised and unsupervised extreme learning machines. IEEE Transactions on Cybernetics 44, 12 (2014), 2405--2417.

[15]

G.-B. Huang, Hongming Zhou, Xiaojian Ding, and Rui Zhang. 2012. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 42, 2 (2012), 513--529.

Digital Library

[16]

Liyanaarachchi Lekamalage Chamara Kasun, Yan Yang, Guang-Bin Huang, and Zhengyou Zhang. 2016. Dimension reduction with extreme learning machine. IEEE Transactions on Image Processing 25, 8 (2016), 3906--3918.

[17]

Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong. 2013. Representational learning with ELMs for big data. IEEE Intelligent Systems 28, 6 (2013), 31--34.

[18]

Young-Min Kim, Massih-Reza Amini, Cyril Goutte, and Patrick Gallinari. 2010. Multi-view clustering of multilingual documents. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 821--822.

Digital Library

[19]

Abhishek Kumar, Piyush Rai, and Hal Daume. 2011. Co-regularized multi-view spectral clustering. In Advances in Neural Information Processing Systems. 1413--1421.

[20]

Shao-Yuan Li, Yuan Jiang, and Zhi-Hua Zhou. 2010. Partial multi-view clustering. In Proceedings of the 24th AAAI Conference on Artificial Intelligence. 1968--1974.

[21]

Jialu Liu, Chi Wang, Jing Gao, and Jiawei Han. 2013. Multi-view clustering via joint nonnegative matrix factorization. In Proceedings of the 2013 SIAM International Conference on Data Mining. 252--260.

[22]

Tianchi Liu, Chamara Kasun Liyanaarachchi Lekamalage, Guang-Bin Huang, and Zhiping Lin. 2018. An adaptive graph learning method based on dual data representations for clustering. Pattern Recognition 77 (2018), 126--139.

Digital Library

[23]

Tianchi Liu, Chamara Kasun Liyanaarachchi Lekamalage, Guang-Bin Huang, and Zhiping Lin. 2018. Extreme learning machine for joint embedding and clustering. Neurocomputing 277 (2018), 78--88.

Digital Library

[24]

Feiping Nie, Guohao Cai, Jing Li, and Xuelong Li. 2018. Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Transactions on Image Processing 27, 3 (2018), 1501--1511.

[25]

Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Ding. 2010. Efficient and robust feature selection via joint &ell;_2,1-norms minimization. In Advances in Neural Information Processing Systems. 1813--1821.

Digital Library

[26]

Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 1881--1887.

[27]

Feiping Nie, Jing Li, and Xuelong Li. 2017. Self-weighted multiview clustering with multiple graphs. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2564--2570.

[28]

Feiping Nie, Zinan Zeng, Ivor W. Tsang, Dong Xu, and Changshui Zhang. 2011. Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering. IEEE Transactions on Neural Networks 22, 11 (2011), 1796--1808.

Digital Library

[29]

Huimin Pei, Kuaini Wang, Qiang Lin, and Ping Zhong. 2018. Robust semi-supervised extreme learning machine. Knowledge-Based Systems 159 (2018), 203--220.

[30]

Michael D. Plummer and László Lovász. 1986. Matching Theory. Vol. 29. North-Holland Publishing Company, Amsterdam.

[31]

Lei Shi, Liang Du, and Yi-Dong Shen. 2014. Robust spectral learning for unsupervised feature selection. In Proceedings of the 2014 IEEE International Conference on Data Mining. 977--982.

Digital Library

[32]

Yu Su, Shiguang Shan, Xilin Chen, and Wen Gao. 2011. Classifiability-based discriminatory projection pursuit. IEEE Transactions on Neural Networks 22, 12 (2011), 2050--2061.

Digital Library

[33]

Chang Tang, Jiajia Chen, Xinwang Liu, Miaomiao Li, Pichao Wang, Minhui Wang, and Peng Lu. 2018. Consensus learning guided multi-view unsupervised feature selection. Knowledge-Based Systems 160 (2018), 49--60.

[34]

Chang Tang, Xinzhong Zhu, Xinwang Liu, Miaomiao Li, Pichao Wang, Changqing Zhang, and Lizhe Wang. 2019. Learning a joint affinity graph for multiview subspace clustering. IEEE Transactions on Multimedia 21, 7 (2019), 1724--1736.

[35]

Chang Tang, Xinzhong Zhu, Xinwang Liu, and Lizhe Wang. 2019. Cross-view local structure preserved diversity and consensus learning for multi-view unsupervised feature selection. In Proceedings of the AAAI Conference on Artificial Intelligence. 595--604.

[36]

Jiexiong Tang, Chenwei Deng, and Guang-Bin Huang. 2016. Extreme learning machine for multilayer perceptron. IEEE Transactions on Neural Networks and Learning Systems 27, 4 (2016), 809--821.

[37]

Jiliang Tang, Xia Hu, Huiji Gao, and Huan Liu. 2013. Unsupervised feature selection for multi-view data in social media. In Proceedings of the 2013 SIAM International Conference on Data Mining. 270--278.

[38]

Wei Tang, Zhengdong Lu, and Inderjit S. Dhillon. 2009. Clustering with multiple graphs. In Proceedings of the 9th IEEE International Conference on Data Mining. 1016--1021.

[39]

Muhammad Uzair and Ajmal Mian. 2017. Blind domain adaptation with augmented extreme learning machine features. IEEE Transactions on Cybernetics 47, 3 (2017), 651--660.

[40]

Hao Wang, Yan Yang, Bing Liu, and Hamido Fujita. 2019. A study of graph-based system for multi-view clustering. Knowledge-Based Systems 163 (2019), 1009--1019.

[41]

Qiang Wang, Yong Dou, Xinwang Liu, Qi Lv, and Shijie Li. 2016. Multi-view clustering with extreme learning machine. Neurocomputing 214 (2016), 483--494.

Digital Library

[42]

Zhelong Wang, Donghui Wu, Raffaele Gravina, Giancarlo Fortino, Yongmei Jiang, and Kai Tang. 2017. Kernel fusion based extreme learning machine for cross-location activity recognition. Information Fusion 37 (2017), 1--9.

Digital Library

[43]

Chi Man Wong, Chi Man Vong, Pak Kin Wong, and Jiuwen Cao. 2018. Kernel-based multilayer extreme learning machines for representation learning. IEEE Transactions on Neural Networks and Learning Systems 29, 3 (2018), 757--762.

[44]

Jinglin Xu, Junwei Han, and Feiping Nie. 2016. Discriminatively embedded k-means for multi-view clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5356--5364.

[45]

Jinglin Xu, Junwei Han, Feiping Nie, and Xuelong Li. 2017. Re-weighted discriminatively embedded -means for multi-view clustering. IEEE Transactions on Image Processing 26, 6 (2017), 3016--3027.

Digital Library

[46]

Yimin Yang and Q. M. Jonathan Wu. 2016. Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Transactions on Cybernetics 46, 11 (2016), 2570--2583.

[47]

Yimin Yang, Q. M. Jonathan Wu, and Yaonan Wang. [n.d.]. Autoencoder with invertible functions for dimension reduction and image reconstruction. IEEE Transactions on Systems, Man, and Cybernetics: Systems 48, 7 ([n.d.]), 1065--1079.

[48]

Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 30. 4279--4287.

[49]

Yongshan Zhang, Jia Wu, Zhihua Cai, Peng Zhang, and Ling Chen. 2016. Memetic extreme learning machine. Pattern Recognition 58 (2016), 135--148.

Digital Library

[50]

Yongshan Zhang, Jia Wu, Chuan Zhou, and Zhihua Cai. 2017. Instance cloned extreme learning machine. Pattern Recognition 68 (2017), 52--65.

Digital Library

[51]

Wentao Zhu, Jun Miao, and Laiyun Qing. 2014. Constrained extreme learning machine: A novel highly discriminative random feedforward neural network. In Proceedings of the 2014 International Joint Conference on Neural Networks. 800--807.

[52]

Xiaofeng Zhu, Xuelong Li, and Shichao Zhang. 2016. Block-row sparse multiview multilabel learning for image classification. IEEE Transactions on Cybernetics 46, 2 (2016), 450--461.

[53]

Fuzhen Zhuang, George Karypis, Xia Ning, Qing He, and Zhongzhi Shi. 2012. Multi-view learning via probabilistic latent semantic analysis. Information Sciences 199 (2012), 20--30.

Digital Library

[54]

Wenzhang Zhuge, Feiping Nie, Chenping Hou, and Dongyun Yi. 2017. Unsupervised single and multiple views feature extraction with structured graph. IEEE Transactions on Knowledge and Data Engineering 29, 10 (2017), 2347--2359.

Digital Library

Cited By

Liu QYue JFang YXia SFang L(2024)HyperMamba: A Spectral-Spatial Adaptive Mamba for Hyperspectral Image ClassificationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.348247362(1-14)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3482473
Li BFang LChen NKang JYue J(2024)Enhancing Hyperspectral Image Classification: Leveraging Unsupervised Information With Guided Group Contrastive LearningIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.335070062(1-17)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3350700
Wibawa IMachbub CRohman AHidayat E(2023)Modified online sequential extreme learning machine algorithm using model predictive control approachIntelligent Systems with Applications10.1016/j.iswa.2023.20019118(200191)Online publication date: May-2023
https://doi.org/10.1016/j.iswa.2023.200191
Show More Cited By

Index Terms

Multi-View Fusion with Extreme Learning Machine for Clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Multi-view clustering with extreme learning machine

Nowadays, data always have multiple representations, and a good feature representation usually leads to a good clustering performance. Existing multi-view clustering works generally integrate multiple complementary information to gain better clustering ...
Local kernel alignment based multi-view clustering using extreme learning machine

A similarity or dissimilarity measure, such as the Euclidean distance, is crucial to discriminative clustering algorithms. These measures used to calculate pairwise similarities between samples rely on data representations in a feature space. However, ...
Multi-view clustering via spectral embedding fusion

Multi-view learning, such as multi-view feature learning and multi-view clustering, has a wide range of applications in machine learning and pattern recognition. Most previous studies employ the multiple data information from various views to improve the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 10, Issue 5

Special Section on Advances in Causal Discovery and Inference and Regular Papers

September 2019

314 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3360733

Editor:
Yu Zheng
JD Finance, China

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2019

Accepted: 01 June 2019

Revised: 01 March 2019

Received: 01 December 2018

Published in TIST Volume 10, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
MQNS
MQEPS
Youth Innovation Promotion Association CAS
MQRSG
US NSF
National Scholarship for Building High Level Universities, China Scholarship Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
597
Total Downloads

Downloads (Last 12 months)200
Downloads (Last 6 weeks)24

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu QYue JFang YXia SFang L(2024)HyperMamba: A Spectral-Spatial Adaptive Mamba for Hyperspectral Image ClassificationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.348247362(1-14)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3482473
Li BFang LChen NKang JYue J(2024)Enhancing Hyperspectral Image Classification: Leveraging Unsupervised Information With Guided Group Contrastive LearningIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.335070062(1-17)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3350700
Wibawa IMachbub CRohman AHidayat E(2023)Modified online sequential extreme learning machine algorithm using model predictive control approachIntelligent Systems with Applications10.1016/j.iswa.2023.20019118(200191)Online publication date: May-2023
https://doi.org/10.1016/j.iswa.2023.200191
Sun YLiu BYu XYu AGao KDing L(2022)Perceiving Spectral Variation: Unsupervised Spectrum Motion Feature Learning for Hyperspectral Image ClassificationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2022.322153460(1-17)Online publication date: 2022
https://doi.org/10.1109/TGRS.2022.3221534
Zhang YWang XJiang XZhou Y(2022)Robust Dual Graph Self-Representation for Unsupervised Hyperspectral Band SelectionIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2022.320320760(1-13)Online publication date: 2022
https://doi.org/10.1109/TGRS.2022.3203207
Zhang YWang XJiang XZhou Y(2022)Marginalized Graph Self-Representation for Unsupervised Hyperspectral Band SelectionIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2021.312167160(1-12)Online publication date: 2022
https://doi.org/10.1109/TGRS.2021.3121671
Ning NLi QZhao KWu B(2021)Multiplex Network Embedding Model with High-Order Node DependenceComplexity10.1155/2021/66441112021(1-18)Online publication date: 6-Mar-2021
https://doi.org/10.1155/2021/6644111
Liu JLiu XLiu CLe BXiao D(2021)Pruning Multilayered ELM Using Cholesky Factorization and Givens Rotation TransformationMathematical Problems in Engineering10.1155/2021/55884262021(1-11)Online publication date: 26-Apr-2021
https://doi.org/10.1155/2021/5588426
Xia ZWu JWu LChen YYang JYu P(2021)A Comprehensive Survey of the Key Technologies and Challenges Surrounding Vehicular Ad Hoc NetworksACM Transactions on Intelligent Systems and Technology10.1145/345198412:4(1-30)Online publication date: 8-Jun-2021
https://dl.acm.org/doi/10.1145/3451984
Shi YZhang SZhou CLiang XYang XLin L(2021)GTAE: Graph Transformer–Based Auto-Encoders for Linguistic-Constrained Text Style TransferACM Transactions on Intelligent Systems and Technology10.1145/344873312:3(1-16)Online publication date: 15-Jun-2021
https://dl.acm.org/doi/10.1145/3448733
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents