research-article

Integrating Global and Local Feature Selection for Multi-Label Learning

Authors:

Xindong WuAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 1

Article No.: 4, Pages 1 - 37

https://doi.org/10.1145/3532190

Published: 20 February 2023 Publication History

Abstract

Multi-label learning deals with the problem where an instance is associated with multiple labels simultaneously. Multi-label data is often of high dimensionality and has many noisy, irrelevant, and redundant features. As an important machine learning task, multi-label feature selection has received considerable attention in recent years due to its promising performance in dealing with high-dimensional multi-label data. Existing multi-label feature selection methods typically select the global features which are shared by all instances in a dataset. However, these multi-label feature selection methods may be suboptimal since they do not consider the specific characteristics of instances. In this paper, we propose a novel algorithm that integrates Global and Local Feature Selection (GLFS) to exploit both the global features and a subset of discriminative features shared only locally by a subgroup of instances in a multi-label dataset. Specifically, GLFS employs linear regression and ℓ_2,1-norm on the regression parameters to achieve simultaneous global and local feature selection. Moreover, the proposed algorithm has an effective mechanism for utilizing label correlations to improve the feature selection. Experiments on real-world multi-label datasets show the superiority of GLFS over the state-of-the-art multi-label feature selection methods.

References

[1]

Richard H. Bartels and George W. Stewart. 1972. Solution of the matrix equation AX + XB = C [F4]. Communications of the ACM 15, 9 (1972), 820–826.

Digital Library

[2]

Zafer Barutcuoglu, Robert E. Schapire, and Olga G. Troyanskaya. 2006. Hierarchical multi-label prediction of gene function. Bioinformatics 22, 7 (2006), 830–836.

Digital Library

[3]

Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. 2004. Learning multi-label scene classification. Pattern Recognition 37, 9 (2004), 1757–1771.

[4]

Forrest Briggs, Xiaoli Z. Fern, Raviv Raich, and Qi Lou. 2013. Instance annotation for multi-instance multi-label learning. ACM Transactions on Knowledge Discovery from Data (TKDD) 7, 3 (2013), 1–30.

Digital Library

[5]

Ricardo S. Cabral, Fernando Torre, Joao P. Costeira, and Alexandre Bernardino. 2011. Matrix completion for multi-label image classification. In Proceedings of the Advances in Neural Information Processing Systems. 190–198.

[6]

Zhiling Cai and William Zhu. 2018. Multi-label feature selection via feature manifold learning and sparsity regularization. International Journal of Machine Learning and Cybernetics 9, 8 (2018), 1321–1334.

[7]

Nicolò Cesa-Bianchi, Matteo Re, and Giorgio Valentini. 2012. Synergy of multi-label hierarchical ensembles, data fusion, and cost-sensitive methods for gene functional inference. Machine Learning 88, 1–2 (2012), 209–241.

Digital Library

[8]

Xiaoya Che, Degang Chen, and Jusheng Mi. 2020. A novel approach for learning label correlation with application to feature selection of multi-label data. Information Sciences 512 (2020), 795–812.

Digital Library

[9]

Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. 2019. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5177–5186.

[10]

Yizong Cheng. 1995. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 17, 8 (1995), 790–799.

Digital Library

[11]

Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, Jan (2006), 1–30.

Digital Library

[12]

Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of the American Statistical Association 56, 293 (1961), 52–64.

[13]

Yumeng Guo, Fulai Chung, Guozheng Li, Jiancong Wang, and James C. Gee. 2019. Leveraging label-specific discriminant mapping features for multi-label learning. ACM Transactions on Knowledge Discovery from Data (TKDD) 13, 2 (2019), 1–23.

Digital Library

[14]

Sujuan Hou, Shangbo Zhou, Ling Chen, Yong Feng, and Karim Awudu. 2016. Multi-label learning with label relevance in advertising video. Neurocomputing 171 (2016), 932–948.

Digital Library

[15]

Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2016. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering 28, 12 (2016), 3309–3323.

Digital Library

[16]

Rui Huang, Weidong Jiang, and Guangling Sun. 2018. Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recognition Letters 112 (2018), 346–352.

[17]

Shuiwang Ji, Lei Tang, Shipeng Yu, and Jieping Ye. 2010. A shared-subspace learning framework for multi-label classification. ACM Transactions on Knowledge Discovery from Data (TKDD) 4, 2 (2010), 1–29.

Digital Library

[18]

Ling Jian, Jundong Li, Kai Shu, and Huan Liu. 2016. Multi-label informed feature selection. In Proceedings of the International Joint Conference on Artificial Intelligence. 1627–1633.

[19]

Deguang Kong, Ji Liu, Bo Liu, and Xuan Bao. 2016. Uncorrelated group lasso. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 1765–1771.

[20]

Cosmin Lazar, Jonatan Taminau, Stijn Meganck, David Steenhoff, Alain Coletta, Colin Molter, Virginie de Schaetzen, Robin Duque, Hugues Bersini, and Ann Nowe. 2012. A survey on filter techniques for feature selection in gene expression microarray analysis. IEEE/ACM Transactions on Computational Biology and Bioinformatics 9, 4 (2012), 1106–1119.

Digital Library

[21]

Feng Li, Duoqian Miao, and Witold Pedrycz. 2017. Granular multi-label feature selection based on mutual information. Pattern Recognition 67 (2017), 410–423.

Digital Library

[22]

Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. 2017. Feature selection: A data perspective. ACM Computing Surveys (CSUR) 50, 6 (2017), 1–45.

Digital Library

[23]

Li Li, Houfeng Wang, Xu Sun, Baobao Chang, Shi Zhao, and Lei Sha. 2015. Multi-label text categorization with joint learning predictions-as-features method. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 835–839.

[24]

Yun Li, Tao Li, and Huan Liu. 2017. Recent advances in feature selection and its applications. Knowledge and Information Systems 53, 3 (2017), 551–577.

Digital Library

[25]

Ming Liang and Xiaolin Hu. 2014. Feature selection in supervised saliency prediction. IEEE Transactions on Cybernetics 45, 5 (2014), 914–926.

[26]

Yaojin Lin, Qinghua Hu, Jinghua Liu, and Jie Duan. 2015. Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168 (2015), 92–103.

Digital Library

[27]

Yaojin Lin, Qinghua Hu, Jinghua Liu, Jinjin Li, and Xindong Wu. 2017. Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Transactions on Fuzzy Systems 25, 6 (2017), 1491–1507.

Digital Library

[28]

Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. 2017. Deep learning for extreme multi-label text classification. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 115–124.

Digital Library

[29]

Yang Liu, Kaiwen Wen, Quanxue Gao, Xinbo Gao, and Feiping Nie. 2018. SVM based multi-label learning with missing labels for image annotation. Pattern Recognition 78 (2018), 307–317.

Digital Library

[30]

Foteini Markatopoulou, Vasileios Mezaris, and Ioannis Patras. 2018. Implicit and explicit concept relations in deep neural networks for multi-label video/image annotation. IEEE Transactions on Circuits and Systems for Video Technology 29, 6 (2018), 1631–1644.

[31]

Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. 2010. Efficient and robust feature selection via joint \(\ell\) 2, 1-norms minimization. In Proceeding of Advances in Neural Information Processing Systems. Vol. 23, 1813–1821.

[32]

Mohsen Paniri, Mohammad Bagher Dowlatshahi, and Hossein Nezamabadi-Pour. 2020. MLACO: A multi-label feature selection algorithm based on ant colony optimization. Knowledge-Based Systems 192 (2020), 105285.

[33]

Hanchuan Peng, Fuhui Long, and Chris Ding. 2005. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 8 (2005), 1226–1238.

Digital Library

[34]

Guo-Jun Qi, Xian-Sheng Hua, Yong Rui, Jinhui Tang, Tao Mei, and Hong-Jiang Zhang. 2007. Correlative multi-label video annotation. In Proceedings of the 15th ACM International Conference on Multimedia. 17–26.

Digital Library

[35]

Alex Rodriguez and Alessandro Laio. 2014. Clustering by fast search and find of density peaks. Science 344, 6191 (2014), 1492–1496.

[36]

Martha Roseberry, Bartosz Krawczyk, and Alberto Cano. 2019. Multi-label punitive kNN with self-adjusting memory for drifting data streams. ACM Transactions on Knowledge Discovery from Data (TKDD) 13, 6 (2019), 1–31.

Digital Library

[37]

Timothy N. Rubin, America Chambers, Padhraic Smyth, and Mark Steyvers. 2012. Statistical topic models for multi-label document classification. Machine Learning 88, 1–2 (2012), 157–208.

Digital Library

[38]

Robert E. Schapire and Yoram Singer. 2000. BoosTexter: A boosting-based system for text categorization. Machine Learning 39, 2 (2000), 135–168.

Digital Library

[39]

Ying Hu, Yong Zhang, and Dunwei Gong. 2020. Multiobjective particle swarm optimization for feature selection with fuzzy cost. IEEE Transactions on Cybernetics 51, 2 (2020), 874–888.

[40]

James Joseph Sylvester. 1884. Sur l’équation en matrices px= xq. Comptes Rendus de l’Académie des Sciences 99, 2 (1884), 67–71.

[41]

Hong Tao, Chenping Hou, Feiping Nie, Yuanyuan Jiao, and Dongyun Yi. 2015. Effective discriminative feature selection with nontrivial solution. IEEE Transactions on Neural Networks and Learning Systems 27, 4 (2015), 796–808.

[42]

Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: An overview. International Journal of Data Warehousing and Mining (IJDWM) 3, 3 (2007), 1–13.

[43]

Alexis Vallet and Hiroyasu Sakamoto. 2015. A multi-label convolutional neural network for automatic image annotation. Journal of Information Processing 23, 6 (2015), 767–775.

[44]

Xiao Wang, Jun Zhang, and Guo-Zheng Li. 2015. Multi-location gram-positive and gram-negative bacterial protein subcellular localization using gene ontology and multi-label classifier ensemble. BMC Bioinformatics 16, 12 (2015), 1–7.

[45]

Tong Wei and Yu-Feng Li. 2019. Learning compact model for large-scale multi-label data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5385–5392.

Digital Library

[46]

Marcel Wever, Alexander Tornede, Felix Mohr, and Eyke Hullermeier. 2021. AutoML for multi-label classification: overview and empirical evaluation. IEEE Transactions on Pattern Analysis & Machine Intelligence 43, 9 (2021), 3037–3054.

[47]

Baoyuan Wu, Zhilei Liu, Shangfei Wang, Bao-Gang Hu, and Qiang Ji. 2014. Multi-label learning with missing labels. In Proceedings of the 2014 22nd International Conference on Pattern Recognition. IEEE, 1964–1968.

Digital Library

[48]

Xi-Zhu Wu and Zhi-Hua Zhou. 2017. A unified view of multi-label performance measures. In Proceedings of the International Conference on Machine Learning. PMLR, 3780–3788.

[49]

Bing Xue, Mengjie Zhang, Will N. Browne, and Xin Yao. 2015. A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20, 4 (2015), 606–626.

Digital Library

[50]

Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. 2011. L2, 1-norm regularized discriminative feature selection for unsupervised. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 1589–1594.

[51]

Zhangjing Yang, Qiaolin Ye, Qiao Chen, Xu Ma, Liyong Fu, Guowei Yang, He Yan, and Fan Liu. 2020. Robust discriminant feature selection via joint L2, 1-norm distance minimization and maximization. Knowledge-Based Systems 207 (2020), 106090.

[52]

Ying Yu, Witold Pedrycz, and Duoqian Miao. 2014. Multi-label classification by exploiting label correlations. Expert Systems with Applications 41, 6 (2014), 2989–3004.

Digital Library

[53]

Jia Zhang, Yidong Lin, Min Jiang, Shaozi Li, Yong Tang, and Kay Chen Tan. 2020. Multi-label feature selection via global relevance and redundancy optimization. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2512–2518.

[54]

Jia Zhang, Zhiming Luo, Candong Li, Changen Zhou, and Shaozi Li. 2019. Manifold regularized discriminative feature selection for multi-label learning. Pattern Recognition 95 (2019), 136–150.

Digital Library

[55]

Jingpu Zhang, Zuping Zhang, Zixiang Wang, Yuting Liu, and Lei Deng. 2018. Ontological function annotation of long non-coding RNAs through hierarchical multi-label classification. Bioinformatics 34, 10 (2018), 1750–1757.

[56]

Min-Ling Zhang, José M. Peña, and Victor Robles. 2009. Feature selection for multi-label naive Bayes classification. Information Sciences 179, 19 (2009), 3218–3229.

Digital Library

[57]

Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition 40, 7 (2007), 2038–2048.

Digital Library

[58]

Min-Ling Zhang and Zhi-Hua Zhou. 2013. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2013), 1819–1837.

[59]

Ping Zhang, Guixia Liu, and Wanfu Gao. 2019. Distinguishing two types of labels for multi-label feature selection. Pattern Recognition 95 (2019), 72–82.

Digital Library

[60]

Yong Zhang, Yan-hu Wang, Dun-wei Gong, and Xiao-yan Sun. 2021. Clustering-guided particle swarm feature selection algorithm for high-dimensional imbalanced data with missing values. IEEE Transactions on Evolutionary Computation. 1–1. DOI:

[61]

Pengfei Zhu, Qian Xu, Qinghua Hu, Changqing Zhang, and Hong Zhao. 2018. Multi-label feature selection with missing labels. Pattern Recognition 74 (2018), 488–502.

Digital Library

[62]

Yue Zhu, James T. Kwok, and Zhi-Hua Zhou. 2017. Multi-label learning with global and local label correlation. IEEE Transactions on Knowledge and Data Engineering 30, 6 (2017), 1081–1094.

Cited By

Zhang ZYao JLiu LLi JLi LWu X(2024)Partial Label Feature Selection: An Adaptive ApproachIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336569136:8(4178-4191)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3365691
Dai JLiu QChen WZhang C(2024)Multilabel Feature Selection Based on Fuzzy Mutual Information and Orthogonal RegressionIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341517632:9(5136-5148)Online publication date: Sep-2024
https://doi.org/10.1109/TFUZZ.2024.3415176
Sun ZXie HLiu JYu Y(2024)Multi-label feature selection via adaptive dual-graph optimizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122884243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122884
Show More Cited By

Index Terms

Integrating Global and Local Feature Selection for Multi-Label Learning
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Recommendations

Multi-label learning with kernel local label information
Abstract
It is important to fully utilize label correlations in multi-label learning. If there is a strong positive correlation between label i and label j, an instance associated with label i also likely has label j simultaneously. So, label ...
Highlights
- Label correlations are used to train a model and predict labels simultaneously.
Feature selection for multi-label learning with streaming label
Highlights
- A novel framework based on inter-class discrimination and intra-class neighbor recognition is designed to generate label-specific features when each label ...
Abstract
Multi-label feature selection has drawn wide attention in recent years. The existing multi-label feature selection algorithms mainly assume that the labels of the training data are obtained before feature selection takes place. However,...
Manifold regularized discriminative feature selection for multi-label learning
Highlights
- Label correlations are incorporated into the framework via manifold regularization.
Abstract
In multi-label learning, objects are essentially related to multiple semantic meanings, and the type of data is confronted with the impact of high feature dimensionality simultaneously, such as the bioinformatics and text mining ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 1

January 2023

375 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3572846

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2023

Online AM: 10 May 2022

Accepted: 14 April 2022

Revised: 14 March 2022

Received: 13 September 2021

Published in TKDD Volume 17, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Program for Changjiang Scholars and Innovative Research Team in University (PCSIRT) of the Ministry of Education of China
Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
732
Total Downloads

Downloads (Last 12 months)271
Downloads (Last 6 weeks)15

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZYao JLiu LLi JLi LWu X(2024)Partial Label Feature Selection: An Adaptive ApproachIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336569136:8(4178-4191)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3365691
Dai JLiu QChen WZhang C(2024)Multilabel Feature Selection Based on Fuzzy Mutual Information and Orthogonal RegressionIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341517632:9(5136-5148)Online publication date: Sep-2024
https://doi.org/10.1109/TFUZZ.2024.3415176
Sun ZXie HLiu JYu Y(2024)Multi-label feature selection via adaptive dual-graph optimizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122884243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122884
Akinnubi AAgarwal NAlassad MAjiboye JRokne JWang D(2023)Knowledge Graph Embedding for Topical and Entity Classification in Multi-Source Social Network DataProceedings of the 2023 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining10.1145/3625007.3627315(530-537)Online publication date: 6-Nov-2023
https://dl.acm.org/doi/10.1145/3625007.3627315
Zhang ZZhang ZYao JLiu LLi JWu GWu X(2023)Multi-Label Feature Selection Via Adaptive Label Correlation EstimationACM Transactions on Knowledge Discovery from Data10.1145/360456017:9(1-28)Online publication date: 10-Aug-2023
https://dl.acm.org/doi/10.1145/3604560
Kraus VBenabdeslem KBenkabou SMansouri DCanitia B(2023)RSMS: Robust Semi-supervised Multi-label Feature Selection for Regression2023 IEEE 35th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI59109.2023.00022(99-105)Online publication date: 6-Nov-2023
https://doi.org/10.1109/ICTAI59109.2023.00022
Wang SGai KYu JZhu L(2023)BDVFL: Blockchain-based Decentralized Vertical Federated Learning2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00072(628-637)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00072
Pronello NIgnaccolo RIppoliti LFontanella S(2023)Penalized model-based clustering of complex functional dataStatistics and Computing10.1007/s11222-023-10288-233:6Online publication date: 25-Aug-2023
https://dl.acm.org/doi/10.1007/s11222-023-10288-2

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents