Abstract
Multi-label twin support vector machine (MLTSVM) is an excellent multi-label classification algorithm, which has attracted much attention. Although MLTSVM can effectively solve the multi-label classification problem, it has some drawbacks. a) MLTSVM uses the same feature representation for each label, but in practice, each label has its own specific features. Therefore, MLTSVM might not obtain the optimal classification results. b) In practical applications, there are a large number of samples with missing labels and only a small number of samples with complete labels, because it is expensive to obtain all labels of samples. However, MLTSVM can only use expensive samples with complete labels, not cheap samples with missing labels. For the above drawbacks, we propose an improved MLTSVM using label-specific features with missing labels (LSFML-MLTSVM) in this paper. LSFML-MLTSVM first extracts label-specific features via using semi-supervised clustering analysis and then obtains the structure information of samples and the geometry information of the marginal distribution. Furthermore, in the label-specific feature space, the above two valuable information is introduced into MLTSVM to reconstruct the classification model. Finally, the successive overrelaxation (SOR) algorithm is used to solve the classification model efficiently. Experimental results on benchmark multi-label datasets show that LSFML-MLTSVM has better classification performance.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
The multi-label benchmark datasets are available in the multi-label classification dataset repository, http://www.uco.es/kdis/mllresources/#HallEtAl2009.
References
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Doroudyan MH, Niaki STA (2021) Pattern recognition in financial surveillance with the ARMA-GARCH time series model using support vector machine. Expert Syst Appl 182:115334
Islam MMM, Prosvirin AE, Kim JM (2021) Data-driven prognostic scheme for rolling-element bearings using a new health index and variants of least-square support vector machines. Mech Syst Signal Process 160:107853
Li X, Yang J, Fan Y, Xie M, Qian X, Li H (2021) Rapid monitoring of heavy metal pollution in lake water using nitrogen and phosphorus nutrients and physicochemical indicators by support vector machine. Chemosphere 280:130599
Sanchez Lasheras JE, Sanchez Lasheras F, Gonzalez Donquiles C, Tardon A, Castano-Vinyals G, Palazuelos C, Salas D, Martin Sanchez V, de Cos Juez FJ (2021) Hybrid algorithm for the classification of prostate cancer patients of the mcc-Spain study based on support vector machines and genetic algorithms. Neurocomputing 452:386–394
Kurilova V, Goga J, Oravec M, Pavlovicova J, Kajan S (2021) Support vector machine and deep-learning object detection for localisation of hard exudates. Sci Rep 11(1):16045
Deng N, Tian Y, Zhang C (2012) Support vector machines: optimization based theory, algorithms, and extensions. Chapman & Hall/CRC, New York
Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910
Che Z, Liu B, Xiao Y, Cai H (2021) Twin support vector machines with privileged information. Inf Sci 573:141–153
Prasad SC, Balasundaram S (2021) On lagrangian L2-norm pinball twin bounded support vector machine via unconstrained convex minimization. Inf Sci 571:279–302
Sharma S, Rastogi R, Chandra S (2021) Large-scale twin parametric support vector machine using pinball loss function. IEEE Trans Syst Man Cybern Syst 51(2):987–1003
Richhariya B, Tanveer M (2021) A fuzzy universum least squares twin support vector machine (FULSTSVM). Neural Comput Applic
Ma J, Yang L, Sun Q (2021) Adaptive robust learning framework for twin support vector machine classification. Knowl-Based Syst 211:106536
Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Aanal 17(4):649–664
Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl-Based Syst 81:131–147
Gao Z, Fang S, Gao X, Luo J, Medhin N (2021) A novel kernel-free least squares twin support vector machine for fast and accurate multi-class classification. Knowl-Based Syst 226:107123
Yang Z, Shao Y, Zhang X (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Applic 22(Supplement-1):153–161
Ding S, Sun Y, An Y, Jia W (2020) Multiple birth support vector machine based on recurrent neural networks. Appl Intell 50(7):2280–2292
Wen Y, Ma J, Yuan C, Yang L (2020) Projection multi-birth support vector machinea for multi-classification. Appl Intell 50(10):3040–3056
Ding S, Zhang J, Zhang X, An Y (2018) Survey on multi class twin support vector machines. Ruan Jian Xue Bao/J Sofw 29(1):89–108
Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl-Based Syst 81:131–147
Gu H, Niu B, Gao Z (2014) A directed acyclic graph algorithm for multi-class classification based on twin support vector machine. J Inf Comput Sci 11(18):6529–6536
Xu Y, Guo R, Wang L (2013) A twin multi-class classification support vector machine. Cogn Comput 5(4):580–588
de Lima MD, Costa NL, Barbosa R (2018) Improvements on least squares twin multi-class classification support vector machine. Neurocomputing 313:196–205
Ai Q, Wang A, Wang Y, Sun H (2019) An improved Twin-KSVC with its applications. Neural Comput Applic 31(10):6615– 6624
Qiang W, Zhang J, Zhen L, Jing L (2020) Robust weighted linear loss twin multi-class support vector regression for large-scale classification. Signal Process 170:107449
Bendjoudi I, Vanderhaegen F, Hamad D, Dornaika F (2021) Multi-label, multi-task CNN approach for context-based emotion recognition. Inf Fusion 76:422–428
Singh S, Majumdar A (2020) Non-intrusive load monitoring via multi-label sparse representation-based classification. IEEE Trans Smart Grid 11(2):1799–1801
Sun Z, Wang C, Zhao Y, Yan C (2020) Multi-label ECG signal classification based on ensemble classifier. IEEE Access 8:117986–117996
Ji Z, Cui B, Zhao Y, Li H, Jiang Y, Xiang T, Hospedales T, Fu Y (2020) Deep ranking for image zero-shot multi-label classification. IEEE Trans Image Process 29:6549–6560
Qin Q, Huang L, Wei Z (2020) Deep multilevel similarity hashing with fine-grained features for multi-label image retrieval. Neurocomputing 409:46–59
Bogaert M, Lootens J, Van den Poel D, Ballings M (2019) Evaluating multi-label classifiers and recommender systems in the financial service sector. Eur J Oper Res 279(2):620–634
Zhang M, Zhou Z (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837
Wu G, Tian Y, Zhang C (2018) A unified framework implementing linear binary relevance for multi-label learning. Neurocomputing 289:86–100
Xie J, Yu L, Zhu L, Duan G (2019) Conditional entropy based classifier chains for multi-label classification. Neurocomputing 335:185–194
Liu B, Tsoumakas G (2020) Dealing with class imbalance in classifier chains via random undersampling. Knowl-Based Syst 192:105292
Rastin N, Taheri M, Jahromi MZ (2021) A stacking weighted k-nearest neighbour with thresholding. Inf Sci 571:605–622
Zhang M, Zhou Z (2007) ML-KNN: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048
Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification Advances in Neural Information Processing Systems, vol 14. MIT Press, Cambridge, pp 681–687
Wu G, Zheng R, Tian Y, Liu D (2020) Joint ranking svm and binary relevance with robust low-rank learning for multi-label classification. Neural Netw 122:24–39
Wang H, Xu Y (2021) Sparse elastic net multi-label rank support vector machine with pinball loss and its applications. Appl Soft Comput 104:107232
Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: A novel twin support vector machine to multi-label learning. Pattern Recogn 52:61–74
Ai Q, Kang Y, Wang A (2021) A novel semi-supervised multi-label twin support vector machine. Intell Autom Soft Comput 27(1):205–220
Ai Q, Kang Y, Wang A, Li X, Li F (2020) An effective semi-supervised multi-label least squares twin support vector machine. IEEE Access 8:213460–213472
Zhang M, Wu L (2015) LIFT: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120
Weng W, Lin Y, Wu S, Li Y, Kang Y (2018) Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273:385–394
Huang J, Li G, Huang Q, Wu X (2018) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889
Ma J, Zhang H, Chow TWS (2021) Multilabel classification with label-specific features and classifiers: A coarse- and fine-tuned framework. IEEE Trans Cybern 51(2):1028–1042
Chen Z, Zhang M (2019) Multi-label learning with regularization enriched label-specific features Proceedings of The Eleventh Asian Conference on Machine Learning, vol 101. PMLR, Nagoya, pp 411–424
Guo Y, Chung F, Li G, Wang J, Gee JC (2019) Leveraging label-specific discriminant mapping features for multi-label learning. ACM Trans Knowl Discov Data 13(2):24
Wang Y, Zheng W, Cheng Y, Zhao D (2020) Joint label completion and label-specific features for multi-label learning algorithm. Soft Comput 24(9):6553–6569
Yu Z, Zhang M (2021) Multi-label classification with label-specific feature generation: a wrapped approach. IEEE Trans Pattern Anal Mach Intell
Chen Y, Weng W, Wu S, Chen B, Fan Y, Liu J (2021) An efficient stacking model with label selection for multi-label classification. Appl Intell 51(1):308–325
Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323
Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Syst 159:148–157
Weng W, Chen Y, Chen C, Wu S, Liu J (2020) Non-sparse label specific features selection for multi-label classification. Neurocomputing 377:85–94
Jia X, Zhu S, Li W (2020) Joint label-specific features and correlation information for multi-label learning. J Comput Sci Technol 35(2):247–258
Huang J, Qin F, Zheng X, Cheng Z, Yuan Z, Zhang W, Huang Q (2019) Improving multi-label classification with missing labels by learning label-specific features. Inf Sci 492:124–146
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(36):2399–2434
Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037
Chang C.-C., Lin C.-J. (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27
Demšar J. (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Acknowledgments
We thank all anonymous reviewers for their helpful comments, which improved the quality of this paper.
Funding
This work was supported in part by the Natural Science Foundation of Liaoning province in China (2020-MS-281) and Talent Cultivation Project of University of Science and Technology Liaoning in China (2018RC05).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that there are no conflicts of interest in the publication of this paper.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ai, Q., Li, F., Li, X. et al. An improved MLTSVM using label-specific features with missing labels. Appl Intell 53, 8039–8060 (2023). https://doi.org/10.1007/s10489-022-03634-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-03634-w