Sparse Learning for Linear Twin Parameter-margin Support Vector Machine
Abstract
Twin Parameter-margin support vector machine (TPMSVM) is a recent very powerful binary classifier. To improve its sparsity, a linear sparse TPMSVM (Lin-STPMSVM) is proposed in this paper. In the primal problem, the vectors defining the hyperplane are replaced with their expression in terms of the dual variables as derived from Karush Khun Tucker (KKT) conditions. Then the new primal problems are directly optimized, thus ensuring the sparsity of the solutions. Numerical experiments show that the solution obtained by new model is more sparse without reducing the accuracy. Therefore, Lin-STPMSVM not only inherits the advantages of TPMSVM, but also has the characteristics of sparsity, stability and robustness in dealing with classification problems.
Supplemental Material
ZIP File
paper
- Download
- 1021.38 KB
References
[1]
Jinbo Bi, Kristin Bennett, Mark Embrechts, Curt Breneman, and Minghu Song. 2003. Dimensionality reduction via sparse support vector machines. Journal of Machine Learning Research 3, Mar (2003), 1229–1243.
[2]
Jair Cervantes, Farid Garcia-Lamont, Lisbeth Rodríguez-Mazahua, and Asdrubal Lopez. 2020. A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing 408 (2020), 189–215.
[3]
Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning 20 (1995), 273–297.
[4]
Nello Cristianini and Bernhard Scholkopf. 2002. Support vector machines and kernel methods: the new generation of learning machines. Ai Magazine 23, 3 (2002), 31–31.
[5]
Renato De Leone, Nadaniela Egidi, and Lorella Fatone. 2020. The use of grossone in elastic net regularization and sparse support vector machines. Soft Computing 24, 23 (2020), 17669–17677.
[6]
Renato De Leone, Nadaniela Egidi, and Lorella Fatone. 2022. The Use of Infinities and Infinitesimals for Sparse Classification Problems. In Numerical Infinities and Infinitesimals in Optimization. Springer, 151–166.
[7]
Pei-Yi Hao. 2010. New support vector algorithms with parametric insensitive/margin model. Neural networks 23, 1 (2010), 60–73.
[8]
Kaizhu Huang, Danian Zheng, Jun Sun, Yoshinobu Hotta, Katsuhito Fujimoto, and Satoshi Naoi. 2010. Sparse learning for support vector classification. Pattern Recognition Letters 31, 13 (2010), 1944–1951.
[9]
Reshma Khemchandani, Suresh Chandra, 2007. Twin support vector machines for pattern classification. IEEE Transactions on pattern analysis and machine intelligence 29, 5 (2007), 905–910.
[10]
Renato De Leone, Francesca Maggioni, and Andrea Spinelli. 2023. A Multiclass Robust Twin Parametric Margin Support Vector Machine with an Application to Vehicles Emissions. sumbmitted 9th International Conference on Machine Learning, Optimization, and Data Science, LOD 2023.
[11]
Renato De Leone, Francesca Maggioni, and Andrea Spinelli. 2023. Robust Twin Parametric Margin Support Vector Machine for Multiclass Classification. arxiv:2306.06213 [cs.LG]
[12]
Olvi L. Mangasarian and Edward W. Wild. 2006. Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1 (jan 2006), 69–74.
[13]
Xinjun Peng. 2011. TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern recognition 44, 10-11 (2011), 2678–2692.
[14]
Zhiquan Qi, Yingjie Tian, and Yong Shi. 2013. Robust twin support vector machine for pattern classification. Pattern recognition 46, 1 (2013), 305–316.
[15]
Bernhard Schölkopf and Alexander J Smola. 2002. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press.
[16]
Bernhard Schölkopf, Alex J Smola, Robert C Williamson, and Peter L Bartlett. 2000. New support vector algorithms. Neural computation 12, 5 (2000), 1207–1245.
[17]
Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding machine learning: From theory to algorithms. Cambridge university press.
[18]
John Shawe-Taylor and Nello Cristianini. 2004. Kernel methods for pattern analysis. Cambridge university press.
[19]
Vladimir N Vapnik. 1999. An overview of statistical learning theory. IEEE transactions on neural networks 10, 5 (1999), 988–999.
[20]
Philip Wolfe. 1961. A duality theorem for non-linear programming. Quarterly of applied mathematics 19, 3 (1961), 239–244.
Index Terms
- Sparse Learning for Linear Twin Parameter-margin Support Vector Machine
Recommendations
Sparse discriminant twin support vector machine for binary classification
AbstractFor a binary classification problem, twin support vector machine (TSVM) has a faster learning speed than support vector machine (SVM) by seeking a pair of nonparallel hyperplanes. However, TSVM has two deficiencies: poor discriminant ability and ...
Twin support vector machine: theory, algorithm and applications
Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and ...
Sparse least square twin support vector machine with adaptive norm
By promoting the parallel hyperplanes to non-parallel ones in SVM, twin support vector machines (TWSVM) have attracted more attention. There are many modifications of them. However, most of the modifications minimize the loss function subject to the I 2-...
Comments
Information & Contributors
Information
Published In
March 2024
478 pages
ISBN:9798400716416
DOI:10.1145/3654823
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Data Availability
Funding Sources
- the European Union - Next Generation EU under the Italian Ministry of University and Research (MUR) National Innovation Ecosystem grant
Conference
CACML 2024
CACML 2024: 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning
March 22 - 24, 2024
Shanghai, China
Acceptance Rates
Overall Acceptance Rate 93 of 241 submissions, 39%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 8Total Downloads
- Downloads (Last 12 months)8
- Downloads (Last 6 weeks)4
Reflects downloads up to 18 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format