article

Recursive projection twin support vector machine via within-class variance minimization

Authors:

Jun LiangAuthors Info & Claims

Pattern Recognition, Volume 44, Issue 10-11

Pages 2643 - 2655

https://doi.org/10.1016/j.patcog.2011.03.001

Published: 01 October 2011 Publication History

Abstract

In this paper, a novel binary classifier coined projection twin support vector machine (PTSVM) is proposed. The idea is to seek two projection directions, one for each class, such that the projected samples of one class are well separated from those of the other class in its respective subspace. In order to further boost performance, a recursive algorithm for PTSVM is proposed to generate more than one projection axis for each class. To overcome the singularity problem, principal component analysis (PCA) is utilized to transform the data in the original space into a low-dimensional subspace wherein the optimization problem of PTSVM is convex and can be solved efficiently. The experimental results on several UCI benchmark data sets and USPS digit database show the feasibility and effectiveness of the proposed method.

References

[1]

Burges, C., A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discovery. v2. 121-167.

Digital Library

[2]

Cristianini, N. and Shawe-Taylor, J., An introduction to support vector machines: and other kernel-based learning methods. 2000. Cambridge University Press, Cambridge; New York.

[3]

Vapnik, V.N., Statistical learning theory. 1998. Wiley, New York.

[4]

Vapnik, V.N., The nature of statistical learning theory. 1995. Springer-Verlag, New York.

[5]

Osuna, E., Freund, R. and Girosi, F., Training support vector machines: an application to face detection. Proc. 1997 IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognition. 130-136.

[6]

T. Joachims, Text categorization with support vector machines: learning with many relevant features, Machine Learning: ECML-98, (1998), pp. 137-142.

Digital Library

[7]

A. Demiriz, K.P. Bennett, C.M. Breneman, M.J. Embrechts, Support vector machine regression in chemometrics, in: Computing Science and Statistics, Proceedings of the 33rd Symposium on the Interface, American Statistical Association for the Interface Foundation of North America, Washington, D.C., 2001.

[8]

Cao, L., Support vector machines experts for time series forecasting. Neurocomputing. v51. 321-339.

[9]

Cortes, C. and Vapnik, V., Support-vector networks. Machine Learning. v20. 273-297.

[10]

Platt, J., Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (Eds.), Advances in Kernel Methods-Support Vector Learning, MIT press, Combridge, MA. pp. 185-208.

[11]

Joachims, T., Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning. 1999. MIT Press, Cambridge, MA.

[12]

Collobert, R. and Bengio, S., SVMTorch: support vector machines for large-scale regression problems. J. Mach. Learn. Res. v1. 143-160.

[13]

C. Chang, C. Lin, LIBSVM: a library for support vector machines,Technical Report, Department of Computer Science and Information Engineering, National Taiwan University, 2001.

[14]

Suykens, J. and Vandewalle, J., Least squares support vector machine classifiers. Neural Process. Lett. v9. 293-300.

Digital Library

[15]

Suykens, J.A.K., Least squares support vector machines. 2002. World Scientific, River Edge, NJ.

[16]

Fung, G. and Mangasarian, O., Multicategory proximal support vector machine classifiers. Machine Learning. v59. 77-97.

Digital Library

[17]

Mangasarian, O.L. and Wild, E.W., Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans. Pattern Anal. Mach. Intell. v28. 69-74.

Digital Library

[18]

Jayadeva, Khemchandani, R. and Chandra, S., Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. v29. 905-910.

Digital Library

[19]

Kumar, M. and Gopal, M., Application of smoothing technique on twin support vector machines. Pattern Recognition Lett. v29. 1842-1848.

Digital Library

[20]

Arun Kumar, M. and Gopal, M., Least squares twin support vector machines for pattern classification. Expert Syst. Appl. v36. 7535-7543.

Digital Library

[21]

Ghorai, S., Mukherjee, A. and Dutta, P., Nonparallel plane proximal classifier. Signal Process. v89. 510-522.

Digital Library

[22]

Ye, Q., Zhao, C., Ye, N. and Chen, Y., Multi-weight vector projection support vector machines. Pattern Recognition Lett. v31 i13. 2006-2011.

[23]

Tao, Q., Chu, D. and Wang, J., Recursive support vector machines for dimensionality reduction. IEEE Trans. Neural Networks. v19. 189-193.

[24]

Xiang, C., Fan, X. and Lee, T., Face recognition using recursive Fisher linear discriminant. IEEE Trans. Image Process. v15 i8. 2097-2105.

[25]

Parlett, B., The symmetric eigenvalue problem. 1998. Society for Industrial Mathematics.

[26]

Bishop, C., Pattern recognition and machine learning. 2006. Springer, New York.

[27]

Tao, Y. and Yang, J., Quotient vs. difference: comparison between the two discriminant criteria. Neurocomputing. v73 i10-12. 1808-1817.

[28]

Li, H., Jiang, T. and Zhang, K., Efficient and robust feature extraction by maximum margin criterion. IEEE Trans. Neural Networks. v17 i1. 157-165.

[29]

Boyd, S. and Vandenberghe, L., Convex optimization. 2004. Cambridge Univ Pr.

[30]

S. Ji, J. Ye, A unified framework for generalized Linear Discriminant Analysis, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 2008, pp. 1-7.

[31]

Belhumeur, P., Hespanha, J. and Kriegman, D., Eigenfaces vs. fisherfaces: recognition using class specific linear projection. Computer Vision ECCV'96. 43-58.

Digital Library

[32]

Yang, J., Why can LDA be performed in PCA transformed space?. Pattern Recognition. v36. 563-566.

[33]

Kotsia, I., Zafeiriou, S. and Pitas, I., Novel multiclass classifiers based on the minimization of the within-class variance. IEEE Trans. Neural Networks. v20. 14-34.

Digital Library

[34]

Wang, X., Chung, F. and Wang, S., On minimum class locality preserving variance support vector machine. Pattern Recognition. v43 i8. 2753-2762.

[35]

C. Blake, C. Merz, UCI repository of machine learning databases, (1998).

[36]

Jin, Z., Yang, J., Hu, Z. and Lou, Z., Face recognition based on the uncorrelated discriminant transformation. Pattern Recognition. v34. 1405-1416.

[37]

Xue, H., Chen, S. and Yang, Q., Discriminatively regularized least-squares classification. Pattern Recognition. v42. 93-104.

Digital Library

[38]

Schölkopf, B. and Smola, A.J., Learning with kernels: support vector machines, regularization, optimization, and beyond. 2002. MIT Press, Cambridge, Mass.

[39]

Yang, J., Yang, J. and Zhang, D., What's wrong with the Fisher criterion?. Pattern Recognition. v35 i11. 2665-2668.

Cited By

Liang ZDing S(2024)Fuzzy Twin Support Vector Machines With Distribution InputsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2023.329650332:1(240-254)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2023.3296503
Chen XGao YYu HWang HCai Y(2023)Driving Style Feature Extraction and Recognition Based on Hyperdimensional Computing and Semi-Supervised Twin Projection Vector MachineIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.329798624:12(13976-13988)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3297986
Yang YXue ZMa JChang X(2023)Robust projection twin extreme learning machines with capped L 1-norm distance metricNeurocomputing10.1016/j.neucom.2022.09.156517:C(229-242)Online publication date: 14-Jan-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.09.156
Show More Cited By

Index Terms

Recursive projection twin support vector machine via within-class variance minimization
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization

Index terms have been assigned to the content through auto-classification.

Recommendations

Twin support vector machine: theory, algorithm and applications

Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and ...
Locality preserving projection least squares twin support vector machine for pattern classification
Abstract
During the last few years, multiple surface classification algorithms, such as twin support vector machine (TWSVM), least squares twin support vector machine (LSTSVM) and least squares projection twin support vector machine (LSPTSVM), have ...
Least squares recursive projection twin support vector machine for classification

In this paper we formulate a least squares version of the recently proposed projection twin support vector machine (PTSVM) for binary classification. This formulation leads to extremely simple and fast algorithm, called least squares projection twin ...

Comments

Information & Contributors

Information

Published In

Copyright © Elsevier Ltd © 2011.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 October 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liang ZDing S(2024)Fuzzy Twin Support Vector Machines With Distribution InputsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2023.329650332:1(240-254)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2023.3296503
Chen XGao YYu HWang HCai Y(2023)Driving Style Feature Extraction and Recognition Based on Hyperdimensional Computing and Semi-Supervised Twin Projection Vector MachineIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.329798624:12(13976-13988)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3297986
Yang YXue ZMa JChang X(2023)Robust projection twin extreme learning machines with capped L 1-norm distance metricNeurocomputing10.1016/j.neucom.2022.09.156517:C(229-242)Online publication date: 14-Jan-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.09.156
Ke TLiao YWu MGe XHuang XZhang CLi J(2023)Maximal margin hyper-sphere SVM for binary pattern classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2022.105615117:PAOnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.engappai.2022.105615
Cui MFan L(2022)A Novel Semismooth Newton Algorithm for PCA-Twin Support Vector MachineProceedings of the 2022 3rd International Conference on Artificial Intelligence in Electronics Engineering10.1145/3512826.3512828(9-15)Online publication date: 11-Jan-2022
https://dl.acm.org/doi/10.1145/3512826.3512828
Liu BHuang RXiao YLiu JWang KLi LChen Q(2022)Adaptive robust Adaboost-based twin support vector machine with universum dataInformation Sciences: an International Journal10.1016/j.ins.2022.07.155609:C(1334-1352)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.ins.2022.07.155
Chen LZhou KJing JFan HLi J(2022)Solution path algorithm for twin multi-class support vector machineExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118361210:COnline publication date: 30-Dec-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.118361
Zhou KZhang QLi J(2022)TSVMPath: Fast Regularization Parameter Tuning Algorithm for Twin Support Vector MachineNeural Processing Letters10.1007/s11063-022-10870-154:6(5457-5482)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s11063-022-10870-1
Tanveer MTiwari AChoudhary RGanaie M(2022)Large-scale pinball twin support vector machinesMachine Language10.1007/s10994-021-06061-z111:10(3525-3548)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10994-021-06061-z
Gupta UGupta D(2022)Least squares structural twin bounded support vector machine on class scatterApplied Intelligence10.1007/s10489-022-04237-153:12(15321-15351)Online publication date: 15-Nov-2022
https://dl.acm.org/doi/10.1007/s10489-022-04237-1
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents