research-article

Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues

Authors:

Olvi L. Mangasarian,

Edward W. WildAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 28, Issue 1

Pages 69 - 74

https://doi.org/10.1109/TPAMI.2006.17

Published: 01 January 2006 Publication History

Abstract

A new approach to support vector machine (SVM) classification is proposed wherein each of two data sets are proximal to one of two distinct planes that are not parallel to each other. Each plane is generated such that it is closest to one of the two data sets and as far as possible from the other data set. Each of the two nonparallel proximal planes is obtained by a single MATLAB command as the eigenvector corresponding to a smallest eigenvalue of a generalized eigenvalue problem. Classification by proximity to two distinct nonlinear surfaces generated by a nonlinear kernel also leads to two simple generalized eigenvalue problems. The effectiveness of the proposed method is demonstrated by tests on simple examples as well as on a number of public data sets. These examples show the advantages of the proposed approach in both computation time and test set correctness.

References

[1]

E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK User's Guide, third ed., Philadelphia: SIAM, 1999,

Digital Library

[2]

K.P. Bennett and O.L. Mangasarian, “Robust Linear Programming Discrimination of Two Linearly Inseparable Sets,” Optimization Methods and Software, vol. 1, pp. 23-34, 1992.

[3]

P.S. Bradley and O.L. Mangasarian, “k-Plane Clustering,” J. Global Optimization, vol. 16, pp. 23-32, 2000,

Digital Library

[4]

N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, Mass.: Cambridge Univ. Press, 2000.

Digital Library

[5]

J.W. Demmel, Applied Numerical Linear Algebra. Philadelphia: SIAM, 1997.

Digital Library

[6]

T. Evgeniou, M. Pontil, and T. Poggio, “Regularization Networks and Support Vector Machines,” Advances in Computational Math., vol. 13, pp. 1-50, 2000.

[7]

G. Fung and O.L. Mangasarian, “Proximal Support Vector Machine Classifiers,” Proc. Knowledge Discovery and Data Mining, F. Provost and R. Srikant, eds., pp. 77-86, 2001,

Digital Library

[8]

G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. Baltimore: The John Hopkins Univ. Press,. 1996.

Digital Library

[9]

T. Joachims, “Making Large-Scale Support Vector Machine Learning Practical,” Advances in Kernel Methods— Support Vector Learning, B. Schölkopf, C.J.C. Burges, and A.J. Smola, eds., pp. 169-184, Cambridge, Mass.: MIT Press, 1999.

Digital Library

[10]

M. Kojima, S. Mizuno, T. Noma, and A. Yoshise, A Unified Approach to Interior Point Algorithms for Linear Complementarity Problems. Berlin: Springer-Verlag, 1991.

[11]

R.H. Leary, J.B. Rosen, and P. Jambeck, “An Optimal Structure-Discrminative Amino Acid Index for Protein Recognition,” Biophysical J., vol. 86, pp. 411-419, 2004.

[12]

Y.-J. Lee and O.L. Mangasarian, “RSVM: Reduced Support Vector Machines,” Proc. First SIAM Int'l Conf. Data Mining, Apr. 2001,

[13]

O.L. Mangasarian, “Least Norm Solution of Non-Monotone Complementarity Problems,” Functional Analysis, Optimization and Mathematical Economics, pp. 217-221, New York: Oxford Univ. Press, 1990.

[14]

O.L. Mangasarian, “Arbitrary-Norm Separating Plane,” Operations Research Letters, vol. 24, pp. 15-23, 1999,

Digital Library

[15]

O.L. Mangasarian, “Generalized Support Vector Machines,” Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, eds., pp. 135-146, Cambridge, Mass.: MIT Press, 2000,

[16]

O.L. Mangasarian and R.R. Meyer, “Nonlinear Perturbation of Linear Programs,” SIAM J. Control and Optimization, vol. 17, no. 6, pp. 745-752, Nov. 1979.

Digital Library

[17]

MATLAB, User's Guide, The MathWorks, Inc., 1994-2001,

[18]

T.M. Mitchell, Machine Learning. Boston: McGraw-Hill, 1997.

Digital Library

[19]

P.M. Murphy and D.W. Aha, “UCI Machine Learning Repository,” 1992,

[20]

D.R. Musicant, “NDC: Normally Distributed Clustered Datasets,” 1998,

[21]

S. Odewahn, E. Stockwell, R. Pennington, R. Humphreys, and W. Zumach, “Automated Star/Galaxy Discrimination with Neural Networks,” Astronomical J., vol. 103, no. 1, pp. 318-331, 1992.

[22]

B.N. Parlett, The Symmetric Eigenvalue Problem. Philadelphia: SIAM, 1998.

Digital Library

[23]

B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, Mass.: MIT Press, 2002.

[24]

Scilab, free scientific software package, 1990-2004,

[25]

J.A.K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific Publishing Co., 2002.

[26]

A.N. Tikhonov and V.Y. Arsen, Solutions of Ill-Posed Problems. New York: John Wiley&Sons, 1977.

[27]

V.N. Vapnik, The Nature of Statistical Learning Theory, second ed. New York: Springer, 2000.

Digital Library

Cited By

Qu SDe Leone RHuang M(2024)Sparse Learning for Linear Twin Parameter-margin Support Vector MachineProceedings of the 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning10.1145/3654823.3654833(50-55)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3654823.3654833
Kumari ATanveer MLin C(2024)Class Probability and Generalized Bell Fuzzy Twin SVM for Imbalanced DataIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336693632:5(3037-3048)Online publication date: 19-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3366936
Liang ZDing S(2024)Fuzzy Twin Support Vector Machines With Distribution InputsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2023.329650332:1(240-254)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2023.3296503
Show More Cited By

Index Terms

Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
2. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results

Recommendations

Proximal support vector machine using local information

Instead of standard support vector machines (SVMs) that classify points to one of two disjoint half-spaces by solving a quadratic program, the plane classifier GEPSVM (proximal SVM classification via generalized eigenvalues) classifies points by ...
Fuzzy multi-category proximal support vector classification via generalized eigenvalues

Given a dataset, where each point is labeled with one of M labels, we propose a technique for multi-category proximal support vector classification via generalized eigenvalues (MGEPSVMs). Unlike Support Vector Machines that classify points by assigning ...
Least squares twin support vector machines for pattern classification

In this paper we formulate a least squares version of the recently proposed twin support vector machine (TSVM) for binary classification. This formulation leads to extremely simple and fast algorithm for generating binary classifiers based on two non-...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 28, Issue 1

January 2006

175 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2006.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 January 2006

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

228
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qu SDe Leone RHuang M(2024)Sparse Learning for Linear Twin Parameter-margin Support Vector MachineProceedings of the 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning10.1145/3654823.3654833(50-55)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3654823.3654833
Kumari ATanveer MLin C(2024)Class Probability and Generalized Bell Fuzzy Twin SVM for Imbalanced DataIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.336693632:5(3037-3048)Online publication date: 19-Feb-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3366936
Liang ZDing S(2024)Fuzzy Twin Support Vector Machines With Distribution InputsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2023.329650332:1(240-254)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2023.3296503
Liang GLai ZKong H(2024)Support vector machine with discriminative low‐rank embeddingCAAI Transactions on Intelligence Technology10.1049/cit2.123299:5(1249-1262)Online publication date: 2-Apr-2024
https://dl.acm.org/doi/10.1049/cit2.12329
Lai ZLiang GZhou JKong HLu Y(2024)A joint learning framework for optimal feature extraction and multi-class SVMInformation Sciences: an International Journal10.1016/j.ins.2024.120656671:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120656
Shao WShao YLi C(2024)A novel regularization method for decorrelation learning of non-parallel hyperplanesInformation Sciences: an International Journal10.1016/j.ins.2024.120461667:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120461
Yan HLiu YLi YYe QYu DQi Y(2024)Robust GEPSVM classifierInformation Sciences: an International Journal10.1016/j.ins.2023.119986657:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.ins.2023.119986
Liu CQian Q(2024)Twin proximal support vector regression with heteroscedastic Gaussian noiseExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123840250:COnline publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123840
Lou CXie X(2024)Multi-view universum support vector machines with insensitive pinball lossExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123480248:COnline publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123480
Moosaei HBazikar FHladík MPardalos P(2024)Sparse least-squares Universum twin bounded support vector machine with adaptive L p-norms and feature selectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123378248:COnline publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123378
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents