article

Experimentally optimal ν in support vector regression for different noise models and parameter settings

Authors:

Athanassia Chalimourda,

Bernhard Schölkopf,

Alex J. SmolaAuthors Info & Claims

Neural Networks, Volume 17, Issue 1

Pages 127 - 141

https://doi.org/10.1016/S0893-6080(03)00209-0

Published: 01 January 2004 Publication History

Abstract

In Support Vector (SV) regression, a parameter v controls the number of Support Vectors and the number of points that come to lie outside of the so-called ε-insensitive tube. For various noise models and SV parameter settings, we experimentally determine the values of v that lead to the lowest generalization error. We find good agreement with the values that had previously been predicted by a theoretical argument based on a the asymptotic efficiency of a simplified model of SV regression. As a side effect of the experiments, valuable information about the generalization behavior of the remaining SVM parameters and their dependencies is gained. The experimental findings are valid even for complex 'real-world' data sets. Based on our results on the role of the v-SVM parameters, we discuss various model selection methods.

References

[1]

Cherkassky, V., & Ma, Y. (2002). Selection of meta-parameters for support vector regression. In Proceedings of the International Conference on Artificial Neural Networks (ICANN), Madrid Spain.

Crossref

Google Scholar

[2]

Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines. Cambridge University Press.

Crossref

Google Scholar

[3]

Duan, K., Keerthi, S. & Poo, A (2001). Evaluation of simple performance measures for tuning SVM hyperparameters (Tech. Rep. No. Control Division Technical Report CD-01-11). Department of Mechanical Engineering, National University of Singapore.

Google Scholar

[4]

Mattera, D., & Haykin, S. (1999). Support vector machines for dynamic reconstruction of a chaotic system. In B. Schöölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods--Support vector learning (pp. 211-241). MIT Press.

Crossref

Google Scholar

[5]

Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion--Determining the number of hidden units for artificial neural networks. IEEE Transactions on Neural Networks, 5, 865-872.

Digital Library

Google Scholar

[6]

Schölkopf, B., Smola, A. J., Williamson, R.C., & Bartlett, P. L. (2000). New support vector algorithms. Neutral Computation, 12(4), 1207-1245.

Digital Library

Google Scholar

[7]

Smola, A. J., Murata, N., Schölkopf, B., & Müüller, K.-R. (1998). Asymptotically optimal choice of ε-loss for support vector machines. In L. Niklasson, M. Bodèn, & T. Ziemke (Eds.), Perspectives in neural computing (pp. 105-110). Proceedings of the Eighth International Conference on Artificial Neural Networks, Springer Verlag.

Google Scholar

[8]

Stitson, M., Gammerman, A., Vapnik, V., Vovk, V., Watkins, C., & Weston, I. (1999). Support vector regression, with ANOVA decomposition kernels. In B. Schöölkopf, C. Burges, & A. Smola (Eds.), Advances in kernel methods--Support vector learning (pp. 285-291). MIT Press.

Crossref

Google Scholar

[9]

Vanderbei, R. J (1994). An interior point code for quadratic programming (Tech. Rep. Nos. TR SOR-94-15, Statistics and Operations Research). Princeton University, NJ.

Google Scholar

[10]

Vapnik, V. (1995). The nature of statistical learning theory. New York: Springer.

Crossref

Google Scholar

[11]

Vapnik, V. (1998). Statistical learning theory. New York: Wiley.

Crossref

Google Scholar

Cited By

View all

Wang JWang XCao Z(2024)Housing price prediction based on multiple regression algorithmsProceedings of the International Conference on Modeling, Natural Language Processing and Machine Learning10.1145/3677779.3677831(317-321)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3677779.3677831
Fiszeder POrzeszko W(2021)Covariance matrix forecasting using support vector regressionApplied Intelligence10.1007/s10489-021-02217-551:10(7029-7042)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s10489-021-02217-5
Carta SRecupero DSaia RStanciu M(2020)A General Approach for Risk Controlled Trading Based on Machine Learning and Statistical ArbitrageMachine Learning, Optimization, and Data Science10.1007/978-3-030-64583-0_44(489-503)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-64583-0_44
Show More Cited By

Index Terms

Recommendations

Asymmetric ?-twin support vector regression

Twin support vector regression (TSVR) aims at finding ??-insensitive up- and down-bound functions for the training points by solving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in the conventional SVR. ...
An overview on twin support vector machines

Twin support vector machines (TWSVM) is based on the idea of proximal SVM based on generalized eigenvalues (GEPSVM), which determines two nonparallel planes by solving two related SVM-type problems, so that its computing cost in the training phase is 1/...
Self-Universum support vector machine

In this paper, for an improved twin support vector machine (TWSVM), we give it a theoretical explanation based on the concept of Universum and then name it Self-Universum support vector machine (SUSVM). For the binary classification problem, SUSVM takes ...

Comments

Information & Contributors

Information

Published In

Neural Networks Volume 17, Issue 1

January 2004

144 pages

ISSN:0893-6080

Issue’s Table of Contents

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 January 2004

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang JWang XCao Z(2024)Housing price prediction based on multiple regression algorithmsProceedings of the International Conference on Modeling, Natural Language Processing and Machine Learning10.1145/3677779.3677831(317-321)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3677779.3677831
Fiszeder POrzeszko W(2021)Covariance matrix forecasting using support vector regressionApplied Intelligence10.1007/s10489-021-02217-551:10(7029-7042)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s10489-021-02217-5
Carta SRecupero DSaia RStanciu M(2020)A General Approach for Risk Controlled Trading Based on Machine Learning and Statistical ArbitrageMachine Learning, Optimization, and Data Science10.1007/978-3-030-64583-0_44(489-503)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-64583-0_44
Zhao FWang JLiu M(2019)CDSVRProceedings of the 2019 2nd International Conference on Service Robotics Technologies10.1145/3325693.3325695(38-42)Online publication date: 22-Mar-2019
https://dl.acm.org/doi/10.1145/3325693.3325695
Wang LWang JWang XZhang Y(2017)3D-LIDAR based branch estimation and intersection location for autonomous vehicles2017 IEEE Intelligent Vehicles Symposium (IV)10.1109/IVS.2017.7995912(1440-1445)Online publication date: 11-Jun-2017
https://dl.acm.org/doi/10.1109/IVS.2017.7995912
Zhang SHu QXie ZMi J(2015)Kernel ridge regression for general noise model with its applicationNeurocomputing10.1016/j.neucom.2014.07.051149:PB(836-846)Online publication date: 3-Feb-2015
https://dl.acm.org/doi/10.1016/j.neucom.2014.07.051
Sumi SZaman MHirose H(2012)A rainfall forecasting method using machine learning models and its application to the Fukuoka city caseInternational Journal of Applied Mathematics and Computer Science10.2478/v10006-012-0062-122:4(841-854)Online publication date: 1-Dec-2012
https://dl.acm.org/doi/10.2478/v10006-012-0062-1
Qu JZuo M(2012)An LSSVR-based algorithm for online system condition prognosticsExpert Systems with Applications: An International Journal10.1016/j.eswa.2011.12.00239:5(6089-6102)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1016/j.eswa.2011.12.002
Priya Rde Souza BRossi Ade Carvalho A(2012)Using genetic algorithms to improve prediction of execution times of ML tasksProceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part I10.1007/978-3-642-28942-2_18(196-207)Online publication date: 28-Mar-2012
https://dl.acm.org/doi/10.1007/978-3-642-28942-2_18
Wu QLaw RWu S(2011)Fault diagnosis of car assembly line based on fuzzy wavelet kernel support vector classifier machine and modified genetic algorithmExpert Systems with Applications: An International Journal10.1016/j.eswa.2010.12.10938:8(9096-9104)Online publication date: 1-Aug-2011
https://dl.acm.org/doi/10.1016/j.eswa.2010.12.109
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Asymmetric ?-twin support vector regression

An overview on twin support vector machines

Self-Universum support vector machine

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations