research-article

SVM optimization: inverse dependence on training set size

Authors:

Shai Shalev-Shwartz and

Nathan SrebroAuthors Info & Claims

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

Pages 928 - 935

https://doi.org/10.1145/1390156.1390273

Published: 05 July 2008 Publication History

Get Access

Abstract

We discuss how the runtime of SVM optimization should decrease as the size of the training data increases. We present theoretical and empirical results demonstrating how a simple subgradient descent approach indeed displays such behavior, at least for linear kernels.

References

[1]

Bartlett, P. L., & Mendelson, S. (2003). Rademacher and gaussian complexities: risk bounds and structural results. J. Mach. Learn. Res., 3, 463--482.

Digital Library

Google Scholar

[2]

Bottou, L. (Web Page). Stochastic gradient descent examples. http://leon.bottou.org/projects/sgd.

Google Scholar

[3]

Bottou, L., & Bousquet, O. (2008). The tradeoffs of large scale learning. Advances in Neural Information Processing Systems 20.

Google Scholar

[4]

Bottou, L., & LeCun, Y. (2004). Large scale online learning. Advances in Neural Information Processing Systems 16.

Google Scholar

[5]

Bottou, L., & Lin, C.-J. (2007). Support vector machine solvers. In L. Bottou, O. Chapelle, D. DeCoste and J. Weston (Eds.), Large scale kernel machines. MIT Press.

Crossref

Google Scholar

[6]

Joachims, T. (1998). Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges and A. Smola (Eds.), Advances in kernel methods---Support Vector learning. MIT Press.

Digital Library

Google Scholar

[7]

Joachims, T. (2006). Training linear svms in linear time. Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD).

Digital Library

Google Scholar

[8]

Lin, C.-J. (2002). A formal analysis of stopping criteria of decomposition methods for support vector machines. IEEE Transactions on Neural Networks, 13, 1045--1052.

Digital Library

Google Scholar

[9]

Platt, J. C. (1998). Fast training of Support Vector Machines using sequential minimal optimization. In B. Schölkopf, C. Burges and A. Smola (Eds.), Advances in kernel methods---Support Vector learning. MIT Press.

Digital Library

Google Scholar

[10]

Shalev-Shwartz, S., Singer, Y., & Srebro, N. (2007). Pegasos: Primal estimated sub-gradient solver for svm. Proceedings of the 24th International Conference on Machine Learning.

Digital Library

Google Scholar

[11]

Smola, A., Vishwanathan, S., & Le, Q. (2008). Bundle methods for machine learning. Advances in Neural Information Processing Systems 20.

Google Scholar

[12]

Sridharan, K. (2008). Fast convergence rates for excess regularized risk with application to SVM. http://ttic.uchicago.edu/~karthik/con.pdf.

Google Scholar

[13]

Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27, 1134--1142.

Digital Library

Google Scholar

Cited By

View all

Beer CManiora JPott C(2024)The Risk of Silence—How the Capital Market Penalizes Social Media PassivityJournal of Information Systems10.2308/ISYS-2023-05938:1(5-38)Online publication date: 20-Feb-2024
https://doi.org/10.2308/ISYS-2023-059
Gentinetta GThomsen ASutter DWoerner S(2024)The complexity of quantum support vector machinesQuantum10.22331/q-2024-01-11-12258(1225)Online publication date: 11-Jan-2024
https://doi.org/10.22331/q-2024-01-11-1225
Li YLi B(2023)Solving the Performance Issues of Epsilon Estimation Method in Differentially Private ERM: Analysis, Solution and EvaluationProceedings of the 2023 2nd International Conference on Networks, Communications and Information Technology10.1145/3605801.3605820(93-99)Online publication date: 16-Jun-2023
https://dl.acm.org/doi/10.1145/3605801.3605820
Show More Cited By

Index Terms

SVM optimization: inverse dependence on training set size

Recommendations

SVM-KM: Speeding SVMs Learning with a priori Cluster Selection and k-Means
SBRN '00: Proceedings of the VI Brazilian Symposium on Neural Networks (SBRN'00)

A procedure called SVM-KM, based on clustering by k-means and to accelerate the training of Support Vector Machine, is the main objective of the present work. During the Support Vector Machines (SVMs) optimization phase, training vectors near the ...
Read More
Multi-Class Wavelet SVM Classifiers Using Quantum Particles Swarm Optimization Algorithm
ISCID '08: Proceedings of the 2008 International Symposium on Computational Intelligence and Design - Volume 01

Based on quantum particle swarm optimization algorithm (QPSO), a novel approach of constructing multi-class least squares wavelet SVM (LS-WSVM) classifiers is presented, regularization parameters and kernel parameters of LS-WSVM can be optimized. ...
Read More
Parameter Optimization of Polynomial Kernel SVM from miniCV
Machine Learning, Optimization, and Data Science
Abstract
Polynomial kernel support vector machine (SVM) is one of the most computational efficient kernel-based SVM. Implementing an iterative optimization method, sequential minimal optimization (SMO) makes it more hardware independent. However, the test ...
Read More

Comments

Information & Contributors

Information

Published In

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

1310 pages

ISBN:9781605582054

DOI:10.1145/1390156

General Chair:
William Cohen
Carnegie Mellon University
,
Program Chairs:
Andrew McCallum
University of Massachusetts Amherst
,
Sam Roweis
University of Toronto and Google

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

ICML '08

Sponsor:

Microsoft Research
Intel
IBM

ICML '08: The 25th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

July 5 - 9, 2008

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

122
Total Citations
View Citations
1,192
Total Downloads

Downloads (Last 12 months)59
Downloads (Last 6 weeks)12

Other Metrics

View Author Metrics

Citations

Cited By

View all

Beer CManiora JPott C(2024)The Risk of Silence—How the Capital Market Penalizes Social Media PassivityJournal of Information Systems10.2308/ISYS-2023-05938:1(5-38)Online publication date: 20-Feb-2024
https://doi.org/10.2308/ISYS-2023-059
Gentinetta GThomsen ASutter DWoerner S(2024)The complexity of quantum support vector machinesQuantum10.22331/q-2024-01-11-12258(1225)Online publication date: 11-Jan-2024
https://doi.org/10.22331/q-2024-01-11-1225
Li YLi B(2023)Solving the Performance Issues of Epsilon Estimation Method in Differentially Private ERM: Analysis, Solution and EvaluationProceedings of the 2023 2nd International Conference on Networks, Communications and Information Technology10.1145/3605801.3605820(93-99)Online publication date: 16-Jun-2023
https://dl.acm.org/doi/10.1145/3605801.3605820
Jandial SKhasbage YPal AKrishnamurthy BBalasubramanian V(2023)RetroKD : Leveraging Past States for Regularizing Targets in Teacher-Student LearningProceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD)10.1145/3570991.3571014(10-18)Online publication date: 4-Jan-2023
https://dl.acm.org/doi/10.1145/3570991.3571014
Tsvieli AWeinberger N(2023)Learning Maximum Margin Channel DecodersIEEE Transactions on Information Theory10.1109/TIT.2023.324386969:6(3597-3626)Online publication date: Jun-2023
https://doi.org/10.1109/TIT.2023.3243869
Gentinetta GSutter DZoufal CFuller BWoerner S(2023)Quantum Kernel Alignment with Stochastic Gradient Descent2023 IEEE International Conference on Quantum Computing and Engineering (QCE)10.1109/QCE57702.2023.00036(256-262)Online publication date: 17-Sep-2023
https://doi.org/10.1109/QCE57702.2023.00036
Toshpulatov MLee WLee S(2023)Talking human face generation: A surveyExpert Systems with Applications10.1016/j.eswa.2023.119678219(119678)Online publication date: Jun-2023
https://doi.org/10.1016/j.eswa.2023.119678
刘蕾(2022)A Stochastic Three-Term Conjugate Gradient Method for Unconstrained Optimization ProblemsAdvances in Applied Mathematics10.12677/AAM.2022.11745211:07(4248-4267)Online publication date: 2022
https://doi.org/10.12677/AAM.2022.117452
Calafiore GFracastoro G(2022) Sparse ℓ 1 - and ℓ 2 -Center Classifiers IEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2020.303683833:3(996-1009)Online publication date: Mar-2022
https://doi.org/10.1109/TNNLS.2020.3036838
Tsvieli AWeinberger N(2022)Learning Maximum Margin Channel Decoders for Non-linear Gaussian Channels2022 IEEE International Symposium on Information Theory (ISIT)10.1109/ISIT50566.2022.9834818(2469-2474)Online publication date: 26-Jun-2022
https://doi.org/10.1109/ISIT50566.2022.9834818
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

SVM-KM: Speeding SVMs Learning with a priori Cluster Selection and k-Means

Multi-Class Wavelet SVM Classifiers Using Quantum Particles Swarm Optimization Algorithm

Parameter Optimization of Polynomial Kernel SVM from miniCV

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

SVM-KM: Speeding SVMs Learning with a priori Cluster Selection and k-Means

Multi-Class Wavelet SVM Classifiers Using Quantum Particles Swarm Optimization Algorithm

Parameter Optimization of Polynomial Kernel SVM from miniCV

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations