Abstract
In real-world machine learning scenarios, labeled data is often rare while unlabeled data can be obtained easily. Semi-supervised approaches aim at improving the prediction performance by taking both the labeled as well as the unlabeled part of the data into account. In particular, semi-supervised support vector machines favor decision hyperplanes which lie in a “low-density area” induced by the unlabeled patterns (while still considering the labeled part of the data). The associated optimization problem, however, is of combinatorial nature and, hence, difficult to solve. In this work, we present an efficient implementation of a simple local search strategy that is based on matrix updates of the intermediate candidate solutions. Our experiments on both artificial and real-world data sets indicate that the approach can successfully incorporate unlabeled data in an efficient manner.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Adankon, M., Cheriet, M., Biem, A.: Semisupervised least squares support vector machine. IEEE Transactions on Neural Networks 20(12), 1858–1870 (2009)
Bie, T.D., Cristianini, N.: Convex methods for transduction. In: Adv. in Neural Information Proc. Systems, vol. 16, pp. 73–80. MIT Press, Cambridge (2004)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge Uni. Press, Cambridge (2004)
Chapelle, O., Chi, M., Zien, A.: A continuation method for semi-supervised SVMs. In: Proc. Int. Conf. on Machine Learning, pp. 185–192 (2006)
Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press, Cambridge (2006)
Chapelle, O., Sindhwani, V., Keerthi, S.S.: Branch and bound for semi-supervised support vector machines. In: Adv. in Neural Information Proc. Systems, vol. 19, pp. 217–224. MIT Press, Cambridge (2007)
Chapelle, O., Zien, A.: Semi-supervised classification by low density separation. In: Proc. 10th Int. Workshop on Artificial Intell. and Statistics, pp. 57–64 (2005)
Collobert, R., Sinz, F., Weston, J., Bottou, L.: Trading convexity for scalability. In: Proc. International Conference on Machine Learning, pp. 201–208 (2006)
Gieseke, F., Pahikkala, T., Kramer, O.: Fast evolutionary maximum margin clustering. In: Proc. Int. Conf. on Machine Learning, pp. 361–368 (2009)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2009)
Joachims, T.: Transductive inference for text classification using support vector machines. In: Proc. Int. Conf. on Machine Learning, pp. 200–209 (1999)
Rifkin, R., Yeo, G., Poggio, T.: Regularized least-squares classification. In: Adv. in Learning Theory: Methods, Models and Applications. IOS Press, Amsterdam (2003)
Schölkopf, B., Herbrich, R., Smola, A.J.: A generalized representer theorem. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 416–426. Springer, Heidelberg (2001)
Sindhwani, V., Keerthi, S., Chapelle, O.: Deterministic annealing for semi-supervised kernel machines. In: Proc. Int. Conf. on Machine Learning, pp. 841–848 (2006)
Steinwart, I., Christmann, A.: Support Vector Machines. Springer, New York (2008)
Vapnik, V., Sterin, A.: On structural risk minimization or overall risk in a problem of pattern recognition. Aut. and Remote Control 10(3), 1495–1503 (1977)
Zhang, K., Kwok, J.T., Parvin, B.: Prototype vector machine for large scale semi-supervised learning. In: Proceedings of the International Conference on Machine Learning (2009)
Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning. Morgan and Claypool (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gieseke, F., Kramer, O., Airola, A., Pahikkala, T. (2011). Speedy Local Search for Semi-Supervised Regularized Least-Squares. In: Bach, J., Edelkamp, S. (eds) KI 2011: Advances in Artificial Intelligence. KI 2011. Lecture Notes in Computer Science(), vol 7006. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24455-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-24455-1_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24454-4
Online ISBN: 978-3-642-24455-1
eBook Packages: Computer ScienceComputer Science (R0)