Abstract
This abstract accompanying a presentation at S+SSPR 2006 explores the use of Support Vector Machines (SVMs) for predicting structured objects like trees, equivalence relations, or alignments. It is shown that SVMs can be extended to these problems in a well-founded way, still leading to a convex quadratic training problem and maintaining the ability to use kernels. While the training problem has exponential size, there is a simple algorithm that allows training in polynomial time. The algorithm is implemented in the SVM-Struct software, and it is discussed how the approach can be applied to problems ranging from natural language parsing to supervised clustering.
Chapter PDF
Similar content being viewed by others
References
Altun, Y., Tsochantaridis, I., Hofmann, T.: Hidden Markov support vector machines. In: International Conference on Machine Learning (ICML) (2003)
Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G.: Discriminative learning of markov random fields for segmentation of 3d scan data. In: International Conference on Computer Vision and Pattern Recognition (CVPR) (2005)
Collins, M.: Discriminative reranking for natural language parsing. In: International Conference on Machine Learning (ICML) (2000)
Collins, M.: Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2002)
Collins, M., Duffy, N.: New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron. In: Conference of the Association for Computational Linguistics (ACL) (2002)
Cortes, C., Vapnik, V.N.: Support–vector networks. Machine Learning Journal 20, 273–297 (1995)
Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research (JMLR) 2, 265–292 (2001)
Durbin, R., Eddy, S., Krogh, A., Mitchison, G.: Biological Sequence Analysis. Cambridge University Press, Cambridge (1998)
Finley, T., Joachims, T.: Supervised clustering with support vector machines. In: International Conference on Machine Learning (ICML) (2005)
Joachims, T.: Learning to align sequences: A maximum-margin approach (online manuscript, August 2003)
Joachims, T.: A support vector method for multivariate performance measures. In: International Conference on Machine Learning (ICML) (2005)
Joachims, T., Galor, T., Elber, R.: Learning to align sequences: A maximum-margin approach. In: Leimkuhler, B. (ed.) New Algorithms for Macromolecular Simulation. LNCS, vol. 49, pp. 57–68. Springer, Heidelberg (2005)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289. Morgan Kaufmann, San Francisco (2001)
Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)
Ng, V., Cardie, C.: Improving machine learning approaches to coreference resolution. In: Annual Meeting of the Assoc. for Comp. Linguistics (ACL) (2002)
Taskar, B., Guestrin, C., Koller, D.: Maximum-margin markov networks. In: Neural Information Processing Systems (NIPS) (2003)
Taskar, B., Klein, D., Collins, M., Koller, D., Manning, C.: Max-margin parsing. In: Empirical Methods in Natural Language Processing (EMNLP) (2004)
Taskar, B., Lacoste-Julien, S., Klein, D.: A discriminative matching approach to word alignment. In: Empirical Methods in Natural Language Processing (EMNLP) (2005)
Taskar, B.: Learning Structured Prediction Models: A Large Margin Approach. PhD thesis, Stanford University (2004)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: International Conference on Machine Learning (ICML) (2004)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research (JMLR) 6, 1453–1484 (2005)
Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)
Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., Vapnik, V.: Kernel dependency estimation. In: Advances in Neural Information Processing Systems 15. MIT Press, Cambridge (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Joachims, T. (2006). Structured Output Prediction with Support Vector Machines. In: Yeung, DY., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds) Structural, Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2006. Lecture Notes in Computer Science, vol 4109. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11815921_1
Download citation
DOI: https://doi.org/10.1007/11815921_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37236-3
Online ISBN: 978-3-540-37241-7
eBook Packages: Computer ScienceComputer Science (R0)