Article

Semi-supervised learning for structured output variables

Authors:

Tobias SchefferAuthors Info & Claims

ICML '06: Proceedings of the 23rd international conference on Machine learning

Pages 145 - 152

https://doi.org/10.1145/1143844.1143863

Published: 25 June 2006 Publication History

Abstract

The problem of learning a mapping between input and structured, interdependent output variables covers sequential, spatial, and relational learning as well as predicting recursive structures. Joint feature representations of the input and output variables have paved the way to leveraging discriminative learners such as SVMs to this class of problems. We address the problem of semi-supervised learning in joint input output spaces. The co-training approach is based on the principle of maximizing the consensus among multiple independent hypotheses; we develop this principle into a semi-supervised support vector learning algorithm for joint input output spaces and arbitrary loss functions. Experiments investigate the benefit of semi-supervised structured models in terms of accuracy and F1 score.

References

[1]

Altun, Y., McAllester, D., & Belkin, M. (2006). Maximum margin semi-supervised learning for structured variables. Adv. in Neural Information Proc. Systems.]]

[2]

Altun, Y., Tsochantaridis, I., & Hofmann, T. (2003). Hidden Markov support vector machines. Proceedings of the International Conference on Machine Learning.]]

[3]

Blum, A., & Mitchell, T. (1998). Combining labeled and unlabeled data with co-training. Proceedings of the Conference on Computational Learning Theory.]]

Digital Library

[4]

Brefeld, U., Büscher, C., & Scheffer, T. (2005). Multi-view discriminative sequential learning. Proceedings of the European Conference on Machine Learning.]]

Digital Library

[5]

Crammer, K., & Singer, Y. (2001). On the algorithmic implementation of multi-class kernel-based vector machines. Journal of Machine Learning Research, 2, 265--292.]]

Digital Library

[6]

Dasgupta, S., Littman, M., & McAllester, D. (2001). PAC generalization bounds for co-training. Advances in Neural Information Processing Systems.]]

[7]

Hardoon, D., Farquhar, J. D. R., Meng, H., Shawe-Taylor, J., & Szedmak, S. (2006). Two view learning: SVM-2K, theory and practice. Advances in Neural Information Processing Systems.]]

[8]

Joachims, T. (1999). Transductive inference for text classification using support vector machines. Proceedings of the International Conference on Machine Learning.]]

Digital Library

[9]

Joachims, T. (2005). A support vector method for multi-variate performance measures. Proceedings of the International Conference on Machine Learning.]]

Digital Library

[10]

Johnson, M. (1999). PCFG models of linguistic tree representations. Computational Linguistics, 24(4), 613--632.]]

Digital Library

[11]

Lafferty, J., Zhu, X., & Liu, Y. (2004). Kernel conditional random fields: representation and clique selection. Proc. of the International Conference on Machine Learning.]]

Digital Library

[12]

Manning, C. D., & Schüütze, H. (1999). Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA.]]

Digital Library

[13]

Nigam, K., & Ghani, R. (2000). Analyzing the effectiveness and applicability of co-training. Proceedings of Information and Knowledge Management.]]

Digital Library

[14]

Taskar, B., Guestrin, C., & Koller, D. (2004). Max-margin Markov networks. Advances in Neural Information Processing Systems.]]

[15]

Tsochantaridis, I., Joachims, T., Hofmann, T., & Altun, Y. (2005). Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6, 1453--1484.]]

Digital Library

Cited By

Lu XChow T(2024)Partial Sequence Labeling With Structured Gaussian ProcessesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319172635:2(2783-2792)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3191726
Lu XChow T(2023)Modeling Sequential Annotations for Sequence Labeling With CrowdsIEEE Transactions on Cybernetics10.1109/TCYB.2021.311770053:4(2335-2345)Online publication date: Apr-2023
https://doi.org/10.1109/TCYB.2021.3117700
Kakandwar SBhushan BKumar A(2023)Integrated machine learning techniques for preserving privacy in Internet of Things (IoT) systemsBlockchain Technology Solutions for the Security of IoT-Based Healthcare Systems10.1016/B978-0-323-99199-5.00012-4(45-75)Online publication date: 2023
https://doi.org/10.1016/B978-0-323-99199-5.00012-4
Show More Cited By

Index Terms

Semi-supervised learning for structured output variables

Recommendations

Manifold regularization in structured output space for semi-supervised structured output prediction

Structured output prediction aims to learn a predictor to predict a structured output from a input data vector . The structured outputs include vector, tree, sequence, etc. We usually assume that we have a training set of input---output pairs to train ...
Maximum margin semi-supervised learning for structured variables
NIPS'05: Proceedings of the 19th International Conference on Neural Information Processing Systems

Many real-world classification problems involve the prediction of multiple inter-dependent variables forming some structural dependency. Recent progress in machine learning has mainly focused on supervised classification of such structured variables. In ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '06: Proceedings of the 23rd international conference on Machine learning

June 2006

1154 pages

ISBN:1595933832

DOI:10.1145/1143844

Program Chairs:
William Cohen,
Andrew Moore

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lu XChow T(2024)Partial Sequence Labeling With Structured Gaussian ProcessesIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.319172635:2(2783-2792)Online publication date: Feb-2024
https://doi.org/10.1109/TNNLS.2022.3191726
Lu XChow T(2023)Modeling Sequential Annotations for Sequence Labeling With CrowdsIEEE Transactions on Cybernetics10.1109/TCYB.2021.311770053:4(2335-2345)Online publication date: Apr-2023
https://doi.org/10.1109/TCYB.2021.3117700
Kakandwar SBhushan BKumar A(2023)Integrated machine learning techniques for preserving privacy in Internet of Things (IoT) systemsBlockchain Technology Solutions for the Security of IoT-Based Healthcare Systems10.1016/B978-0-323-99199-5.00012-4(45-75)Online publication date: 2023
https://doi.org/10.1016/B978-0-323-99199-5.00012-4
He WJiang Z(2022)Semi-Supervised Learning With the EM Algorithm: A Comparative Study Between Unstructured and Structured PredictionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.301903834:6(2912-2920)Online publication date: 1-Jun-2022
https://doi.org/10.1109/TKDE.2020.3019038
Lu XChow T(2022)Weak Disambiguation for Partial Structured Output LearningIEEE Transactions on Cybernetics10.1109/TCYB.2020.300005352:2(1258-1268)Online publication date: Feb-2022
https://doi.org/10.1109/TCYB.2020.3000053
Li ZYang LLi Z(2020)Mixture-Model-Based Graph for Privacy-Preserving Semi-Supervised LearningIEEE Access10.1109/ACCESS.2019.29611268(789-801)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2019.2961126
Osojnik APanov PDžeroski S(2020)Incremental predictive clustering trees for online semi-supervised multi-target regressionMachine Learning10.1007/s10994-020-05918-zOnline publication date: 28-Oct-2020
https://doi.org/10.1007/s10994-020-05918-z
Sun BLi YHu XCheng GChen CLiu Z(2019)Incremental Learning for Transductive SVMs2019 IEEE 14th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)10.1109/ISKE47853.2019.9170294(622-629)Online publication date: Nov-2019
https://doi.org/10.1109/ISKE47853.2019.9170294
Kikkawa RSekiguchi HTsuge ISaito SBise R(2019)Semi-Supervised Learning With Structured Knowledge For Body Hair Detection In Photoacoustic Image2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)10.1109/ISBI.2019.8759249(1411-1415)Online publication date: Apr-2019
https://doi.org/10.1109/ISBI.2019.8759249
Sheikh Hassani MGreen J(2019)Multi-view Co-training for microRNA PredictionScientific Reports10.1038/s41598-019-47399-89:1Online publication date: 29-Jul-2019
https://doi.org/10.1038/s41598-019-47399-8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten