Article

Learning and inference over constrained output

Authors:

Vasin Punyakanok,

Dan Roth,

Wen-tau Yih,

Dav ZimakAuthors Info & Claims

IJCAI'05: Proceedings of the 19th international joint conference on Artificial intelligence

Pages 1124 - 1129

Published: 30 July 2005 Publication History

Abstract

We study learning structured output in a discriminative framework where values of the output variables are estimated by local classifiers. In this framework, complex dependencies among the output variables are captured by constraints and dictate which global labels can be inferred. We compare two strategies, learning independent classifiers and inference based training, by observing their behaviors in different conditions. Experiments and theoretical justification lead to the conclusion that using inference based learning is superior when the local classifiers are difficult to learn but may require many examples before any discernible difference can be observed.

References

[1]

Anthony and Bartlett, 1999} M. Anthony and P. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.

Digital Library

Google Scholar

[2]

{Carreras and Màrquez, 2003} X. Carreras and Lluís Màrquez. On-line learning via global feedback for phrase recognition. In Advances in Neural Information Processing Systems 15, 2003.

Google Scholar

[3]

{Collins, 2002} M. Collins. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of EMNLP, 2002.

Digital Library

Google Scholar

[4]

{Har-Peled et al., 2003} S. Har-Peled, D. Roth, and D. Zimak. Constraint classification: A new approach to multiclass classification and ranking. In Advances in Neural Information Processing Systems 15, 2003.

Google Scholar

[5]

{Kingsbury and Palmer, 2002} P. Kingsbury and M. Palmer. From Treebank to PropBank. In Proceedings of LREC, 2002.

Google Scholar

[6]

{Lafferty et al., 2001} J. Lafferty, A. McCallum, and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. ICML-01, pages 282-289, 2001.

Digital Library

Google Scholar

[7]

{Marcus et al., 1993} M. P. Marcus, B. Santorini, and M. Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330, June 1993.

Digital Library

Google Scholar

[8]

{McCallum et al., 2000} A. McCallum, D. Freitag, and F. Pereira. Maximum entropy Markov models for information extraction and segmentation. In Proceedings of ICML-00, Stanford, CA, 2000.

Digital Library

Google Scholar

[9]

{Punyakanok and Roth, 2001} V. Punyakanok and D. Roth. The use of classifiers in sequential inference. In Advances in Neural Information Processing Systems 13, 2001.

Google Scholar

[10]

{Punyakanok et al., 2004} V. Punyakanok, D. Roth, W. Yih, and D. Zimak. Semantic role labeling via integer linear programming inference. In Proceedings of COLING-04, 2004.

Digital Library

Google Scholar

[11]

{Ramshaw and Marcus, 1995} L. A. Ramshaw and M. P. Marcus. Text chunking using transformation-based learning. In Proceedings of the Third Annual Workshop on Very Large Corpora, 1995.

Google Scholar

[12]

{Roth and Yih, 2004} D. Roth and W. Yih. A linear programming formulation for global inference in natural language tasks. In Proceedings of CoNLL-2004, pages 1-8, 2004.

Google Scholar

[13]

{Taskar et al., 2004} B. Taskar, C. Guestrin, and D. Koller. Maxmargin markov networks. In Advances in Neural Information Processing Systems 16, 2004.

Google Scholar

Cited By

View all

Wang KHe HNguyen TKumar PRoth DKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)On regularization and inference with label constraintsProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619893(35740-35762)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619893
Deshpande PMarathe KDe ASarawagi SLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Long Horizon Forecasting with Temporal Point ProcessesProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441740(571-579)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441740
Sachan MDubey AMitchell TRoth DXing E(2018)Learning pipelines with limited data and domain knowledgeProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3326943.3326957(140-151)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3326943.3326957
Show More Cited By

Learning and inference over constrained output
1. Computing methodologies

Recommendations

Learning and inference order in structured output elements classification
ACIIDS'12: Proceedings of the 4th Asian conference on Intelligent Information and Database Systems - Volume Part I

In the paper three learning and inference ordering approaches in the method for structured output classification are presented. As it was previously presented by authors, classification of single element in output structure can be performed by ...
Constrained Bayesian inference for low rank multitask learning
UAI'13: Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence

We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of ...
From inductive inference to algorithmic learning theory
Abstract
We present two phenomena which were discovered in pure recursion-theoretic inductive inference, namely inconsistent learning (learing strategies producing apparently “senseless” hypotheses can solve problems unsolvable by “reasonable” learning ...

Comments

Information & Contributors

Information

Published In

IJCAI'05: Proceedings of the 19th international joint conference on Artificial intelligence

July 2005

1775 pages

Publisher

Morgan Kaufmann Publishers Inc.

San Francisco, CA, United States

Publication History

Published: 30 July 2005

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang KHe HNguyen TKumar PRoth DKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)On regularization and inference with label constraintsProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619893(35740-35762)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619893
Deshpande PMarathe KDe ASarawagi SLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Long Horizon Forecasting with Temporal Point ProcessesProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441740(571-579)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441740
Sachan MDubey AMitchell TRoth DXing E(2018)Learning pipelines with limited data and domain knowledgeProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3326943.3326957(140-151)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3326943.3326957
Kordjamshidi PRoth DWu H(2015)SaulProceedings of the 24th International Conference on Artificial Intelligence10.5555/2832415.2832505(1844-1851)Online publication date: 25-Jul-2015
https://dl.acm.org/doi/10.5555/2832415.2832505
Martins AFigueiredo MAguiar PSmith NXing E(2015)AD3The Journal of Machine Learning Research10.5555/2789272.278928816:1(495-545)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.5555/2789272.2789288
Kordjamshidi PMoens M(2015)Global machine learning for spatial ontology populationWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2014.06.00130:C(3-21)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.websem.2014.06.001
Smith JEisner JChu-Carroll J(2012)Unsupervised learning on an approximate corpusProceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies10.5555/2382029.2382048(131-141)Online publication date: 3-Jun-2012
https://dl.acm.org/doi/10.5555/2382029.2382048
Srikumar VRoth DMerlo PBarzilay RJohnson M(2011)A joint model for extended semantic role labelingProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145447(129-139)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145447
Spitkovsky VAlshawi HJurafsky DGoldwater SManning C(2011)PunctuationProceedings of the Fifteenth Conference on Computational Natural Language Learning10.5555/2018936.2018939(19-28)Online publication date: 23-Jun-2011
https://dl.acm.org/doi/10.5555/2018936.2018939
Pasternack JRoth DJoshi AHuang CJurafsky D(2010)Knowing what to believe (when you already know something)Proceedings of the 23rd International Conference on Computational Linguistics10.5555/1873781.1873880(877-885)Online publication date: 23-Aug-2010
https://dl.acm.org/doi/10.5555/1873781.1873880
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Learning and inference order in structured output elements classification

Constrained Bayesian inference for low rank multitask learning

From inductive inference to algorithmic learning theory

Comments

Published In

Sponsors

Publisher

Publication History

Qualifiers

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

Abstract

References

Cited By

Recommendations

Learning and inference order in structured output elements classification

Constrained Bayesian inference for low rank multitask learning

From inductive inference to algorithmic learning theory

Comments

Information

Published In

Sponsors

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations