Article

Free access

Beyond n-grams: can linguistic sophistication improve language modeling?

Authors:

Eric Brill,

Radu Florian,

John C. Henderson,

Lidia ManguAuthors Info & Claims

ACL '98/COLING '98: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 1

Pages 186 - 190

https://doi.org/10.3115/980845.980875

Published: 10 August 1998 Publication History

PDF eReader

Abstract

It seems obvious that a successful model of natural language would incorporate a great deal of both linguistic and world knowledge. Interestingly, state of the art language models for speech recognition are based on a very crude linguistic model, namely conditioning the probability of a word on a small fixed number of preceding words. Despite many attempts to incorporate more sophisticated information into the models, the n-gram model remains the state of the art, used in virtually all speech recognition systems. In this paper we address the question of whether there is hope in improving language modeling by incorporating more sophisticated linguistic and world knowledge, or whether the n-grams are already capturing the majority of the information that can be employed.

References

[1]

Brill E., Harris D., Lowe S., Luo X., Rao P., Ristad E and Roukos S. (1996). A hidden tag model for language. In "Research Notes", Center for Language and speech processing. The Johns Hopkins University. Chapter 2.

Google Scholar

[2]

Chelba C., Eagle D., Jelinek F., Jimenez V., Khudanpur S., Mangu L., Printz H., Ristad E., Rosenfeld R., Stolcke A. and Wu D. (1997) Structure and Performance of a Dependency Language Model. In Eurospeech '97. Rhodes, Greece.

Crossref

Google Scholar

[3]

Chomsky N. (1956) Three models for the description of language. IRE Trans. On Inform. Theory. IT-2, 113-124.

Google Scholar

[4]

Della Pietra S., Della Pietra V., Gillett J., Lafferty J., Printz H. and tires L. (1994) Inference and Estimation of a Long-Range Trigram Model. In Proceedings of the Second International Colloquium on Grammatical Inference. Alicante, Spain.

Digital Library

Google Scholar

[5]

Fong E. and Wu D. (1995) Learning restricted probabilistic link grammars. IJCAI Workshop on New Approaches to Learning for Natural Language Processing, Montreal.

Digital Library

Google Scholar

[6]

Golding A and Roth D. (1996) Applying Winnow to Context-Sensitive Spelling Correction. In Proceedings of ICML '96.

Google Scholar

[7]

Jelinek F., Lafferty J. D. and Mercer R. L. (1992) Basic Methods of Probabilistic Context-Free Grammars. In "Speech Recognition and Understanding. Recent Advances, Trends, and Applications", Volume F75, 345--360. Berlin:Springer Verlag.

Google Scholar

[8]

Jurafsky D., Wooters C., Segal J., Stolcke A., Fosler E., Tajchman G. and Morgan N. (1995) Using a stochastic context-free grammar as a language model for speech recognition. In ICASSP '95.

Crossref

Google Scholar

[9]

Knight K and Chandler I. (1994). Automated Postediting of Documents. Proceedings, Twelfth National Conference on Artificial Intelligence.

Digital Library

Google Scholar

[10]

Seymore K. and Rosenfeld R. (1997) Using Story Topics for Language Model Adaptation. In Eurospeech '97. Rhodes, Greece.

Google Scholar

[11]

Stolcke A. (1997) Linguistic Knowledge and Empirical Methods in Speech Recognition. In AI Magazine, Volume 18, 25--31, No.4.

Google Scholar

Cited By

View all

Georgescul MRayner MBouillon P(2010)Spoken language understanding via supervised learning and linguistically motivated featuresProceedings of the Natural language processing and information systems, and 15th international conference on Applications of natural language to information systems10.5555/1894525.1894541(117-128)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.5555/1894525.1894541
Zhou LShi YSears A(2010)Third-party error detection support mechanisms for dictation speech recognitionInteracting with Computers10.1016/j.intcom.2010.02.00222:5(375-388)Online publication date: 1-Sep-2010
https://dl.acm.org/doi/10.1016/j.intcom.2010.02.002
Zhou LShi YZhang DSears A(2006)Discovering Cues to Error Detection in Speech Recognition OutputJournal of Management Information Systems10.2753/MIS0742-122222040922:4(237-270)Online publication date: 1-Apr-2006
https://dl.acm.org/doi/10.2753/MIS0742-1222220409
Show More Cited By

Beyond n-grams: can linguistic sophistication improve language modeling?
1. Computing methodologies
  1. Artificial intelligence
2. Hardware
  1. Power and energy
    1. Power estimation and optimization

Recommendations

Inferring selectional preferences from part-of-speech N-grams
EACL '12: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics

We present the PONG method to compute selectional preferences using part-of-speech (POS) N-grams. From a corpus labeled with grammatical dependencies, PONG learns the distribution of word relations for each POS N-gram. From the much larger but unlabeled ...
A probabilistic model based on n-grams for bilingual word sense disambiguation
MICAI'10: Proceedings of the 9th Mexican international conference on Advances in artificial intelligence: Part I

Word Sense Disambiguation (WSD) is considered one of the most important problems in Natural Language Processing. Even if the problem of WSD is difficult, when we consider its bilingual version, this problem becomes to be much more complex. In this case, ...
Topical N-Grams: Phrase and Topic Discovery, with an Application to Information Retrieval
ICDM '07: Proceedings of the 2007 Seventh IEEE International Conference on Data Mining

Most topic models, such as latent Dirichlet allocation, rely on the bag-of-words assumption. However, word order and phrases are often critical to capturing the meaning of text in many text mining tasks. This paper presents topical n-grams, a topic ...

Comments

Information & Contributors

Information

Published In

ACL '98/COLING '98: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics - Volume 1

August 1998

768 pages

Program Chairs:
Christian Boitet
Université Joseph Fourier, Grenoble, France
,
Pete Whitelock
Sharp Laboratories of Europe Ltd., Oxford, United Kingdom

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 10 August 1998

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
514
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)5

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Georgescul MRayner MBouillon P(2010)Spoken language understanding via supervised learning and linguistically motivated featuresProceedings of the Natural language processing and information systems, and 15th international conference on Applications of natural language to information systems10.5555/1894525.1894541(117-128)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.5555/1894525.1894541
Zhou LShi YSears A(2010)Third-party error detection support mechanisms for dictation speech recognitionInteracting with Computers10.1016/j.intcom.2010.02.00222:5(375-388)Online publication date: 1-Sep-2010
https://dl.acm.org/doi/10.1016/j.intcom.2010.02.002
Zhou LShi YZhang DSears A(2006)Discovering Cues to Error Detection in Speech Recognition OutputJournal of Management Information Systems10.2753/MIS0742-122222040922:4(237-270)Online publication date: 1-Apr-2006
https://dl.acm.org/doi/10.2753/MIS0742-1222220409
Shi YZhou LMooney R(2005)Error detection using linguistic featuresProceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing10.3115/1220575.1220581(41-48)Online publication date: 6-Oct-2005
https://dl.acm.org/doi/10.3115/1220575.1220581
Stoness STetreault JAllen J(2004)Incremental parsing with reference interactionProceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together10.5555/1613148.1613152(18-25)Online publication date: 25-Jul-2004
https://dl.acm.org/doi/10.5555/1613148.1613152

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Inferring selectional preferences from part-of-speech N-grams

A probabilistic model based on n-grams for bilingual word sense disambiguation

Topical N-Grams: Phrase and Topic Discovery, with an Application to Information Retrieval

Comments

Published In

Sponsors

Publisher

Publication History

Qualifiers

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

PDF

eReader

Login options

Full Access

Abstract

References

Cited By

Recommendations

Inferring selectional preferences from part-of-speech N-grams

A probabilistic model based on n-grams for bilingual word sense disambiguation

Topical N-Grams: Phrase and Topic Discovery, with an Application to Information Retrieval

Comments

Information

Published In

Sponsors

Publisher

Publication History

Qualifiers

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations