short-paper

A Position-Aware Language Modeling Framework for Extractive Broadcast News Speech Summarization

Authors:

Wen-Lian HsuAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), Volume 16, Issue 4

Article No.: 27, Pages 1 - 13

https://doi.org/10.1145/3099472

Published: 16 August 2017 Publication History

Abstract

Extractive summarization, a process that automatically picks exemplary sentences from a text (or spoken) document with the goal of concisely conveying key information therein, has seen a surge of attention from scholars and practitioners recently. Using a language modeling (LM) approach for sentence selection has been proven effective for performing unsupervised extractive summarization. However, one of the major difficulties facing the LM approach is to model sentences and estimate their parameters more accurately for each text (or spoken) document. We extend this line of research and make the following contributions in this work. First, we propose a position-aware language modeling framework using various granularities of position-specific information to better estimate the sentence models involved in the summarization process. Second, we explore disparate ways to integrate the positional cues into relevance models through a pseudo-relevance feedback procedure. Third, we extensively evaluate various models originated from our proposed framework and several well-established unsupervised methods. Empirical evaluation conducted on a broadcast news summarization task further demonstrates performance merits of the proposed summarization methods.

References

[1]

J. Carbonell and J. Goldstein. 1998. The use of MMR, diversity based reranking for reordering documents and producing summaries. In Proceedings of ACM SIGIR Conference. 335--336.

Digital Library

[2]

A. Celikyilmaz and D. Hakkani-Tur. 2010. A hybrid hierarchical model for multi-document summarization. In Proceedings of the Association for Computational Linguistics. 815--824.

Digital Library

[3]

B. Chen and K.-Y. Chen. 2013. Leveraging relevance cues for language modeling in speech recognition. Inf. Process. Manag. 49, 4, 807--816.

Digital Library

[4]

B. Chen, J.-W. Kuo, and W.-H. Tsai. 2004. Lightly supervised and data-driven approaches to mandarin broadcast news transcription. In Proceedings of the IEEE International Conference on Acoustic Speech Signal Processing. 777--780.

[5]

B. Chen, S.-H. Lin, Y.-M. Chang, and J.-W. Liu. 2013. Extractive speech summarization using evaluation metric-related training criteria. Inf. Process. Manag. 49, 1, 1--12.

Digital Library

[6]

H.-S. Chiu, K.-Y. Chen, and B. Chen. 2014. Leveraging topical and positional cues for language modeling in speech recognition. Multimed. Tools Appl. 72, 2, 1465--1481.

Digital Library

[7]

G. Erkan and D. R. Radev. 2004. LexRank: Graph-based lexical centrality as salience in text summarization. J. Artif. Intelli. Res. 22, 1, 457--479.

Digital Library

[8]

M. A. Fattah and F. Ren. 2009. GA, MR, FFNN, PNN and GMM based models for automatic text summarization. Comput. Speech Lang. 23, 1, 126--144.

Digital Library

[9]

S. Furui, L. Deng, M. Gales, H. Ney, and K. Tokuda. 2012. Fundamental technologies in modern speech recognition. IEEE Signal Process. Mag. 29, 6, 16--17.

[10]

S. Furui, T. Kikuchi, Y. Shinnaka, and C. Hori. 2004. Speech-to-text and speech-to-speech summarization of spontaneous speech. IEEE Trans. Speech Audio Process. 12, 4, 401--408.

[11]

M. Galley. 2006. Skip-chain conditional random field for ranking meeting utterances by importance. In Proceedings of Empirical Methods in Natural Language Processing. 364--372.

Digital Library

[12]

Y. Gong and X. Liu. 2001. Generic text summarization using relevance measure and latent semantic analysis. In Proceedings of ACM SIGIR Conference. 19--25.

Digital Library

[13]

A. Haghighi and L. Vanderwende. 2009. Exploring content models for multi-document summarization. In Proceedings of the NAACL HLT Conference. 362--370.

Digital Library

[14]

M. Kagebäck, O. Mogren, N. Tahmasebi, and D. Dubhashi. 2014. Extractive summarization using continuous vector space models. In Proceedings of Workshop on Continuous Vector Space Models and their Compositionality. 31--39.

[15]

J. Kupie, J. Pedersen, and F. Chen. 1995. A trainable document summarizer. In Proceedings of the ACM SIGIR Conference. 68--73.

Digital Library

[16]

V. Lavrenko and B. Croft. 2001. Relevance-based language models. In Proceedings of the ACM SIGIR Conference. 120--127.

Digital Library

[17]

L.-S. Lee and B. Chen. 2005. Spoken document understanding and organization. IEEE Signal Process. Mag. 22, 5, 42--60.

[18]

H. Lin and J. Bilmes. 2010. Multi-document summarization via budgeted maximization of submodular functions. In Proceedings of the NAACL HLT Conference. 912--920.

Digital Library

[19]

S.-H. Lin, Y.-M. Yeh, and B. Chen. 2011. Leveraging kullback-leibler divergence measures and information-rich cues for speech summarization. Trans. Audio, Speech, Lang. Process. 19, 4, 871--882.

Digital Library

[20]

C.-Y. Lin. 2003. ROUGE: Recall-oriented understudy for gisting evaluation. Retrieved from http://haydn.isi.edu/ROUGE/.

[21]

Y. Liu and D. Hakkani-Tur. 2011. Speech Summarization. Chapter 13, Wiley, New York.

[22]

S.-H. Liu, K.-Y. Chen, B. Chen, H.-M. Wang, H.-C. Yen, and W.-L. Hsu. 2015. Positional language modeling for extractive broadcast news speech summarization. In Proceedings of Interspeech.

[23]

Y. Lv and C.-X. Zhai. 2010. Positional relevance model for pseudo-relevance feedback. In Proceedings of the ACM SIGIR Conference. 579--586.

Digital Library

[24]

I. Mani and M. T. Maybury (Eds.) 1999. Advances in Automatic Text Summarization. MIT Press, Cambridge, MA.

Digital Library

[25]

K. McKeown, J. Hirschberg, M. Galley, and S. Maskey. 2005. From text to speech summarization. In Proceedings of the IEEE International Conference on Acoustic Speech Signal Processing. 997--1000

[26]

A. Nenkova and K. McKeown. 2011. Automatic summarization. Found. Trends Inf. Retr. 5, 2--3, 103--233.

[27]

M. Ostendorf. 2008. Speech technology and information access. IEEE Signal Process. Mag. 25, 3, 150--152.

[28]

H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, and R. Ward. 2015. Deep sentence embedding using the long short term memory network: Analysis and application to information retrieval. In Proceedings of the International Conference on Machine Learning.

[29]

G. Penn and X. Zhu. 2008. A critical reassessment of evaluation baselines for speech summarization. In Proceedings of the Association of Computational Linguistics. 470--478.

[30]

K. Riedhammer, B. Favrec, and D. Hakkani-Tür. 2010. Long story short—global unsupervised models for keyphrase based meeting summarization. Speech Commun. 52, 10, 801--815.

Digital Library

[31]

Torres-Moreno Juan-Manuel. 2014. Automatic Text Summarization. Wiley-ISTE.

[32]

X. Wan and J. Yang. 2008. Multi-document summarization using cluster-based link analysis. In Proceedings of the ACM SIGIR Conference. 299--306.

Digital Library

[33]

H.-M. Wang, B. Chen, J.-W. Kuo, and S.-S. Cheng. 2005. MATBN: A mandarin chinese broadcast news corpus. Int. J. Comput. Linguist. and Chin. Lang. Process. 10, 2, 219--236.

[34]

X. Wang, Y. Yoshida, T. Hirao, K. Sudoh, and M. Nagata. 2015. Summarization based on task-oriented discourse parsing. IEEE/ACM Trans. Audio, Speech, Lang. Process. 23, 8, 1358--1367.

Digital Library

[35]

C.-X. Zhai. 2008. Statistical language models for information retrieval: A critical review. Found. Trends Inf. Retr. 2, 3, 137--213.

Digital Library

[36]

J. Zhang and P. Fung. 2007. Speech summarization without lexical features for Mandarin broadcast news. In Proceedings of the NAACL HLT. 213--216.

Digital Library

[37]

X. Zhu, G. Penn, and F. Rudzicz. 2009. Summarizing multiple spoken documents: Finding evidence from untranscribed audio. In Proceedings of the Joint Conference of ACL and IJCNLP. 549--557.

Digital Library

Cited By

Lo TChen YChen KWang HChen B(2017)Neural relevance-aware query modeling for spoken document retrieval2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)10.1109/ASRU.2017.8268973(466-473)Online publication date: Dec-2017
https://doi.org/10.1109/ASRU.2017.8268973

Index Terms

A Position-Aware Language Modeling Framework for Extractive Broadcast News Speech Summarization
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Enhanced Language Modeling with Proximity and Sentence Relatedness Information for Extractive Broadcast News Summarization

The primary task of extractive summarization is to automatically select a set of representative sentences from a text or spoken document that can concisely express the most important theme of the original document. Recently, language modeling (LM) has ...
Extractive text summarization using clustering-based topic modeling
Abstract
Text summarization is the process of converting the input document into a short form, provided that it preserves the overall meaning associated with it. Primarily, text summarization is achieved in two ways, i.e., abstractive and extractive. ...
Combining relevance language modeling and clarity measure for extractive speech summarization

Extractive speech summarization, which purports to select an indicative set of sentences from a spoken document so as to succinctly represent the most important aspects of the document, has garnered much research over the years. In this paper, we cast ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 16, Issue 4

December 2017

146 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3097269

Editor:
Nianwen Xue
Brandeis University, Waltham, USA

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 August 2017

Accepted: 01 May 2017

Revised: 01 December 2016

Received: 01 April 2016

Published in TALLIP Volume 16, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed

Funding Sources

Ministry of Education, Taiwan
Ministry of Science and Technology, Taiwan
“Aim for the Top University Project” of National Taiwan Normal University (NTNU)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
216
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lo TChen YChen KWang HChen B(2017)Neural relevance-aware query modeling for spoken document retrieval2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)10.1109/ASRU.2017.8268973(466-473)Online publication date: Dec-2017
https://doi.org/10.1109/ASRU.2017.8268973

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents