Abstract
This paper presents a summary evaluation method based on a complex network measure. We show how to model summaries as complex networks and establish a possible correlation between summary quality and the measure known as dynamics of the network growth. It is a generic and language independent method that enables easy and fast comparative evaluation of summaries. We evaluate our approach using manually produced summaries and automatic summaries produced by three automatic text summarizers for the Brazilian Portuguese language. The results are in agreement with human intuition and showed to be statistically significant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Antiqueira, L., Nunes, M.G.V., Oliveira Jr., O.N., Costa, L.F.: Modelando Textos como Redes Complexas. In: Anais do III (ed.) Workshop em Tecnologia da Informação e da Linguagem Humana – TIL. São Leopoldo-RS, Brazil, July 22-26 (2005a)
Antiqueira, L., Nunes, M.G.V., Oliveira Jr., O.N., Costa, L.F.: Complex networks in the assessment of text quality. physics/0504033 (2005b)
Barabási, A.L.: Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life. Plume, New York (2003)
Cancho, R.F., Capocci, A., Caldarelli, G.: Spectral methods cluster words of the same class in a syntactic dependency network. cond-mat/0504165 (2005)
Casella, J., Berger, R.L.: Statistical Inference, Duxbury, Belmont, California (2001)
Costa, L.F.: Whats in a name? International Journal of Modern Physics C 15, 371–379 (2004)
Dorogovtsev, S.N., Mendes, J.F.F.: Evolution of networks. Advances in Physics 51(4), 1079–1187 (2002)
Erkan, G., Radev, D.R.: Lexrank: Graph-based centrality as salience in text summarization. Journal of Artificial Intelligence Research – JAIR 22, 457–479 (2004)
Kupiec, J., Pedersen, J., Chen, F.: A trainable document summarizer. In: The Proceedings of the 18th ACM-SIGIR Conference on Research & Development in Information Retrieval, pp. 68–73 (1995)
Lin, C.-Y., Hovy, E.H.: Automatic Evaluation of Summaries Using N-gram Cooccurrence Statistics. In: The Proceedings of Language Technology Conference – HLT, Edmonton, Canada, May 27 - June 1 (2003)
Luhn, H.: The automatic creation of literature abstracts. IBM Journal of Research and Development 2, 159–165 (1958)
Mani, I.: Automatic Summarization. John Benjamin’s Publishing Company (2001)
Mihalcea, R.: Language Independent Extractive Summarization. In: The Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, Michigan (2005)
Miller, G.A.: Wordnet: a dictionary browser. In: The Proceedings of the First International Conference on Information in Data, University of Waterloo, Waterloo (1985)
Módolo, M.: SuPor: um Ambiente para a Exploração de Métodos Extrativos para a Sumarização Automática de Textos em Português. Master thesis. Departamento de Computação, UFSCar (2003)
Motter, A.E., Moura, A.P.S., Lai, Y.C., Dasgupta, P.: Topology of the conceptual network of language. Physical Review E 65, 65102 (2002)
Newman, M.E.J.: The structure and function of complex networks. SIAM Review 45, 167–256 (2003)
Pardo, T.A.S., Rino, L.H.M.: TeMário: Um Corpus para Sumarização Automática de Textos. NILC technical report. NILC-TR-03-09. São Carlos-SP, October, 13p (2003)
Pardo, T.A.S., Rino, L.H.M.: Descrição do GEI - Gerador de Extratos Ideais para o Português do Brasil. NILC technical report. NILC-TR-04-07. São Carlos-SP, August, 10p (2004)
Pardo, T.A.S., Rino, L.H.M., Nunes, M.G.V.: GistSumm: A Summarization Tool Based on a New Extractive Method. In: Mamede, N.J., Baptista, J., Trancoso, I., Nunes, M.d.G.V. (eds.) PROPOR 2003. LNCS, vol. 2721, pp. 210–218. Springer, Heidelberg (2003)
Rino, L.H.M., Pardo, T.A.S., Silla Jr., C.N., Kaestner, C.A., Pombo, M.: A Comparison of Automatic Summarization Systems for Brazilian Portuguese Texts. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 235–244. Springer, Heidelberg (2004)
Santos Jr., E., Mohamed, A.A., Zhao, Q.: Automatic Evaluation of Summaries Using Document Graphs. In: The Proceedings of the Workshop on Text Summarization Branches Out, pp. 66–73 (2004)
Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information Processing and Management 24, 513–523 (1988)
Sigman, M., Cecchi, G.A.: Global Organization of the Wordnet Lexicon. In: The Proceedings of the National Academy of Sciences, vol. 99, pp. 1742–1747 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pardo, T.A.S., Antiqueira, L., Nunes, M.d.G.V., Oliveira, O.N., da Fontoura Costa, L. (2006). Modeling and Evaluating Summaries Using Complex Networks. In: Vieira, R., Quaresma, P., Nunes, M.d.G.V., Mamede, N.J., Oliveira, C., Dias, M.C. (eds) Computational Processing of the Portuguese Language. PROPOR 2006. Lecture Notes in Computer Science(), vol 3960. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11751984_1
Download citation
DOI: https://doi.org/10.1007/11751984_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34045-4
Online ISBN: 978-3-540-34046-1
eBook Packages: Computer ScienceComputer Science (R0)