research-article

HISum: Hyperbolic Interaction Model for Extractive Multi-Document Summarization

Authors:

Liping JingAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 1427 - 1436

https://doi.org/10.1145/3543507.3583197

Published: 30 April 2023 Publication History

Abstract

Extractive summarization helps provide a short description or a digest of news or other web texts. It enhances the reading experience of users, especially when they are reading on small displays (e.g., mobile phones). Matching-based methods are recently proposed for the extractive summarization task, which extracts a summary from a global view via a document-summary matching framework. However, these methods only calculate similarities between candidate summaries and the entire document embeddings, insufficiently capturing interactions between different contextual information in the document to accurately estimate the importance of candidates. In this paper, we propose a new hyperbolic interaction model for extractive multi-document summarization (HISum). Specifically, HISum first learns document and candidate summary representations in the same hyperbolic space to capture latent hierarchical structures and then estimates the importance scores of candidates by jointly modeling interactions between each candidate and the document from global and local views. Finally, the importance scores are used to rank and extract the best candidate as the extracted summary. Experimental results on several benchmarks show that HISum outperforms the state-of-the-art extractive baselines1.

References

[1]

Mehdi Allahyari, Seyed Amin Pouriyeh, Mehdi Assefi, Saeid Safaei, Elizabeth D. Trippe, Juan B. Gutierrez, and Krys J. Kochut. 2017. Text Summarization Techniques: A Brief Survey. CoRR abs/1707.02268 (2017). arXiv:1707.02268http://arxiv.org/abs/1707.02268

[2]

Gary Bécigneul and Octavian-Eugen Ganea. 2019. Riemannian Adaptive Optimization Methods. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum¿id=r1eiqi09K7

[3]

Jane Bromley, James W. Bentz, Léon Bottou, Isabelle Guyon, Yann LeCun, Cliff Moore, Eduard Säckinger, and Roopak Shah. 1993. Signature Verification Using A "Siamese" Time Delay Neural Network. Int. J. Pattern Recognit. Artif. Intell. 7, 4 (1993), 669–688. https://doi.org/10.1142/S0218001493000339

[4]

Jaime G. Carbonell and Jade Goldstein. 2017. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. SIGIR Forum 51, 2 (2017), 209–210. https://doi.org/10.1145/3130348.3130369

Digital Library

[5]

Boli Chen, Yao Fu, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, and Liping Jing. 2021. Probing BERT in Hyperbolic Spaces. In International Conference on Learning Representations. https://openreview.net/forum¿id=17VnwXYZyhH

[6]

Boli Chen, Xin Huang, Lin Xiao, Zixin Cai, and Liping Jing. 2020. Hyperbolic Interaction Model for Hierarchical Multi-Label Classification. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, 7496–7503. https://ojs.aaai.org/index.php/AAAI/article/view/6247

[7]

Yen-Chun Chen and Mohit Bansal. 2018. Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 675–686. https://doi.org/10.18653/v1/P18-1063

[8]

Jianpeng Cheng and Mirella Lapata. 2016. Neural Summarization by Extracting Sentences and Words. In ACL (1). The Association for Computer Linguistics. http://dblp.uni-trier.de/db/conf/acl/acl2016-1.html#0001L16

[9]

Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers), Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 615–621. https://doi.org/10.18653/v1/n18-2097

[10]

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, and Jingjing Liu. 2020. APo-VAE: Text Generation in Hyperbolic Space.CoRR abs/2005.00054 (2020). http://dblp.uni-trier.de/db/journals/corr/corr2005.html#abs-2005-00054

[11]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171–4186. http://dblp.uni-trier.de/db/conf/naacl/naacl2019-1.html#DevlinCLT19

[12]

Günes Erkan and Dragomir R. Radev. 2011. LexRank: Graph-based Lexical Centrality as Salience in Text Summarization. CoRR abs/1109.2128 (2011). arXiv:1109.2128http://arxiv.org/abs/1109.2128

[13]

Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. 2019. Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 1074–1084. https://doi.org/10.18653/v1/p19-1102

[14]

Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018. Hyperbolic Neural Networks. In NeurIPS, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 5350–5360. http://dblp.uni-trier.de/db/conf/nips/nips2018.html#GaneaBH18

[15]

Sebastian Gehrmann, Yuntian Deng, and Alexander M. Rush. 2018. Bottom-Up Abstractive Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4098–4109. https://doi.org/10.18653/v1/d18-1443

[16]

Min Gui, Zhengkun Zhang, Zhenglu Yang, Yanhui Gu, and Guandong Xu. 2018. An Effective Joint Framework for Document Summarization. In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon, France, April 23-27, 2018, Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis (Eds.). ACM, 121–122. https://doi.org/10.1145/3184558.3186959

Digital Library

[17]

Matthias Hamann. 2018. On the tree-likeness of hyperbolic spaces. In Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 164. 345–361.

[18]

C. Hopper and B. Andrews. 2011. The Ricci Flow in Riemannian Geometry. The Ricci flow in Riemannian geometry.

[19]

Wan Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss. In ACL (1). Association for Computational Linguistics, 132–141. http://dblp.uni-trier.de/db/conf/acl/acl2018-1.html#SunHLLMT18

[20]

Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015. LCSTS: A Large Scale Chinese Short Text Summarization Dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). The Association for Computational Linguistics, 1967–1972. https://doi.org/10.18653/v1/d15-1229

[21]

Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan V. Oseledets, and Victor S. Lempitsky. 2020. Hyperbolic Image Embeddings. In CVPR. IEEE, 6417–6427. http://dblp.uni-trier.de/db/conf/cvpr/cvpr2020.html#KhrulkovMUOL20

[22]

Logan Lebanoff, Kaiqiang Song, and Fei Liu. 2018. Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, 4131–4141. https://doi.org/10.18653/v1/d18-1446

[23]

C. Y. Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS). Barcelona, Spain.

[24]

Yang Liu. 2019. Fine-tune BERT for Extractive Summarization.CoRR abs/1903.10318 (2019). http://dblp.uni-trier.de/db/journals/corr/corr1903.html#abs-1903-10318

[25]

Yizhu Liu, Qi Jia, and Kenny Q. Zhu. 2021. Keyword-aware Abstractive Summarization by Extracting Set-level Intermediate Summaries. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia (Eds.). ACM / IW3C2, 3042–3054. https://doi.org/10.1145/3442381.3449906

Digital Library

[26]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach.CoRR abs/1907.11692 (2019). http://dblp.uni-trier.de/db/journals/corr/corr1907.html#abs-1907-11692

[27]

Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, and Yee Whye Teh. 2019. Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 12544–12555. https://proceedings.neurips.cc/paper/2019/hash/0ec04cb3912c4f08874dd03716f80df1-Abstract.html

[28]

Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing Order into Text. In EMNLP. ACL, 404–411. http://dblp.uni-trier.de/db/conf/emnlp/emnlp2004.html#MihalceaT04

[29]

Ramesh Nallapati, Bowen Zhou, and Mingbo Ma. 2016. Classify or Select: Neural Architectures for Extractive Document Summarization.CoRR abs/1611.04244 (2016). http://dblp.uni-trier.de/db/journals/corr/corr1611.html#NallapatiZM16

[30]

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Ranking Sentences for Extractive Summarization with Reinforcement Learning. In NAACL-HLT, Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 1747–1759. http://dblp.uni-trier.de/db/conf/naacl/naacl2018-1.html#NarayanCL18

[31]

Maximilian Nickel and Douwe Kiela. 2017. Poincaré Embeddings for Learning Hierarchical Representations. In NIPS. 6338–6347. http://dblp.uni-trier.de/db/conf/nips/nips2017.html#NickelK17

[32]

Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A Deep Reinforced Model for Abstractive Summarization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum¿id=HkAClQgA-

[33]

Haggai Roitman, Guy Feigenblat, Doron Cohen, Odellia Boni, and David Konopnicki. 2020. Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-Focused Extractive Summarization. In WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 2577–2584. https://doi.org/10.1145/3366423.3380009

Digital Library

[34]

Christopher De Sa, Albert Gu, Christopher Ré, and Frederic Sala. 2018. Representation Tradeoffs for Hyperbolic Embeddings. CoRR abs/1804.03329 (2018). arxiv:1804.03329http://arxiv.org/abs/1804.03329

[35]

Rik Sarkar. 2011. Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane. In Graph Drawing(Lecture Notes in Computer Science, Vol. 7034), Marc J. van Kreveld and Bettina Speckmann (Eds.). Springer, 355–366. http://dblp.uni-trier.de/db/conf/gd/gd2011.html#Sarkar11

[36]

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get To The Point: Summarization with Pointer-Generator Networks. In ACL (1), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1073–1083. http://dblp.uni-trier.de/db/conf/acl/acl2017-1.html#SeeLM17

[37]

Jiaxin Shi, Chen Liang, Lei Hou, Juanzi Li, Zhiyuan Liu, and Hanwang Zhang. 2019. DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization. In AAAI. AAAI Press, 6999–7006. http://dblp.uni-trier.de/db/conf/aaai/aaai2019.html#ShiLHL0Z19

[38]

Mingyang Song, Yi Feng, and Liping Jing. 2022. Hyperbolic Relevance Matching for Neural Keyphrase Extraction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, Marine Carpuat, Marie-Catherine de Marneffe, and Iván Vladimir Meza Ruíz (Eds.). Association for Computational Linguistics, 5710–5720. https://doi.org/10.18653/v1/2022.naacl-main.419

[39]

Mingyang Song, Yi Feng, and Liping Jing. 2022. A Preliminary Exploration of Extractive Multi-Document Summarization in Hyperbolic Space. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (Atlanta, GA, USA) (CIKM ’22). Association for Computing Machinery, New York, NY, USA, 4505–4509. https://doi.org/10.1145/3511808.3557538

Digital Library

[40]

Alexandru Tifrea, Gary Bécigneul, and Octavian-Eugen Ganea. 2019. Poincare Glove: Hyperbolic Word Embeddings. In ICLR (Poster). OpenReview.net. http://dblp.uni-trier.de/db/conf/iclr/iclr2019.html#TifreaBG19

[41]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998–6008. http://dblp.uni-trier.de/db/conf/nips/nips2017.html#VaswaniSPUJGKP17

[42]

Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. 2016. Order Matters: Sequence to sequence for sets. In ICLR (Poster), Yoshua Bengio and Yann LeCun (Eds.). http://dblp.uni-trier.de/db/conf/iclr/iclr2016.html#VinyalsBK15

[43]

Danqing Wang, Pengfei Liu, Yining Zheng, Xipeng Qiu, and Xuanjing Huang. 2020. Heterogeneous Graph Neural Networks for Extractive Document Summarization. In ACL, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6209–6219. http://dblp.uni-trier.de/db/conf/acl/acl2020.html#WangLZQH20

[44]

Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral Multi-Perspective Matching for Natural Language Sentences. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, Carles Sierra (Ed.). ijcai.org, 4144–4150. https://doi.org/10.24963/ijcai.2017/579

[45]

Chao Zhao, Tenghao Huang, Somnath Basu Roy Chowdhury, Muthu Kumar Chandrasekaran, Kathleen R. McKeown, and Snigdha Chaturvedi. 2022. Read Top News First: A Document Reordering Approach for Multi-Document News Summarization. CoRR abs/2203.10254 (2022). https://doi.org/10.48550/arXiv.2203.10254 arXiv:2203.10254

[46]

Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive Summarization as Text Matching. In ACL, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6197–6208. http://dblp.uni-trier.de/db/conf/acl/acl2020.html#ZhongLCWQH20

[47]

Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural Document Summarization by Jointly Learning to Score and Select Sentences. In ACL (1), Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, 654–663. http://dblp.uni-trier.de/db/conf/acl/acl2018-1.html#ZhaoZWYHZ18

[48]

Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, and Qun Liu. 2020. HyperText: Endowing FastText with Hyperbolic Geometry. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020(Findings of ACL, Vol. EMNLP 2020), Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 1166–1171. https://doi.org/10.18653/v1/2020.findings-emnlp.104

Cited By

Song MLiu HJing LFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Diversity in Unsupervised Keyphrase Extraction with Determinantal Point ProcessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615141(4294-4299)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615141
Li XWang JJing L(2023)Interpretable Image Recognition by Screening Class-Specific and Class-Shared PrototypesArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44210-0_32(397-408)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44210-0_32

Index Terms

HISum: Hyperbolic Interaction Model for Extractive Multi-Document Summarization
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction

Recommendations

Exploring events and distributed representations of text in multi-document summarization

We explore an event detection framework to improve multi-document summarizationWe use distributed representations of text to address different lexical realizationsSummarization is based on the hierarchical combination of single-document summariesWe ...
Research on Multi-document Summarization Based on LDA Topic Model
IHMSC '14: Proceedings of the 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 02

Compared with VSM (Vector Space Model) and graph-ranking models, LDA (Latent Dirichlet Allocation) Model can discover latent topics in the corpus and latent topics are beneficial to use sentence-ranking mechanisms to form a good summary. In the paper, ...
Latent dirichlet allocation based multi-document summarization
AND '08: Proceedings of the second workshop on Analytics for noisy unstructured text data

Extraction based Multi-Document Summarization Algorithms consist of choosing sentences from the documents using some weighting mechanism and combining them into a summary. In this article we use Latent Dirichlet Allocation to capture the events being ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
258
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)12

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Song MLiu HJing LFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Diversity in Unsupervised Keyphrase Extraction with Determinantal Point ProcessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615141(4294-4299)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615141
Li XWang JJing L(2023)Interpretable Image Recognition by Screening Class-Specific and Class-Shared PrototypesArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44210-0_32(397-408)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44210-0_32

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents