research-article

Free access

Title-guided encoding for keyphrase generation

AUTHORs:

Michael R. LyuAuthors Info & Claims

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

Article No.: 769, Pages 6268 - 6275

https://doi.org/10.1609/aaai.v33i01.33016268

Published: 27 January 2019 Publication History

PDF eReader Publisher Site

Abstract

Keyphrase generation (KG) aims to generate a set of keyphrases given a document, which is a fundamental task in natural language processing (NLP). Most previous methods solve this problem in an extractive manner, while recently, several attempts are made under the generative setting using deep neural networks. However, the state-of-the-art generative methods simply treat the document title and the document main body equally, ignoring the leading role of the title to the overall document. To solve this problem, we introduce a new model called Title-Guided Network (TG-Net) for automatic keyphrase generation task based on the encoder-decoder architecture with two new features: (i) the title is additionally employed as a query-like input, and (ii) a title-guided encoder gathers the relevant information from the title to each word in the document. Experiments on a range of KG datasets demonstrate that our model outperforms the state-of-the-art models with a large margin, especially for documents with either very low or very high title length ratios.

References

[1]

Bahdanau, D.; Cho, K.; and Bengio, Y. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

[2]

Berend, G. 2011. Opinion expression mining by exploiting keyphrase extraction. In IJCNLP, 1162-1170.

[3]

Cho, K.; van Merrienboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP, 1724-1734.

[4]

Dauphin, Y. N.; Fan, A.; Auli, M.; and Grangier, D. 2017. Language modeling with gated convolutional networks. In ICML, 933-941.

[5]

Florescu, C., and Caragea, C. 2017. A position-biased pagerank algorithm for keyphrase extraction. In AAAI Student Abstracts, 4923-4924.

[6]

Florescu, C., and Jin, W. 2018. Learning feature representations for keyphrase extraction. In AAAI Student Abstracts.

[7]

Gao, Y.; Bing, L.; Li, P.; King, I.; and Lyu, M. R. 2018. Generating distractors for reading comprehension questions from real examinations. arXiv preprint arXiv: 1809.02768.

[8]

Gehring, J.; Auli, M.; Grangier, D.; Yarats, D.; and Dauphin, Y. N. 2017. Convolutional sequence to sequence learning. In ICML, 1243-1252.

[9]

Gollapalli, S. D.; Li, X.; and Yang, P. 2017. Incorporating expert knowledge into keyphrase extraction. In AAAI, 3180-3187.

[10]

Gu, J.; Lu, Z.; Li, H.; and Li, V. O. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In ACL, volume 1, 1631-1640.

[11]

Hai, Y., and Lu, W. 2018. Semi-supervised learning for neural keyphrase generation. arXiv preprint arXiv: 1808.06773.

[12]

Hulth, A., and Megyesi, B. B. 2006. A study on automatically extracted keywords in text categorization. In COLING and ACL, 537-544.

[13]

Hulth, A. 2003. Improved automatic keyword extraction given more linguistic knowledge. In EMNLP, 216-223.

[14]

Jones, S., and Staveley, M. S. 1999. Phrasier: a system for interactive document retrieval using keyphrases. In SIGIR, 160-167.

[15]

Jun, C.; Xiaoming, Z.; Yu, W.; Zhao, Y.; and Zhoujun, L. 2018. Keyphrase generation with correlation constraints. arXiv preprint arXiv: 1808.07185.

[16]

Kim, S. N.; Medelyan, O.; Kan, M.-Y.; and Baldwin, T. 2010. Semeval-2010 task 5 : Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, 21-26.

[17]

Kingma, D. P., and Ba, J. 2015. Adam: A method for stochastic optimization. In ICLR.

[18]

Klein, G.; Kim, Y.; Deng, Y.; Senellart, J.; and Rush, A. 2017. Opennmt: Open-source toolkit for neural machine translation. In ACL System Demonstrations, 67-72.

[19]

Krapivin, M.; Autaeu, A.; and Marchese, M. 2009. Large dataset for keyphrases extraction. Technical report, University of Trento.

[20]

Li, D.; Li, S.; Li, W.; Wang, W.; and Qu, W. 2010. A semisupervised key phrase extraction approach: Learning from title phrases through a document semantic network. In ACL Short, 296-300.

[21]

Liu, Z.; Chen, X.; Zheng, Y.; and Sun, M. 2011. Automatic keyphrase extraction by bridging vocabulary gap. In CoNLL, 135-144.

[22]

Luan, Y.; Ostendorf, M.; and Hajishirzi, H. 2017. Scientific information extraction with semi-supervised neural tagging. In EMNLP, 2641-2651.

[23]

Luong, T.; Pham, H.; and Manning, C. D. 2015. Effective approaches to attention-based neural machine translation. In EMNLP, 1412-1421.

[24]

Manning, C.; Surdeanu, M.; Bauer, J.; Finkel, J.; Bernard, S.; and McClosky, D. 2014. The Stanford corenlp natural language processing toolkit. In ACL System Demonstrations, 55-60.

[25]

Medelyan, O.; Frank, E.; and Witten, I. H. 2009. Human-competitive tagging using automatic keyphrase extraction. In EMNLP, 1318-1327.

Digital Library

[26]

Meng, R.; Zhao, S.; Han, S.; He, D.; Brusilovsky, P.; and Chi, Y. 2017. Deep keyphrase generation. In ACL, volume 1, 582-592.

[27]

Mihalcea, R., and Tarau, P. 2004. Textrank: Bringing order into text. In EMNLP.

[28]

Nema, P.; Khapra, M. M.; Laha, A.; and Ravindran, B. 2017. Diversity driven attention model for query-based abstractive summarization. In ACL, volume 1, 1063-1072.

[29]

Nguyen, T. D., and Kan, M.-Y. 2007. Keyphrase extraction in scientific publications. In ICADL, 317-326.

[30]

Nguyen, T. D., and Luong, M.-T. 2010. Wingnus: Keyphrase extraction utilizing document logical structure. In Proceedings of the 5th International Workshop on Semantic Evaluation, 166-169.

[31]

Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer, A. 2017. Automatic differentiation in pytorch. In NIPS-W.

[32]

See, A.; Liu, P. J.; and Manning, C. D. 2017. Get to the point: Summarization with pointer-generator networks. In ACL, volume 1, 1073-1083.

[33]

Song, L.; Wang, Z.; and Hamza, W. 2017. A unified query-based generative model for question generation and question answering. arXiv preprint arXiv: 1709.01058.

[34]

Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NIPS, 3104-3112.

Digital Library

[35]

Wan, X., and Xiao, J. 2008. Single document keyphrase extraction using neighborhood knowledge. In AAAI, 855-860.

[36]

Wang, W.; Yang, N.; Wei, F.; Chang, B.; and Zhou, M. 2017. Gated self-matching networks for reading comprehension and question answering. In ACL, volume 1, 189-198.

[37]

Witten, I. H.; Paynter, G. W.; Frank, E.; Gutwin, C.; and Nevill-Manning, C. G. 1999. Kea: Practical automatic keyphrase extraction. In Proceedings of the fourth ACM conference on Digital libraries, 254-255.

[38]

Zhang, Q.; Wang, Y.; Gong, Y.; and Huang, X. 2016. Keyphrase extraction using deep recurrent neural networks on twitter. In EMNLP, 836-845.

[39]

Zhang, Y.; Fang, Y.; and Weidong, X. 2017. Deep keyphrase generation with a convolutional sequence to sequence model. In ICSAI, 1477-1485.

Cited By

Yu BGao CZhang S(2024)Training with One2MultiSeq: CopyBART for social media keyphrase generationThe Journal of Supercomputing10.1007/s11227-024-06050-880:11(15517-15544)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11227-024-06050-8
Duan WRao HDuan LWang N(2023)Mutual-Attention NetComputational Intelligence and Neuroscience10.1155/2023/86854882023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/8685488
Liu YWu HHuang ZWang HNing YMa JLiu QChen E(2023)TechPat: Technical Phrase Extraction for Patent MiningACM Transactions on Knowledge Discovery from Data10.1145/359660317:9(1-31)Online publication date: 15-Jun-2023
https://dl.acm.org/doi/10.1145/3596603
Show More Cited By

Index Terms

Title-guided encoding for keyphrase generation
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Natural language generation
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Single document keyphrase extraction using neighborhood knowledge
AAAI'08: Proceedings of the 23rd national conference on Artificial intelligence - Volume 2

Existing methods for single document keyphrase extraction usually make use of only the information contained in the specified document. This paper proposes to use a small number of nearest neighbor documents to provide more knowledge to improve single ...
Gazetteer-Guided Keyphrase Generation from Research Papers
Advances in Knowledge Discovery and Data Mining
Abstract
The task of keyphrase generation aims to generate the key phrases that capture the primary content of a document. An external domain-specific gazetteer can assist in generating keyphrases that are literally absent in the document (i.e., do not ...
Domain-specific keyphrase extraction
CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge management

Document keyphrases provide semantic metadata characterizing documents and producing an overview of the content of a document. They can be used in many text-mining and knowledge management related applications. This paper describes a Keyphrase ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

January 2019

10088 pages

ISBN:978-1-57735-809-1

Copyright © 2019 Association for the Advancement of Artificial Intelligence.

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 27 January 2019

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
24
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)3

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu BGao CZhang S(2024)Training with One2MultiSeq: CopyBART for social media keyphrase generationThe Journal of Supercomputing10.1007/s11227-024-06050-880:11(15517-15544)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11227-024-06050-8
Duan WRao HDuan LWang N(2023)Mutual-Attention NetComputational Intelligence and Neuroscience10.1155/2023/86854882023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/8685488
Liu YWu HHuang ZWang HNing YMa JLiu QChen E(2023)TechPat: Technical Phrase Extraction for Patent MiningACM Transactions on Knowledge Discovery from Data10.1145/359660317:9(1-31)Online publication date: 15-Jun-2023
https://dl.acm.org/doi/10.1145/3596603
Dong YWu SMeng FZhou JWang XLin JSu JEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise FilteringProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612413(3897-3907)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612413
Diao SKeh SPan LTian ZSong YZhang T(2023)Hashtag-Guided Low-Resource Tweet ClassificationProceedings of the ACM Web Conference 202310.1145/3543507.3583194(1415-1426)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583194
Xie BSong JShao LWu SWei XYang BLin HXie JSu J(2023)From statistical methods to deep learning, automatic keyphrase predictionInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10338260:4Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.ipm.2023.103382
Wang ASong LLiu QMi HWang LTu ZSu JYu D(2023)Search-engine-augmented dialogue response generation with cheaply supervised query productionArtificial Intelligence10.1016/j.artint.2023.103874319:COnline publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.artint.2023.103874
Ma JCheng JZhang Y(2022)A Novel Keyword Generation Model Based on Topic-Aware and Title-GuideComputational Intelligence and Neuroscience10.1155/2022/17873692022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/1787369
Yao SHu JSun CGao ZLiu N(2022)Key Phrase Extraction based on Pre-trained Language ModelsProceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering10.1145/3573428.3573598(941-945)Online publication date: 21-Oct-2022
https://dl.acm.org/doi/10.1145/3573428.3573598
Zhang YJiang TYang TLi XWang SAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)HTKGProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531990(1044-1054)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531990
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents