Generating Factoid Questions with Question Type Enhanced Representation and Attention-based Copy Mechanism
Article No.: 31, Pages 1 - 18
Abstract
Question generation over knowledge bases is an important research topic. How to deal with rare and low-frequency words in traditional generation models is a key challenge for question generation. Although the copy mechanism provides significant performance improvements, the original copy mechanism weakens the focus on aspect generation in the overall representations. In this article, we present a novel method to improve question generation with a question type enhanced representation and attention-based copy mechanism. The proposed method exploits the advantages of the generate mode in the copy mechanism and replaces objects in the factual triples with question types, which attempts to improve the output quality in the generate mode and effectively generate questions with proper interrogative words. We evaluate the proposed method on two standard benchmark datasets. The experimental results demonstrate that our proposed method can produce higher-quality questions than these of the Encoder-Decoder-based and CopyNet-based methods.
References
[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of 12th USENIX Symposium on OSDI. 265–283. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi.
[2]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR’15) 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.0473.
[3]
Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O’Reilly. http://www.oreilly.de/catalog/9780596516499/index.html.
[4]
Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the ACM SIGMOD. 1247–1250. DOI:
[5]
Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale Simple Question Answering with Memory Networks. CoRR abs/1506.02075 (2015). arxiv:1506.02075http://arxiv.org/abs/1506.02075.
[6]
Wei Chen, Gregory Aist, and Jack Mostow. 2009. Generating Questions Automatically from Informational Text. In Proceedings of the 2nd Workshop on Question Generation, 17–24.
[7]
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (Proceedings of the EMNLP). 1724–1734. DOI:
[8]
Sérgio Curto, Ana Cristina Mendes, and Luísa Coheur. 2012. Question Generation Based on Lexico-Syntactic Patterns Learned from the Web. Dialogue and Discourse 3, 2 (2012), 147–175. http://dad.uni-bielefeld.de/index.php/dad/article/view/1469.
[9]
Michael J. Denkowski and Alon Lavie. 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, WMT@ACL 2014, June 26–27, 2014, Baltimore, Maryland, USA. 376–380. https://www.aclweb.org/anthology/W14-3348/.
[10]
Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017. Learning to Paraphrase for Question Answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), Copenhagen, Denmark, September 9–11, 2017. 875–886. https://www.aclweb.org/anthology/D17-1091/.
[11]
Daniel Duma and Ewan Klein. 2013. Generating Natural Language from Linked Data: Unsupervised Template Extraction. In Proceedings of the 10th International Conference on Computational Semantics (IWCS’13) – Long Papers. 83–94. http://aclweb.org/anthology/W13-0108.
[12]
Hady Elsahar, Christophe Gravier, and Frédérique Laforest. 2018. Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), New Orleans, Louisiana, USA, June 1–6, 2018, Volume 1 (Long Papers). 218–228. https://www.aclweb.org/anthology/N18-1020/.
[13]
Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. In Proceedings the ACL (Volume 1: Long Papers). 1631–1640. https://doi.org/10.18653/v1/P16-1154
[14]
Daya Guo, Yibo Sun, Duyu Tang, Nan Duan, Jian Yin, Hong Chi, James Cao, Peng Chen, and Ming Zhou. 2018. Question Generation from SQL Queries Improves Neural Semantic Parsing. In Proceedings of the EMNLP. 1597–1607. http://aclweb.org/anthology/D18-1188.
[15]
Shizhu He, Cao Liu, Kang Liu, and Jun Zhao. 2017. Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 199–208. DOI:
[16]
Michael Heilman and Noah A. Smith. 2010. Good Question! Statistical Ranking for Question Generation. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2–4, 2010, Los Angeles, California, USA. 609–617. https://www.aclweb.org/anthology/N10-1086/.
[17]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780. DOI:https://doi.org/10.1162/neco.1997.9.8.1735
[18]
Zhiting Hu, Zichao Yang, Ruslan R. Salakhutdinov, Lianhui Qin, Xiaodan Liang, Haoye Dong, and Eric P. Xing. 2018. Deep Generative Models with Learnable Knowledge Constraints. In Proceedings of the NeurIPS. 10522–10533. http://papers.nips.cc/paper/8250-deep-generative-models-with-learnable-knowledge-constraints.
[19]
Saidalavi Kalady, Ajeesh Elikkottil, and Rajarshi Das. 2010. Natural Language Question Generation Using Syntax and Keywords. In Proceedings of QG2010: The Third Workshop on Question Generation, Vol. 2.
[20]
Mitesh M. Khapra, Dinesh Raghu, Sachindra Joshi, and Sathish Reddy. 2017. Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17), Valencia, Spain, April 3–7, 2017, Volume 1: Long Papers. 376–385. https://www.aclweb.org/anthology/E17-1036/.
[21]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). arxiv:1412.6980http://arxiv.org/abs/1412.6980.
[22]
Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, and Hannaneh Hajishirzi. 2019. Text Generation from Knowledge Graphs with Graph Transformers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers). 2284–2293. https://www.aclweb.org/anthology/N19-1238/.
[23]
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. 2015. DBpedia - A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web 6, 2 (2015), 167–195. DOI:
[24]
Chin Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS’04).
[25]
Bang Liu, Mingjun Zhao, Di Niu, Kunfeng Lai, Yancheng He, Haojie Wei, and Yu Xu. 2019. Learning to Generate Questions by Learning What Not to Generate. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, USA, 1106–1118. DOI:https://doi.org/10.1145/3308558.3313737
[26]
Cao Liu, Shizhu He, Kang Liu, and Jun Zhao. 2018. Curriculum Learning for Natural Answer Generation. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI’18), July 13–19, 2018, Stockholm, Sweden, Jérôme Lang (Ed.). ijcai.org, 4223–4229. DOI:https://doi.org/10.24963/ijcai.2018/587
[27]
Cao Liu, Kang Liu, Shizhu He, Zaiqing Nie, and Jun Zhao. 2019. Generating Questions for Knowledge Bases via Incorporating Diversified Contexts and Answer-Aware Loss. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 2431–2441. DOI:
[28]
Tianyu Liu, Bingzhen Wei, Baobao Chang, and Zhifang Sui. 2017. Large-Scale Simple Question Generation by Template-Based Seq2seq Learning. In Natural Language Processing and Chinese Computing - 6th CCF International Conference (NLPCC’17), Dalian, China, November 8–12, 2017, Proceedings (Lecture Notes in Computer Science), Vol. 10619. Springer, 75–87. DOI:
[29]
Thang Luong, Hieu Pham, and Christopher D. Manning. 2015a. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the EMNLP. 1412–1421. http://aclweb.org/anthology/D/D15/D15-1166.pdf.
[30]
Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. 2015b. Addressing the Rare Word Problem in Neural Machine Translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 11–19. https://www.aclweb.org/anthology/P15-1002/.
[31]
Claire Cooper McConnell, Prashanth Mannem, Rashmi Prasad, and Aravind K. Joshi. 2011. A New Approach to Ranking Over-Generated Questions. In AAAI Fall Symposium: Question Generation (AAAI Technical Report), Vol. FS-11-04. AAAI.
[32]
Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Margaret Mitchell, Xiaodong He, and Lucy Vanderwende. 2016. Generating Natural Questions about an Image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1170/.
[33]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the ACL. http://aclweb.org/anthology/P02-1040.
[34]
Yehong Peng, Yizhen Fang, Zhiwen Xie, and Guangyou Zhou. 2019. Topic-enhanced Emotional Conversation Generation with Attention Mechanism. Knowl.-Based Syst. 163 (2019), 429–437. DOI:
[35]
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. 1532–1543. https://www.aclweb.org/anthology/D14-1162/.
[36]
Jiazuo Qiu and Deyi Xiong. 2019. Generating Highly Relevant Questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019. Association for Computational Linguistics, 5982–5986.
[37]
Iulian Vlad Serban, Alberto García-Durán, Çaglar Gülçehre, Sungjin Ahn, Sarath Chandar, Aaron C. Courville, and Yoshua Bengio. 2016. Generating Factoid Questions with Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1056/.
[38]
Dominic Seyler, Mohamed Yahya, and Klaus Berberich. 2015. Generating Quiz Questions from Knowledge Graphs. In Proceedings of the 24th International Conference on World Wide Web Companion (WWW’15), Florence, Italy, May 18–22, 2015 - Companion Volume. 113–114. DOI:https://doi.org/10.1145/2740908.2742722
[39]
Dominic Seyler, Mohamed Yahya, and Klaus Berberich. 2017. Knowledge Questions from Knowledge Graphs. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR’17), Amsterdam, The Netherlands, October 1–4, 2017. 11–18. DOI:https://doi.org/10.1145/3121050.3121073
[40]
Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural Responding Machine for Short-Text Conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 1577–1586. https://www.aclweb.org/anthology/P15-1152/.
[41]
Linfeng Song, Zhiguo Wang, Wael Hamza, Yue Zhang, and Daniel Gildea. 2018. Leveraging Context Information for Natural Question Generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1–6, 2018, Volume 2 (Short Papers). 569–574. https://www.aclweb.org/anthology/N18-2090/.
[42]
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the NeurIPS. 3104–3112. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.
[43]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA. 5998–6008. http://papers.nips.cc/paper/7181-attention-is-all-you-need.
[44]
Hao Wang, Xiaodong Zhang, and Houfeng Wang. 2018. A Neural Question Generation System Based on Knowledge Base. In Proceedings of the NLPCC. 133–142. DOI:
[45]
Dirk Weissenborn, Georg Wiese, and Laura Seiffe. 2017. FastQA: A Simple and Efficient Neural Architecture for Question Answering. CoRR abs/1703.04816 (2017).
[46]
Xiuyu Wu, Nan Jiang, and Yunfang Wu. 2020. A Question Type Driven and Copy Loss Enhanced Framework for Answer-Agnostic Neural Question Generation. In Proceedings of the Fourth Workshop on Neural Generation and Translation. Association for Computational Linguistics, 69–78. DOI:
[47]
Zhiwen Xie, Zhao Zeng, Guangyou Zhou, and Weijun Wang. 2017. Topic enhanced deep structured semantic models for knowledge base question answering. SCIENCE CHINA Information Sciences 60, 11 (2017), 110103:1–110103:15. DOI:
[48]
Shijie Zhang, Lizhen Qu, Shaodi You, Zhenglu Yang, and Jiawan Zhang. 2017. Automatic Generation of Grounded Visual Questions. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017. 4235–4243. DOI:https://doi.org/10.24963/ijcai.2017/592
[49]
Guangyou Zhou, Yizhen Fang, Yehong Peng, and Jiaheng Lu. 2020. Neural Conversation Generation with Auxiliary Emotional Supervised Models. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 2 (2020), 19:1–19:17.
[50]
Guangyou Zhou and Jimmy Xiangji Huang. 2017. Modeling and Learning Distributed Word Representation with Metadata for Question Retrieval. IEEE Trans. Knowl. Data Eng. 29, 6 (2017), 1226–1239.
[51]
Guangyou Zhou, Zhiwen Xie, Jimmy Xiangji Huang, and Tingting He. 2016. Bi-Transferring Deep Neural Networks for Domain Adaptation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1031/.
[52]
Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019a. Multi-Task Learning with Language Modeling for Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 3392–3397.
[53]
Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019b. Question-type Driven Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 6031–6036.
Index Terms
- Generating Factoid Questions with Question Type Enhanced Representation and Attention-based Copy Mechanism
Recommendations
Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
WWW '20: Proceedings of The Web Conference 2020The ability to ask questions is important in both human and machine intelligence. Learning to ask questions helps knowledge acquisition, improves question-answering and machine reading comprehension tasks, and helps a chatbot to keep the conversation ...
Automatically Generating and Validating Reading-Check Questions
ITS '08: Proceedings of the 9th international conference on Intelligent Tutoring SystemsThe number of texts used by the REAP tutor and challenges of automatically generating reading comprehension questions necessitated a simpler automated type of question. We describe an algorithm for generating such reading-check questions for arbitrary ...
A Survey of Evaluation Metrics Used for NLG Systems
In the last few years, a large number of automatic evaluation metrics have been proposed for evaluating Natural Language Generation (NLG) systems. The rapid development and adoption of such automatic evaluation metrics in a relatively short time has ...
Comments
Information & Contributors
Information
Published In
March 2022
413 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 January 2022
Accepted: 01 July 2021
Revised: 01 February 2021
Received: 01 March 2020
Published in TALLIP Volume 21, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
- Refereed
Funding Sources
- National Natural Science Foundation of China
- Natural Sciences and Engineering Research Council (NSERC) of Canada and York Research Chairs (YRC) program
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 415Total Downloads
- Downloads (Last 12 months)73
- Downloads (Last 6 weeks)11
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format