research-article

Open access

A Review on Question Generation from Natural Language Text

Authors:

Xueqi ChengAuthors Info & Claims

ACM Transactions on Information Systems (TOIS), Volume 40, Issue 1

Article No.: 14, Pages 1 - 43

https://doi.org/10.1145/3468889

Published: 08 September 2021 Publication History

All formats PDF

Abstract

Question generation is an important yet challenging problem in Artificial Intelligence (AI), which aims to generate natural and relevant questions from various input formats, e.g., natural language text, structure database, knowledge base, and image. In this article, we focus on question generation from natural language text, which has received tremendous interest in recent years due to the widespread applications such as data augmentation for question answering systems. During the past decades, many different question generation models have been proposed, from traditional rule-based methods to advanced neural network-based methods. Since there have been a large variety of research works proposed, we believe it is the right time to summarize the current status, learn from existing methodologies, and gain some insights for future development. In contrast to existing reviews, in this survey, we try to provide a more comprehensive taxonomy of question generation tasks from three different perspectives, i.e., the types of the input context text, the target answer, and the generated question. We take a deep look into existing models from different dimensions to analyze their underlying ideas, major design principles, and training strategies We compare these models through benchmark tasks to obtain an empirical understanding of the existing techniques. Moreover, we discuss what is missing in the current literature and what are the promising and desired future directions.

References

[1]

Naveed Afzal and Ruslan Mitkov. 2014. Automatic generation of multiple choice questions using dependency-based semantic relations. Soft Comput. 18 (07 2014), 1269–1281.

[2]

Manish Agarwal, Rakshit Shah, and Prashanth Mannem. 2011. Automatic question generation using discourse cues. In Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, 1–9.

[3]

Renlong Ai, Sebastian Krause, Walter Kasper, Feiyu Xu, and Hans Uszkoreit. 2015. Semi-automatic generation of multiple-choice tests from mentions of semantic relations. In Proceedings of the 2nd Workshop on Natural Language Processing Techniques for Educational Applications. Association for Computational Linguistics, 26–33.

[4]

Husam Ali, Yllias Chali, and Sadid A. Hasan. 2010. Automation of question generation from sentences. In Proceedings of the Third Workshop on Question Generation. 58–67.

[5]

Husam Ali, Yllias Chali, and Sadid A. Hasan. 2011. Automation of question generation from sentences. In Proceedings of the Actes de la 17e Conference sur le Traitement Automatique des Langues Naturelles Articles courts. 213–218.

[6]

Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeff Dalton, and Mikhail Burtsev. 2020. ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ). arXiv preprint arXiv:2009.11352 (2020).

[7]

Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 475–484.

[8]

James Allan. 2005. HARD Track Overview in TREC 2003 High Accuracy Retrieval from Documents. Technical Report. Massachusetts University Amherst Center For Intelligent Information Retrieval.

[9]

Jacopo Amidei, P. Piwek, and A. Willis. 2018. Evaluation methodologies in automatic question generation 2013–2018. In INLG.

[10]

L. Anderson, D. Krathwohl, and B. Bloom. 2000. A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives Longman.

[11]

R. Anderson and W. B. Biddle. 1975. On asking people questions about what they are reading. Psychol. Learn. Motiv. 9 (1975), 89–132.

[12]

J. Araki, Dheeraj Rajagopal, S. Sankaranarayanan, Susan Holm, Yukari Yamakawa, and T. Mitamura. 2016. Generating questions and multiple-choice answers using semantic analysis of texts. In COLING.

[13]

Tina Baghaee. 2017. Automatic Neural Question Generation using Community-based Question Answering Systems. Ph.D. Dissertation. University of Lethbridge, Deptartment of Mathematics, Lethbridge, Alberta, Canada.

[14]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

[15]

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, X. Liu, Yu Wang, Songhao Piao, Jianfeng Gao, M. Zhou, and H. Hon. 2020. UniLMv2: Pseudo-masked language models for unified language model pre-training. ArXiv abs/2002.12804 (2020).

[16]

Junwei Bao, Yeyun Gong, N. Duan, M. Zhou, and T. Zhao. 2018. Question generation with doubly adversarial nets. IEEE/ACM Trans. Audio, Speech Lang. Process. 26 (2018), 2230–2239.

Digital Library

[17]

Rohan Bhambhoria, Luna Feng, Dawn Sepehr, John Chen, Conner Cowling, Sedef Kocak, and Elham Dolatabadi. 2020. A smart system to generate and validate question answer pairs for COVID-19 literature. In Proceedings of the 1st Workshop on Scholarly Document Processing. 20–30.

[18]

Parminder Bhatia, Kristjan Arumae, and B. Celikkaya. 2020. Dynamic transfer learning for named entity recognition. In Proceedings of the International Workshop on Health Intelligence. Springer, 69–81.

[19]

K. Boyer and P. Piwek. 2010. In Proceedings of the 3rd Workshop on Question Generation. questiongeneration.org.

[20]

Pavel Braslavski, Denis Savenkov, Eugene Agichtein, and Alina Dubatovka. 2017. What do you mean exactly? Analyzing clarification questions in CQA. In Proceedings of the Conference on Human Information Interaction and Retrieval. 345–348.

[21]

Daniel Fernando Campos, T. Nguyen, M. Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, L. Deng, and Bhaskar Mitra. 2016. MS MARCO: A human-generated machine reading comprehension dataset. ArXiv abs/1611.09268 (2016).

[22]

Yang Trista Cao, Sudha Rao, and Hal Daumé III. 2019. Controlling the specificity of clarification question generation. In Proceedings of the Workshop on Widening NLP. Association for Computational Linguistics, 53–56.

[23]

A. Celikyilmaz, E. Clark, and J. Gao. 2020. Evaluation of text generation: A survey. arXiv preprint arXiv:2006.14799.

[24]

D. Ch and S. Saha. 2020. Automatic multiple choice question generation from text: A survey. IEEE Trans. Learn. Technol. 13 (2020), 14–25.

[25]

Zi Chai and Xiaojun Wan. 2020. Learning to ask more: Semi-autoregressive sequential question generation under dual-graph interaction. In Proceedings of the 58th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 225–237.

[26]

Yllias Chali and Tina Baghaee. 2018. Automatic opinion question generation. In Proceedings of the 11th International Conference on Natural Language Generation. Association for Computational Linguistics, 152–158.

[27]

Yllias Chali and Sadid A. Hasan. 2015. Towards topic-to-question generation. Comput. Ling. 41 (2015), 1–20.

Digital Library

[28]

Ying-Hong Chan and Yao-Chung Fan. 2019. A recurrent BERT-based model for question generation. In MRQA@EMNLP.

[29]

Danqi Chen, A. Fisch, J. Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. ArXiv abs/1704.00051 (2017).

[30]

Guanliang Chen, Jie Yang, and Dragan Gasevic. 2019. A comparative study on question-worthy sentence selection strategies for educational question generation. In Artificial Intelligence in Education, Seiji Isotani, Eva Millán, Amy Ogan, Peter Hastings, Bruce McLaren, and Rose Luckin (Eds.). Springer International Publishing, Cham, 59–70.

[31]

Guanliang Chen, Jie Yang, C. Hauff, and G. Houben. 2018. LearningQ: A large-scale dataset for educational question generation. In ICWSM.

[32]

Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang, Yang Wang, and Meng Wang. 2020. Try this instead: Personalized and interpretable substitute recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 891–900.

Digital Library

[33]

Wei Chen and Gregory Aist. 2009. Generating questions automatically from informational text. In Proceedings of AIED Workshop on Question Generation. 17–24.

[34]

Wang Chen, Hou Pong Chan, Piji Li, Lidong Bing, and Irwin King. 2019. An integrated approach for keyphrase generation via exploring the power of retrieval and extraction. In NAACL-HLT.

[35]

Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2019. Natural question generation with reinforcement learning based graph-to-sequence model. CoRR abs/1910.08832 (2019).

[36]

Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2020. Reinforcement learning based graph-to-sequence model for natural question generation. ArXiv abs/1908.04942 (2020).

[37]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.

[38]

W. S. Cho, Y. Zhang, Sudha Rao, Chris Brockett, and S. Lee. 2019. Generating a common question from multiple documents using multi-source encoder-decoder models. ArXiv abs/1910.11483 (2019).

[39]

W. S. Cho, Y. Zhang, Sudha Rao, A. Çelikyilmaz, Chenyan Xiong, Jianfeng Gao, M. Wang, and B. Dolan. 2019. Contrastive multi-document question generation. arXiv preprint arXiv:1911.03047.

[40]

Eunsol Choi, H. He, Mohit Iyyer, Mark Yatskar, W. Yih, Yejin Choi, Percy Liang, and L. Zettlemoyer. 2018. QuAC : Question answering in context. ArXiv abs/1808.07036 (2018).

[41]

Yunjey Choi, Min-Je Choi, Munyoung Kim, Jung-Woo Ha, S. Kim, and J. Choo. 2018. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8789–8797.

[42]

Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 815–824.

Digital Library

[43]

Anni Coden, Daniel Gruhl, Neal Lewis, and Pablo N. Mendes. 2015. Did you mean A or B? Supporting clarification dialog for entity disambiguation. In SumPre-HSWI@ ESWC.

[44]

P. Connor-Greene. 2000. Assessing and promoting student learning: Blurring the line between teaching and testing. Teach. Psychol. 27 (2000), 84–88.

[45]

P. Costa and Ivandré Paraboni. 2019. Personality-dependent neural text summarization. In RANLP.

[46]

Sérgio Curto, Ana Cristina Mendes, and Luísa Coheur. 2011. Exploring linguistically-rich patterns for question generation. In Proceedings of the UCNLG+Eval: Language Generation and Evaluation Workshop (UCNLG+EVAL’11). Association for Computational Linguistics, 33–38.

Digital Library

[47]

Yang Deng, Yuexiang Xie, Y. Li, Min Yang, Nan Du, W. Fan, Kai Lei, and Ying Shen. 2019. Multi-task learning with multi-view attention for answer selection and knowledge base question answering. ArXiv abs/1812.02354 (2019).

[48]

Michael J. Denkowski and A. Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In WMT@ACL.

[49]

Emily L. Denton, Soumith Chintala, Arthur Szlam, and R. Fergus. 2015. Deep generative image models using a laplacian pyramid of adversarial networks. In NIPS.

[50]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[51]

Kaustubh Dhole and Christopher D. Manning. 2020. Syn-QG: Syntactic and shallow semantic rules for question generation. In Proceedings of the 58th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 752–765.

[52]

Chuong B. Do and A. Ng. 2005. Transfer learning for text classification. In NIPS.

[53]

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 13063–13075. Retrieved from http://papers.nips.cc/paper/9464-unified-language-model-pre-training-for-natural-language-understanding-and-generation.pdf.

[54]

Xiaozheng Dong, Yu Hong, Xin Chen, Weikang Li, Min Zhang, and Qiaoming Zhu. 2018. Neural question generation with semantics of question type. In Natural Language Processing and Chinese Computing, Min Zhang, Vincent Ng, Dongyan Zhao, Sujian Li, and Hongying Zan (Eds.). Springer International Publishing, Cham, 213–223.

[55]

Xinya Du and Claire Cardie. 2017. Identifying where to focus in reading comprehension for neural question generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2067–2073.

[56]

X. Du and Claire Cardie. 2018. Harvesting paragraph-level question-answer pairs from Wikipedia. In ACL.

[57]

Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. In Proceedings of the 55th Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1342–1352.

[58]

Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question generation for question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 866–874.

[59]

M. Dunn, Levent Sagun, M. Higgins, V. U. Güney, Volkan Cirik, and K. Cho. 2017. SearchQA: A new Q&A dataset augmented with context from a search engine. ArXiv abs/1704.05179 (2017).

[60]

Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. arXiv preprint arXiv:2005.03754 (2020).

[61]

Ibrahim Eldesoky. 2015. Semantic question generation using artificial immunity. I.J. Mod. Educ. Comput. Sci. 7 (01 2015), 1–8.

[62]

Hady ElSahar, C. Gravier, and F. Laforest. 2018. Zero-shot question generation from knowledge graphs for unseen predicates and entity types. ArXiv abs/1802.06842 (2018).

[63]

A. R. Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, and B. Xiang. 2020. Template-based question generation from retrieved sentences for improved unsupervised question answering. In ACL.

[64]

Angela Fan, David Grangier, and M. Auli. 2018. Controllable abstractive summarization. In NMT@ACL.

[65]

Zhihao Fan, Zhongyu Wei, Piji Li, Yanyan Lan, and X. Huang. 2018. A question type driven framework to diversify visual question generation. In IJCAI.

[66]

Zhihao Fan, Zhongyu Wei, S. Wang, Y. Liu, and X. Huang. 2018. A reinforcement learning framework for natural question generation using bi-discriminators. In COLING.

[67]

Yuwei Fang, Shuohang Wang, Zhe Gan, S. Sun, and Jing jing Liu. 2020. Accelerating real-time question answering via question generation. ArXiv abs/2009.05167 (2020).

[68]

M. Flor and Brian Riordan. 2018. A semantic role-based approach to open-domain automatic question generation. In BEA@NAACL-HLT.

[69]

Michael Flor and Brian Riordan. 2018. A semantic role-based approach to open-domain automatic question generation. In Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics, 254–263.

[70]

Yifan Gao, Lidong Bing, Wang Chen, Michael Lyu, and Irwin King. 2019. Difficulty controllable generation of reading comprehension questions. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, 4968–4974.

[71]

Yifan Gao, Piji Li, Irwin King, and Michael R. Lyu. 2019. Interconnected question generation with coreference alignment and conversation flow modeling. In Proceedings of the 57th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 4853–4862.

[72]

Donna Gates. 2008. Automatically Generating Reading Comprehension Look-back Strategy: Questions from Expository Texts. Technical Report. Carnegie-Mellon Univ Pittsburgh Pa School of Computer Science.

[73]

Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2018. Soft layer-specific multi-task summarization with entailment and question generation. In ACL.

[74]

Deepak Gupta, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2020. Reinforced multi-task approach for multi-hop question generation. CoRR abs/2004.02143 (2020).

[75]

D. Gupta, H. Chauhan, Asif Ekbal, and P. Bhattacharyya. 2020. Reinforced multi-task approach for multi-hop question generation. ArXiv abs/2004.02143 (2020).

[76]

Xu-Wang Han, Hai-Tao Zheng, J. Chen, and Cong-Zhi Zhao. 2019. Diverse decoding for abstractive document summarization. Appl. Sci. 9 (2019), 386.

[77]

Sanda Harabagiu, Andrew Hickl, John Lehmann, and Dan Moldovan. 2005. Experiments with interactive question-answering. In Proceedings of the 43rd Meeting of the Association for Computational Linguistics (ACL’05). 205–214.

Digital Library

[78]

V. Harrison and M. Walker. 2018. Neural generation of diverse questions using answer focus, contextual and linguistic features. In INLG.

[79]

Amartya Hatua, T. T. Nguyen, and A. Sung. 2019. Dialogue generation using self-attention generative adversarial network. In Proceedings of the IEEE International Conference on Conversational Data & Knowledge Engineering (CDKE’19). 33–38.

[80]

T. He, J. Zhang, Z. Zhou, and J. Glass. 2021. Quantifying exposure bias for neural language generation. arXiv preprint arXiv:1905.10617 (2021).

[81]

Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, X. Xiao, Yulong Liu, Yizhong Wang, H. Wu, Qiaoqiao She, Xuan Liu, T. Wu, and H. Wang. 2018. DuReader: A Chinese machine reading comprehension dataset from real-world applications. ArXiv abs/1711.05073 (2018).

[82]

Michael Heilman and Noah A. Smith. 2009. Question Generation via Overgenerating Transformations and Ranking. Technical Report. Carnegie-Mellon Univ Pittsburgh pa language technologies insT.

[83]

Michael Heilman and Noah A. Smith. 2010. Extracting simplified statements for factual question generation.

Digital Library

[84]

Michael Heilman and Noah A. Smith. 2010. Good question! Statistical ranking for question generation. In HLT-NAACL.

Digital Library

[85]

Felix Hill, Antoine Bordes, S. Chopra, and J. Weston. 2016. The goldilocks principle: Reading children’s books with explicit memory representations. CoRR abs/1511.02301 (2016).

[86]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neur. Comput. 9, 8 (1997), 1735–1780.

Digital Library

[87]

T. Hosking and S. Riedel. 2019. Evaluating rewards for question generation models. In NAACL-HLT.

[88]

Wenpeng Hu, Bing Liu, Jinwen Ma, Dongyan Zhao, and Rui Yan. 2018. Aspect-based question generation. In Proceedings of the 6th International Conference on Learning Representations (ICLR’18). OpenReview.net. Retrieved from https://openreview.net/forum?id=rkRR1ynIf.

[89]

Wenpeng Hu, B. Liu, Jinwen Ma, Dongyan Zhao, and R. Yan. 2018. Workshop track-ICLR 2018 Aspect-based question generation.

[90]

Wenpeng Hu, B. Liu, R. Yan, Dongyan Zhao, and Jinwen Ma. 2018. Topic-based question generation. In ICLR.

[91]

Yan Huang and L. He. 2016. Automatic generation of short answer questions for reading comprehension assessment. Nat. Lang. Eng. 22 (2016), 457–489.

[92]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and A. Efros. 2017. Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5967–5976.

[93]

Masaru Isonuma, Toru Fujino, Junichiro Mori, Y. Matsuo, and I. Sakata. 2017. Extractive summarization using multi-task learning with document classification. In EMNLP.

[94]

S. Johansen and K. Juselius. 2005. Maximum likelihood estimation and inference on cointegration—with applications to the demand for money. Oxford Bull. Econ. Statist. 52 (2005), 169–210.

[95]

Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In ACL.

[96]

Mitesh M. Khapra, Dinesh Raghu, S. Joshi, and Sathish Reddy. 2017. Generating natural language question-answer pairs from a knowledge graph using an RNN-based question generation model. In EACL.

[97]

Johannes Kiesel, Arefeh Bahrami, Benno Stein, Avishek Anand, and Matthias Hagen. 2018. Toward voice query clarification. In Proceedings of the International ACM SIGIR Conference on Research & Development in Information Retrieval. 1257–1260.

Digital Library

[98]

Yanghoon Kim, Hwanhee Lee, Joongbo Shin, and K. Jung. 2019. Improving neural question generation using answer separation. ArXiv abs/1809.02393 (2019).

[99]

Diederik P. Kingma and M. Welling. 2014. Auto-encoding variational Bayes. CoRR abs/1312.6114 (2014).

[100]

Thomas Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. ArXiv abs/1609.02907 (2017).

[101]

T. Klein and M. Nabi. 2019. Learning to answer by learning to ask: Getting the best of GPT-2 and BERT worlds. ArXiv abs/1911.02365 (2019).

[102]

Wei-Jen Ko, Te-Yuan Chen, Yiyan Huang, Greg Durrett, and Junyi Jessy Li. 2020. Inquisitive question generation for high level text comprehension. ArXiv abs/2010.01657 (2020).

[103]

Tomás Kociský, Jonathan Schwarz, P. Blunsom, Chris Dyer, K. Hermann, Gábor Melis, and E. Grefenstette. 2018. The narrative QA reading comprehension challenge. Trans. Assoc. Comput. Ling. 6 (2018), 317–328.

[104]

Antonios Minas Krasakis, Mohammad Aliannejadi, Nikos Voskarides, and Evangelos Kanoulas. 2020. Analysing the effect of clarifying questions on document ranking in conversational search. In Proceedings of the ACM SIGIR on International Conference on Theory of Information Retrieval. 129–132.

Digital Library

[105]

Kalpesh Krishna and Mohit Iyyer. 2019. Generating question-answer hierarchies. arXiv preprint arXiv:1906.02622 (2019).

[106]

V. Kumar, Yuncheng Hua, G. Ramakrishnan, G. Qi, L. Gao, and Yuan-Fang Li. 2019. Difficulty-controllable multi-hop question generation from knowledge graphs. In SEMWEB.

[107]

Vishwajeet Kumar, Yuncheng Hua, Ganesh Ramakrishnan, Guilin Qi, Lianli Gao, and Yuan-Fang Li. 2019. Difficulty-controllable multi-hop question generation from knowledge graphs. In International Semantic Web Conference. Springer, 382–398.

Digital Library

[108]

Vishwajeet Kumar, Nitish Joshi, Arijit Mukherjee, Ganesh Ramakrishnan, and Preethi Jyothi. 2019. Cross-lingual training for automatic question generation. In Proceedings of the 57th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 4863–4872.

[109]

V. Kumar, G. Ramakrishnan, and Yuan-Fang Li. 2019. Putting the horse before the cart: A generator-evaluator framework for question generation from text. In CoNLL.

[110]

Hidenobu Kunichika, Tomoki Katayama, Tsukasa Hirashima, and Akira Takeuchi. 2004. Automated question generation methods for intelligent English learning systems and its evaluation. In Proceedings of the ICCE.

[111]

Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. 2019. A systematic review of automatic question generation for educational purposes. Int. J. Artif. Intell. Educ. 30 (2019), 121–204.

[112]

Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. 2020. A systematic review of automatic question generation for educational purposes. Int. J. Artif. Intell. Educ. 30, 1 (2020), 121–204.

[113]

Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, and Kilian Q. Weinberger. 2015. From word embeddings to document distances. In ICML.

Digital Library

[114]

T. Kwiatkowski, J. Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, C. Alberti, D. Epstein, Illia Polosukhin, J. Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Q. Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. Trans. Assoc. Comput. Ling. 7 (2019), 453–466.

[115]

Philippe Laban, John Canny, and Marti A. Hearst. 2020. What’s the latest? A question-driven news chatbot. In Proceedings of the 58th Meeting of the Association for Computational Linguistics: System Demonstrations. 380–387.

[116]

Igor Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understanding. In Proceedings of the 53rd Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, 889–898.

[117]

I. Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understanding. In ACL.

[118]

Guokun Lai, Qizhe Xie, Hanxiao Liu, Y. Yang, and E. Hovy. 2017. RACE: Large-scale reading comprehension dataset from examinations. In EMNLP.

[119]

C. Ledig, L. Theis, Ferenc Huszár, J. Caballero, Andrew Aitken, Alykhan Tejani, J. Totz, Zehan Wang, and W. Shi. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 105–114.

[120]

D. Lee, Seanie Lee, Woo Tae Jeong, Donghwan Kim, and Sung Ju Hwang. 2020. Generating diverse and consistent QA pairs from contexts with information-maximizing hierarchical conditional VAEs. In ACL.

[121]

Wen-Yu Lee, Y. Kuo, Peng-Ju Hsieh, Wen-Feng Cheng, Ting-Hsuan Chao, Hui-Lan Hsieh, Chieh-En Tsai, Hsiao-Ching Chang, Jia-Shin Lan, and W. Hsu. 2015. Unsupervised latent aspect discovery for diverse event summarization. In MM’15.

[122]

Roger Levy and Galen Andrew. 2006. Tregex and Tsurgeon: Tools for querying and manipulating tree data structures. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC’06). European Language Resources Association (ELRA).

[123]

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019).

[124]

J. Li, M. Galley, Chris Brockett, Jianfeng Gao, and W. Dolan. 2016. A diversity-promoting objective function for neural conversation models. In HLT-NAACL.

[125]

Jingjing Li, Yifan Gao, Lidong Bing, Irwin King, and Michael R. Lyu. 2019. Improving question generation with to the point context. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, 3216–3226.

[126]

J. Li, Alexander H. Miller, S. Chopra, Marc’Aurelio Ranzato, and J. Weston. 2017. Learning through dialogue interactions by asking questions. In ICLR.

[127]

Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. ArXiv abs/1801.07606 (2018).

[128]

Yikang Li, N. Duan, B. Zhou, X. Chu, Wanli Ouyang, and X. Wang. 2018. Visual question generation as dual task of visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6116–6124.

[129]

Yin-Hsiang Liao and Jia-Ling Koh. 2020. Question generation through transfer learning. In IEA/AIE.

[130]

Chenghua Lin, Dong Liu, Wei Pang, and Edward Apeh. 2015. Automatically predicting quiz difficulty level using similarity measures. In Proceedings of the 8th International Conference on Knowledge Capture (K-CAP’15). Association for Computing Machinery.

Digital Library

[131]

Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In ACL 2004.

[132]

David Lindberg, Fred Popowich, John Nesbit, and Phil Winne. 2013. Generating natural language questions to support learning on-line. In Proceedings of the 14th European Workshop on Natural Language Generation. Association for Computational Linguistics, 105–114.

[133]

Yanxiang Ling, Fei Cai, Honghui Chen, and Maarten de Rijke. 2020. Leveraging context for neural question generation in open-domain dialogue systems. In WWW’20: The Web Conference 2020, Taipei, Taiwan, April 20–24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 2486–2492.

Digital Library

[134]

B. Liu, Haojie Wei, Di Niu, Haolan Chen, and Yancheng He. 2020. Asking questions the human way: Scalable question-answer generation from text corpus. In Proceedings of the Web Conference.

Digital Library

[135]

Bang Liu, Mingjun Zhao, Di Niu, Kunfeng Lai, Yancheng He, Haojie Wei, and Yu Xu. 2019. Learning to generate questions by learning what not to generate. CoRR abs/1902.10418 (2019).

[136]

B. Liu, Mingjun Zhao, Di Niu, K. Lai, Yancheng He, Haojie Wei, and Y. Xu2019. Learning to generate questions by learning what not to generate. In Proceedings of the World Wide Web Conference. 1106–1118.

[137]

Dayiheng Liu, Yeyun Gong, Jie Fu, Wei Liu, Yu Yan, B. Shao, Daxin Jiang, J. Lv, and N. Duan. 2020. Diverse, controllable, and keyphrase-aware: A corpus and method for news multi-headline generation. ArXiv abs/2004.03875 (2020).

[138]

L. Liu, Yao Lu, Min Yang, Q. Qu, Jia Zhu, and H. Li. 2018. Generative adversarial network for abstractive text summarization. ArXiv abs/1711.09357 (2018).

[139]

T. Liu, Bingzhen Wei, B. Chang, and Z. Sui. 2017. Large-scale simple question generation by template-based seq2seq learning. In NLPCC.

[140]

Xiyao Ma, Qile Zhu, Yanlin Zhou, and Xiaolin Li. 2020. Improving question generation with sentence-level semantic matching and answer position inferring. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 8464–8471. Retrieved from https://aaai.org/ojs/index.php/AAAI/article/view/6366.

[141]

Xiyao Ma, Qile Zhu, Y. Zhou, Xiaolin Li, and Dapeng Wu. 2020. Asking complex questions with multi-hop answer-focused reasoning. ArXiv abs/2009.07402 (2020).

[142]

Mukta Majumder and Sujan Kumar Saha. 2015. A system for generating multiple choice questions: With a novel approach for sentence selection. In Proceedings of the 2nd Workshop on Natural Language Processing Techniques for Educational Applications. Association for Computational Linguistics, 64–72.

[143]

Prashanth Mannem, Rashmi Prasad, and Aravind Joshi. 2010. Question generation from paragraphs at U Penn: QGSTEC system description. In Proceedings of the QG2010: The Third Workshop on Question Generation. 84–91.

[144]

Karen Mazidi and Rodney D. Nielsen. 2015. Leveraging multiple views of text for automatic question generation. In Artificial Intelligence in Education, Cristina Conati, Neil Heffernan, Antonija Mitrovic, and M. Felisa Verdejo (Eds.). Springer International Publishing, Cham, 257–266.

[145]

Karen Mazidi and Paul Tarau. 2016. Infusing NLU into automatic question generation. In INLG 2016 - Proceedings of the Ninth International Natural Language Generation Conference, September 5-8, 2016, Edinburgh, UK, Amy Isard, Verena Rieser, and Dimitra Gkatzia (Eds.). The Association for Computer Linguistics, 51–60.

[146]

J. Mcmillan. 2005. Secondary teachers’ classroom assessment and grading practices. Educ. Measur.: Iss. Pract. 20 (2005), 20–32.

[147]

R. Mitkov. 2003. Computer-aided generation of multiple-choice tests. In Proceedings of the International Conference on Natural Language Processing and Knowledge Engineering.

Digital Library

[148]

R. Mitkov, L. Ha, and N. Karamanis. 2006. A computer-aided environment for generating multiple-choice test items. Nat. Lang. Eng. 12 (2006), 177–194.

Digital Library

[149]

Ruslan Mitkov and Le An Ha. 2003. Computer-aided generation of multiple-choice tests. In Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing. 17–22.

Digital Library

[150]

N. Mostafazadeh, I. Misra, J. Devlin, M. Mitchell, X. He, and L. Vanderwende. 2016. Generating natural questions about an image. ArXiv abs/1603.06059 (2016).

[151]

Jack Mostow and Wei Chen. 2009. Generating Instruction Automatically for the Reading Strategy of Self-Questioning. IOS Press, NLD, 465–472.

[152]

Mao Nakanishi, Tetsunori Kobayashi, and Yoshihiko Hayashi. 2019. Towards answer-unaware conversational question generation. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering. Association for Computational Linguistics, 63–71.

[153]

S. Narayan, Gonçalo Simães, Ji Ma, Hannah Craighead, and R. McDonald. 2020. QURIOUS: Question generation pretraining for text generation. ArXiv abs/2004.11026 (2020).

[154]

Preksha Nema and Mitesh M. Khapra. 2018. Towards a better metric for evaluating question generation systems. ArXiv abs/1808.10192 (2018).

[155]

Preksha Nema, Mitesh M. Khapra, Anirban Laha, and B. Ravindran. 2017. Diversity driven attention model for query-based abstractive summarization. In ACL.

[156]

Preksha Nema, Akash Kumar Mohankumar, Mitesh M. Khapra, Balaji Vasan Srinivasan, and Balaraman Ravindran. 2019. Let’s ask again: Refine network for automatic question generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 3314–3323.

[157]

Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019. From doc2query to docTTTTTquery. Online Preprint (2019).

[158]

Mohammad Norouzi, S. Bengio, Z. Chen, Navdeep Jaitly, Mike Schuster, Y. Wu, and Dale Schuurmans. 2016. Reward augmented maximum likelihood for neural structured prediction. In NIPS.

[159]

L. Odilinye, F. Popowich, E. Zhang, J. Nesbit, and P. H. Winne. 2015. Aligning automatically generated questions to instructor goals and learner behaviour. In Proceedings of the IEEE 9th International Conference on Semantic Computing (IEEE ICSC’15). 216–223.

[160]

A. Olney, A. Graesser, and N. Person. 2012. Question generation from concept maps. Dialog. Disc. 3 (2012), 75–99.

[161]

Boyuan Pan, Hao Li, Ziyu Yao, Deng Cai, and Huan Sun. 2019. Reinforced dynamic reasoning for conversational question generation. In Proceedings of the 57th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2114–2124.

[162]

Liangming Pan, Wenqiang Lei, Tat-Seng Chua, and Min-Yen Kan. 2019. Recent advances in neural question generation. ArXiv abs/1905.08949 (2019).

[163]

Liangming Pan, Yuxi Xie, Yansong Feng, Tat-Seng Chua, and Min-Yen Kan. 2020. Semantic graphs for generating deep questions. In Proceedings of the 58th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1463–1475.

[164]

Liangming Pan, Yuxi Xie, Yansong Feng, Tat-Seng Chua, and Min-Yen Kan. 2020. Semantic graphs for generating deep questions. In ACL.

[165]

Youcheng Pan, Baotian Hu, Qingcai Chen, Yang Xiang, and X. Wang. 2020. Learning to generate diverse questions from keywords. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’20). 8224–8228.

[166]

Kishore Papineni, S. Roukos, T. Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In ACL.

[167]

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. (2018).

[168]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.

[169]

Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proceedings of the Conference on Human Information Interaction and Retrieval. 117–126.

Digital Library

[170]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019).

[171]

Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. ArXiv abs/1806.03822 (2018).

[172]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100, 000+ questions for machine comprehension of text. ArXiv abs/1606.05250 (2016).

[173]

Sheetal Rakangor and Dr. Y. R. Ghodasara. 2015. Literature review of automatic question generation systems. International Journal of Scientific and Research Publications 5, 1 (2015), 1–5.

[174]

Sudha Rao and Hal Daumé. 2019. Answer-based adversarial training for generating clarification questions. In NAACL-HLT.

[175]

Sudha Rao and Hal Daumé III. 2018. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. arXiv preprint arXiv:1805.04655 (2018).

[176]

Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Trans. Assoc. Comput. Ling. 7 (2019), 249–266.

[177]

Sathish Reddy, Dinesh Raghu, Mitesh M. Khapra, and Sachindra Joshi. 2017. Generating natural language question-answer pairs from a knowledge graph using an RNN-based question generation model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics. 376–385. Retrieved from https://www.aclweb.org/anthology/E17-1036.

[178]

Xuhui Ren, Hongzhi Yin, Tong Chen, Hao Wang, Zi Huang, and Kai Zheng. 2021. Learning to ask appropriate questions in conversational recommendation. arXiv preprint arXiv:2105.04774 (2021).

[179]

Xuhui Ren, Hongzhi Yin, Tong Chen, Hao Wang, Nguyen Quoc Viet Hung, Zi Huang, and Xiangliang Zhang. 2020. CRSAL: Conversational recommender systems with adversarial learning. ACM Trans. Inf. Syst. 38, 4 (2020), 1–40.

Digital Library

[180]

D. Rezende, S. Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML.

[181]

M. Richardson, C. Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP.

[182]

H. Rodrigues, Luísa Coheur, and Eric Nyberg. 2016. QGASP: A framework for question generation based on different levels of linguistic information. In INLG.

[183]

Melissa Roemmele, Deep Sidhpura, Steve DeNeefe, and Ling Tsou. 2021. AnswerQuest: A system for generating question-answer items from multi-paragraph documents. arXiv preprint arXiv:2103.03820 (2021).

[184]

Michael Roth and K. Woodsend. 2014. Composition of word representations improves semantic role labelling. In EMNLP.

[185]

Vasile Rus and Graesser Art. 2009. The question generation task and evaluation challenge. In Proceedings of the Workshop Report.

[186]

V. Rus, Z. Cai, and A. Graesser. 2007. Experiments on generating questions about facts. In CICLing.

[187]

V. Rus and J. Lester. 2009. The 2nd workshop on question generation. In AIED.

[188]

Vasile Rus, Brendan Wyse, Paul Piwek, Mihai Lintean, Svetlana Stoyanchev, and Cristian Moldovan. 2010. The first question generation shared task evaluation challenge. In Proceedings of the 6th International Natural Language Generation Conference. 251–257.

Digital Library

[189]

V. Rus, B. Wyse, P. Piwek, Mihai C. Lintean, Svetlana Stoyanchev, and C. Moldovan. 2012. A detailed account of the first question generation shared task evaluation challenge. Dialog. Disc. 3 (2012), 177–204.

[190]

Mrinmaya Sachan and E. Xing. 2018. Self-training for jointly learning to ask and answer questions. In NAACL-HLT.

[191]

Claude Sammut and Ranan B. Banerji. 1986. Learning concepts by asking questions. Mach. Learn.: Artif. Intell. Appr. 2 (1986), 167–192.

[192]

Erik F. Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050 (2003).

[193]

M. Schlichtkrull and Weiwei Cheng. 2020. Evaluating for diversity in question generation over text. ArXiv abs/2008.07291 (2020).

[194]

M. Schlichtkrull, Thomas Kipf, P. Bloem, R. V. Berg, Ivan Titov, and M. Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC.

[195]

Thomas Scialom, Benjamin Piwowarski, and Jacopo Staiano. 2019. Self-attention architectures for answer-agnostic neural question generation. In Proceedings of the 57th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 6027–6032.

[196]

I. Serban, Alberto García-Durán, Çaglar Gülçehre, Sungjin Ahn, A. Chandar, A. Courville, and Y. Bengio. 2016. Generating factoid questions with recurrent neural networks: The 30M factoid question-answer corpus. ArXiv abs/1603.06807 (2016).

[197]

Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.

Digital Library

[198]

Kazutoshi Shinoda and Akiko Aizawa. 2020. Variational question-answer pair generation for machine reading comprehension. ArXiv abs/2004.03238 (2020).

[199]

Michael Heilman. 2011. Automatic Factual Question Generation from Text. Carnegie Mellon University.

Digital Library

[200]

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1631–1642.

[201]

Kihyuk Sohn, H. Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In NIPS.

[202]

Linfeng Song, Zhiguo Wang, and Wael Hamza. 2017. A unified query-based generative model for question generation and question answering. CoRR abs/1709.01058 (2017).

[203]

Linfeng Song, Z. Wang, W. Hamza, Yue Zhang, and Daniel Gildea. 2018. Leveraging context information for natural question generation. In NAACL-HLT.

[204]

Linfeng Song and Lin Zhao. 2016. Domain-specific question generation from a knowledge base. ArXiv abs/1610.03807 (2016).

[205]

Asa Cooper Stickland and I. Murray. 2019. BERT and PALs: Projected attention layers for efficient adaptation in multi-task learning. In ICML.

[206]

Janell Straach and Klaus Truemper. 1999. Learning to ask relevant questions. Artif. Intell. 111, 1 (1999), 301–327.

Digital Library

[207]

Dan Su, Yan Xu, Genta Indra Winata, Peng Xu, H. Kim, Zihan Liu, and Pascale Fung. 2019. Generalizing question answering system with pre-trained language model fine-tuning. In MRQA@EMNLP.

[208]

Sandeep Subramanian, Tong Wang, Xingdi Yuan, and Adam Trischler. 2017. Neural models for key phrase detection and question generation. CoRR abs/1706.04560 (2017).

[209]

Md Arafat Sultan, Shubham Chandel, Ramón Fernández Astudillo, and V. Castelli. 2020. On the importance of diversity in question generation for QA. In ACL.

[210]

Xingwu Sun, Jing Liu, Yajuan Lyu, Wei He, Yanjun Ma, and Shi Wang. 2018. Answer-focused and position-aware neural question generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 3930–3939.

[211]

Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In Proceedings of the 41st International Acm Sigir Conference on Research & Development in Information Retrieval. 235–244.

Digital Library

[212]

Duyu Tang, Nan Duan, Tao Qin, and Ming Zhou. 2017. Question answering and question generation as dual tasks. CoRR abs/1706.02027 (2017).

[213]

Duyu Tang, Nan Duan, T. Qin, and M. Zhou. 2017. Question answering and question generation as dual tasks. ArXiv abs/1706.02027 (2017).

[214]

Duyu Tang, Nan Duan, Zhao Yan, Zhirui Zhang, Yibo Sun, Shujie Liu, Yuanhua Lv, and Ming Zhou. 2018. Learning to collaborate for question answering and asking. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 1564–1574.

[215]

Leila Tavakoli. 2020. Generating clarifying questions in conversational search systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 3253–3256.

Digital Library

[216]

Adam Trischler, T. Wang, Xingdi Yuan, J. Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A machine comprehension dataset. In Rep4NLP@ACL.

[217]

Luu Anh Tuan, Darsh J. Shah, and Regina Barzilay. 2020. Capturing greater context for question generation. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 9065–9072. Retrieved from https://aaai.org/ojs/index.php/AAAI/article/view/6440.

[218]

Leo C. Ureel, Kenneth D. Forbus, Christopher K. Riesbeck, and Lawrence A. Birnbaum. 2005. Question generation for learning by reading. In AAAI Workshop. 22–26.

[219]

Stalin Varanasi, Saadullah Amin, and G. Neumann. 2020. CopyBERT: A unified approach to question generation with self-attention. In NLP4CONVAI.

[220]

A. Varga. 2010. WLV: A question generation system for the QGSTEC 2010 task B.

[221]

Cai Zhiqiang Vasile Rus and Graesser Art. 2008. Question generation: Example of a multi-year evaluation campaign. In WS on the QGSTEC.

[222]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, L. Kaiser, and Illia Polosukhin. 2017. Attention is all you need. ArXiv abs/1706.03762 (2017).

[223]

Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. arXiv preprint arXiv:2004.04228 (2020).

[224]

Bingning Wang, Xiao chuan Wang, Ting Tao, Qi Zhang, and Jingfang Xu. 2020. Neural question generation with answer pivot. In AAAI.

[225]

Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, and Hongning Wang. 2019. Adversarial domain adaptation for machine reading comprehension. ArXiv abs/1908.09209 (2019).

[226]

Siyuan Wang, Zhongyu Wei, Zhihao Fan, Yang Liu, and Xuanjing Huang. A multi-agent communication framework for question-worthy phrase extraction and question generation. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. 7168–7175.

[227]

T. Wang, Xingdi Yuan, and Adam Trischler. 2017. A joint model for question answering and question generation. ArXiv abs/1706.01450 (2017).

[228]

W. Wang, Shi Feng, D. Wang, and Y. Zhang. 2019. Answer-guided and semantic coherent question generation in open-domain conversation. In EMNLP/IJCNLP.

[229]

Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and M. Zhou. 2020. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. ArXiv abs/2002.10957 (2020).

[230]

Yansen Wang, Chenyi Liu, Minlie Huang, and Liqiang Nie. 2018. Learning to ask questions in open-domain conversational systems with typed decoders. In Proceedings of the 56th Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2193–2203.

[231]

Yutong Wang, Jiyuan Zheng, Qijiong Liu, Zhou Zhao, Jun Xiao, and Yueting Zhuang. 2019. Weak supervision enhanced generative network for question generation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). International Joint Conferences on Artificial Intelligence Organization, 3806–3812.

[232]

Zhenduo Wang and Qingyao Ai. 2021. Controlling the risk of conversational search via reinforcement learning. arXiv preprint arXiv:2101.06327 (2021).

[233]

Z. Wang, Andrew S. Lan, Weili Nie, A. Waters, Phillip J. Grimaldi, and Richard Baraniuk. 2018. QG-net: A data-driven question generation model for educational content. In Proceedings of the 5th ACM Conference on Learning at Scale.

Digital Library

[234]

John H. Wolfe. 1976. Automatic question generation from text—An aid to independent study. In SIGCSE. Association for Computing Machinery, 104–112.

[235]

Xiuyu Wu, Nan Jiang, and Yunfang Wu. 2020. A question type driven and copy loss enhanced framework for answer-agnostic neural question generation. ArXiv abs/2005.11665 (2020).

[236]

Dongling Xiao, Han Zhang, Yukun Li, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE-GEN: An enhanced multi-flow pre-training and fine-tuning framework for natural language generation. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20). International Joint Conferences on Artificial Intelligence Organization, 3997–4003.

[237]

Chen Xing, W. Wu, Yu Wu, J. Liu, Yalou Huang, M. Zhou, and W. Ma. 2017. Topic aware neural response generation. In AAAI.

[238]

Xinchen Yan, Jimei Yang, Kihyuk Sohn, and H. Lee. 2016. Attribute2Image: Conditional image generation from visual attributes. ArXiv abs/1512.00570 (2016).

[239]

Yu Yan, Weizhen Qi, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. ProphetNet: Predicting future n-gram for sequence-to-sequence pre-training. CoRR abs/2001.04063 (2020).

[240]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 5753–5763.

[241]

Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, and William Cohen. 2017. Semi-supervised QA with generative domain-adaptive nets. In Proceedings of the 55th Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1040–1050.

[242]

Z. Yang, Peng Qi, Saizheng Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. ArXiv abs/1809.09600 (2018).

[243]

Kaichun Yao, L. Zhang, Tiejian Luo, Lili Tao, and Y. Wu. 2018. Teaching machines to ask questions. In IJCAI.

[244]

Xuchen Yao. 2010. Question generation with minimal recursion semantics. In Proceedings of QG2010: The Third Workshop on Question Generation. 68–75.

[245]

Xuchen Yao, Emma Tosch, G. Chen, E. Nouri, Ron Artstein, A. Leuski, Kenji Sagae, and D. Traum. 2012. Creating conversational characters using question generation tools. Dialog. Disc. 3 (2012), 125–146.

[246]

Xuchen Yao and Yi Zhang. 2010. Question generation with minimal recursion semantics. In Proceedings of the 3rd Workshop on Question Generation. Citeseer, 68–75.

[247]

Qian Yu, Lidong Bing, Qiong Zhang, Wai Lam, and Luo Si. 2020. Review-based question generation with adaptive instance transfer and augmentation. In Proceedings of the 58th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 280–290.

[248]

Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Saizheng Zhang, Sandeep Subramanian, and Adam Trischler. 2017. Machine comprehension by text-to-text neural question generation. In Proceedings of the 2nd Workshop on Representation Learning for NLP. Association for Computational Linguistics, 15–25.

[249]

Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, and Gord Lueck. 2020. Generating clarifying questions for information retrieval. In Proceedings of the Web Conference. 418–428.

Digital Library

[250]

Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul N. Bennett, Nick Craswell, and Susan T. Dumais. 2020. Analyzing and learning from user interactions for search clarification. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1181–1190.

[251]

Poorya Zaremoodi, Wray L. Buntine, and Gholamreza Haffari. 2018. Adaptive knowledge sharing in multi-task learning: Improving low-resource neural machine translation. In ACL.

[252]

Jingqing Zhang, Y. Zhao, Mohammad Saleh, and Peter J. Liu. 2019. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. ArXiv abs/1912.08777 (2019).

[253]

R. Zhang, J. Guo, Yixing Fan, Yanyan Lan, and Xueqi Cheng. 2020. Dual-factor generation model for conversation. ACM Trans. Inf. Syst. 38 (2020), 1–31.

Digital Library

[254]

R. Zhang, J. Guo, Y. Fan, Yanyan Lan, J. Xu, and X. Cheng. 2018. Learning to control the specificity in neural response generation. In ACL.

[255]

Shiyue Zhang and Mohit Bansal. 2019. Addressing semantic drift in question generation for semi-supervised question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 2495–2509.

[256]

S. Zhang, Lizhen Qu, Shaodi You, Zhenglu Yang, and Jiawan Zhang. 2017. Automatic generation of grounded visual questions. ArXiv abs/1612.06530 (2017).

[257]

Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 177–186.

[258]

Y. Zhang and Qiang Yang. 2017. A survey on multi-task learning. ArXiv abs/1707.08114 (2017).

[259]

Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 3901–3910.

[260]

Hai-Tao Zheng, Jinxin Han, J. Chen, and A. K. Sangaiah. 2018. A novel framework for automatic Chinese question generation based on multi-feature neural network model. Comput. Sci. Inf. Syst. 15 (2018), 487–499.

[261]

Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2018. Neural question generation from text: A preliminary study. In Natural Language Processing and Chinese Computing, Xuanjing Huang, Jing Jiang, Dongyan Zhao, Yansong Feng, and Yu Hong (Eds.). Springer International Publishing, Cham, 662–671.

[262]

Qingyu Zhou, Nan Yang, Furu Wei, and M. Zhou. 2018. Sequential copying networks. In AAAI.

[263]

Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019. Multi-task learning with language modeling for question generation. ArXiv abs/1908.11813 (2019).

[264]

Wenjie Zhou, Minghua Zhang, and Yunfang Wu. 2019. Question-type driven question generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 6032–6037.

[265]

Kangli Zi, Xingwu Sun, Yanan Cao, Shi Wang, X. Feng, Zhaobo Ma, and C. Cao. 2019. Answer-focused and position-aware neural network for transfer learning in question generation. In KSEM.

[266]

Barret Zoph, Deniz Yuret, Jonathan May, and K. Knight. 2016. Transfer learning for low-resource neural machine translation. ArXiv abs/1604.02201 (2016).

[267]

Çaglar Gülçehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. Pointing the unknown words. ArXiv abs/1603.08148 (2016).

Cited By

Sindiramutty SPrabagaran KAkbar RHussain MMalik N(2024)Overview of Generative AI Techniques for CybersecurityReshaping CyberSecurity With Generative AI Techniques10.4018/979-8-3693-5415-5.ch001(1-52)Online publication date: 26-Jul-2024
https://doi.org/10.4018/979-8-3693-5415-5.ch001
Lopez CMorrison MDeacon M(2024)Modelos de lenguaje para la generación de preguntas de programación con diferentes niveles de dificultadLanguage models for generating programming questions with varying difficulty levelsEuropean Public & Social Innovation Review10.31637/epsir-2024-7609(1-19)Online publication date: 12-Sep-2024
https://doi.org/10.31637/epsir-2024-760
Utami IHwang WHariyanti U(2024)Contextualized and Personalized Math Word Problem Generation in Authentic Contexts Using Generative Pre-trained Transformer and Its Influences on Geometry LearningJournal of Educational Computing Research10.1177/07356331241249225Online publication date: 29-May-2024
https://doi.org/10.1177/07356331241249225
Show More Cited By

Index Terms

A Review on Question Generation from Natural Language Text

Index terms have been assigned to the content through auto-classification.

Recommendations

Quiz-Style Question Generation for News Stories
WWW '21: Proceedings of the Web Conference 2021

A large majority of American adults get at least some of their news from the Internet. Even though many online news products have the goal of informing their users about the news, they lack scalable and reliable tools for measuring how well they are ...
A Survey of Natural Language Generation
This article offers a comprehensive review of the research on Natural Language Generation (NLG) over the past two decades, especially in relation to data-to-text generation and text-to-text generation deep learning methods, as well as new applications of ...
Research on Text Generation Techniques Combining Machine Learning and Deep Learning
IPEC '22: Proceedings of the 3rd Asia-Pacific Conference on Image Processing, Electronics and Computers

Natural language generation (NLG) is a part of natural language processing (NLP), the main purpose of which is to build a natural language text generation system capable of generating human-understandable languages such as Chinese and English through ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems

ACM Transactions on Information Systems Volume 40, Issue 1

January 2022

599 pages

ISSN:1046-8188

EISSN:1558-2868

DOI:10.1145/3483337

Editor:
Min Zhang
Tsinghua University, China

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2021

Accepted: 01 May 2021

Revised: 01 May 2021

Received: 01 March 2021

Published in TOIS Volume 40, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China (NSFC)
Beijing Academy of Artificial Intelligence (BAAI)
Youth Innovation Promotion Association CAS
Lenovo-CAS Joint Lab Youth Scientist Project
K.C. Wong Education Foundation
Foundation and Frontier Research Key Program of Chongqing Science and Technology Commission

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
4,456
Total Downloads

Downloads (Last 12 months)2,123
Downloads (Last 6 weeks)278

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sindiramutty SPrabagaran KAkbar RHussain MMalik N(2024)Overview of Generative AI Techniques for CybersecurityReshaping CyberSecurity With Generative AI Techniques10.4018/979-8-3693-5415-5.ch001(1-52)Online publication date: 26-Jul-2024
https://doi.org/10.4018/979-8-3693-5415-5.ch001
Lopez CMorrison MDeacon M(2024)Modelos de lenguaje para la generación de preguntas de programación con diferentes niveles de dificultadLanguage models for generating programming questions with varying difficulty levelsEuropean Public & Social Innovation Review10.31637/epsir-2024-7609(1-19)Online publication date: 12-Sep-2024
https://doi.org/10.31637/epsir-2024-760
Utami IHwang WHariyanti U(2024)Contextualized and Personalized Math Word Problem Generation in Authentic Contexts Using Generative Pre-trained Transformer and Its Influences on Geometry LearningJournal of Educational Computing Research10.1177/07356331241249225Online publication date: 29-May-2024
https://doi.org/10.1177/07356331241249225
Alajmi AAltabaa HAbed SAhmad I(2024)Arabic Question Generation Using TransformersACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3701559Online publication date: 28-Oct-2024
https://doi.org/10.1145/3701559
Pham QLe HDang Nhat MTran T. KTran-Tien MDang VVu HNguyen MPhan X(2024)Towards Vietnamese Question and Answer Generation: An Empirical StudyACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3675781Online publication date: 29-Jun-2024
https://doi.org/10.1145/3675781
Ekle OEberle W(2024)Anomaly Detection in Dynamic Graphs: A Comprehensive SurveyACM Transactions on Knowledge Discovery from Data10.1145/3669906Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3669906
Hu JGuo JTang NMa XYao YYang CXu Y(2024)Designing the Conversational Agent: Asking Follow-up Questions for Information ElicitationProceedings of the ACM on Human-Computer Interaction10.1145/36373208:CSCW1(1-30)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637320
Almeida JSilva JAfonso LAlmeida TAntunes RJonker RReis JSilva DMatos SOliveira J(2024)HealthDBFinder: a question-answering task for health database discovery2024 IEEE 37th International Symposium on Computer-Based Medical Systems (CBMS)10.1109/CBMS61543.2024.00025(100-105)Online publication date: 26-Jun-2024
https://doi.org/10.1109/CBMS61543.2024.00025
Alshibly HAlwreikat AMorgos RAbuaddous M(2024)Examining the mediating role of customer empowerment: the impact of chatbot usability on customer satisfaction in Jordanian commercial banksCogent Business & Management10.1080/23311975.2024.238719611:1Online publication date: 8-Aug-2024
https://doi.org/10.1080/23311975.2024.2387196
Alhashedi SSuaib NBakri A(2024)Arabic automatic question generation using transformer modelPROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON GREEN ENGINEERING & TECHNOLOGY 2022 (ICONGETECH 2022)10.1063/5.0199032(050035)Online publication date: 2024
https://doi.org/10.1063/5.0199032
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents