Asking the Crowd: Question Analysis, Evaluation and Generation for Open Discussion on Online Forums

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5032–5046 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5032 Asking the Crowd: Question Analysis, Evaluation and Generation for Open Discussion on Online Forums Zi Chai1,2, Xinyu Xing1,2, Xiaojun Wan1,2, Bo Huang3 1Institute of Computer Science and Technology, Peking University 2The MOE Key Laboratory of Computational Linguistics, Peking University 3Zhihu Institude {chaizi, xingxinyu, wanxiaojun}@pku.edu.cn, huangbo@zhihu.com Abstract Teaching machines to ask questions is an im- portant yet challenging task. Most prior work focused on generating questions with fixed an- swers. As contents are highly limited by given answers, these questions are often not worth discussing. In this paper, we take the first step on teaching machines to ask open-answered questions from real-world news for open dis- cussion (openQG). To generate high-qualified questions, effective ways for question evalua- tion are required. We take the perspective that the more answers a question receives, the bet- ter it is for open discussion, and analyze how language use affects the number of answers. Compared with other factors, e.g. topic and post time, linguistic factors keep our evalua- tion from being domain-specific. We carefully perform variable control on 11.5M questions from online forums to get a dataset, OQRanD, and further perform question analysis. Based on these conclusions, several models are built for question evaluation. For openQG task, we construct OQGenD, the first dataset as far as we know, and propose a model based on con- ditional generative adversarial networks and our question evaluation model. Experiments show that our model can generate questions with higher quality compared with commonly- used text generation methods. 1 Introduction Teaching machines to ask questions from given corpus, i.e. question generation (QG), is an im- portant yet challenging task in natural language processing. In recent years, QG has received in- creasing attention from both the industrial and academic communities due to its wide applica- tions. Dialog systems can be proactive by ask- ing users questions (Wang et al., 2018), question answering (QA) systems can benefit from the cor- pus produced by a QG model (Duan et al., 2017), education (Heilman and Smith, 2010) and clini- cal (Weizenbaum et al., 1966; Colby et al., 1971) systems require QG as well. We can divide all questions into two categories. Fixed-answered questions have standard answers, e.g. “who invented the car? (Karl Benz)”. In con- trast, different people may have distinct answers over open-answered questions like “what do you think of the self-driving car?”. Most prior work about QG (QA) aimed to generate (answer) fixed- answered questions. As questions are targeting on answers which are certain spans of given corpus, they are always not worth discussing. Nowadays, with the help of online QA forums (e.g. Quora and Zhihu1), open-answered questions can greatly arouse open discussion that helps people under different backgrounds to share knowledge and ideas (high-qualified questions can help to attract more visitors for QA forums as well). This kind of questions are also useful for many …

Content truncated to 500 words