Neural Generative Question Answering

Proceedings of 2016 NAACL Human-Computer Question Answering Workshop, pages 36–42, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Neural Generative Question Answering Jun Yin1∗ Xin Jiang2 Zhengdong Lu2, Lifeng Shang2, Hang Li2, Xiaoming Li1 1School of Electronic Engineering and Computer Science, Peking University 2Noah’s Ark Lab, Huawei Technologies {jun.yin,lxm}@pku.edu.cn, {jiang.xin, lu.zhengdong, shang.lifeng, hangli.hl}@huawei.com Abstract This paper presents an end-to-end neural net- work model, named Neural Generative Ques- tion Answering (GENQA), that can generate answers to simple factoid questions, based on the facts in a knowledge-base. More specif- ically, the model is built on the encoder- decoder framework for sequence-to-sequence learning, while equipped with the ability to en- quire the knowledge-base, and is trained on a corpus of question-answer pairs, with their as- sociated triples in the knowledge-base. Em- pirical study shows the proposed model can effectively deal with the variations of ques- tions and answers, and generate right and nat- ural answers by referring to the facts in the knowledge-base. The experiment on ques- tion answering demonstrates that the proposed model can outperform an embedding-based QA model as well as a neural dialogue model trained on the same data. 1 Introduction Question answering (QA) can be viewed as a special case of single-turn dialogue: QA aims at providing correct answers to the questions in natural language, while dialogue emphasizes on generating relevan- t and fluent responses to the messages also in natural language (Shang et al., 2015; Vinyals and Le, 2015). Recent progress in deep learning has raised the pos- sibility of realizing generation-based QA in a purely neutralized way. That is, the answer is generated by a neural network (e.g., recurrent neural network, or ∗ The work is done when the first author worked as intern at Noah’s Ark Lab, Huawei Technologies. RNN) based on the question, which is able to handle the flexibility and diversity of language. More im- portantly, the model is trained in an end-to-end fash- ion, and thus there is no need in building the system using linguistic knowledge, e.g., creating a semantic parser. There is however one serious limitation of this generation-based approach to QA. It is practically impossible to store all the knowledge in a neural network to achieve a desired precision and cover- age in real world QA. This is a fundamental diffi- culty, rooting deeply in the way in which knowledge is acquired, represented and stored. The neural net- work, and more generally the fully distributed way of representation, is good at representing smooth and shared patterns, i.e., modeling the flexibility and diversity of language, but improper for representing discrete and isolated concepts, i.e., depicting the lex- icon of language. On the other hand, the recent success of memory- based neural network models has greatly extended the ways of storing and accessing text information, in both short-term memory (e.g., in (Bahdanau et al., 2015)) and long-term memory (e.g., in (West- on et al., 2015)). It is hence a natural choice to connect a neural model for QA with …

Content truncated to 500 words