No Need to Pay Attention: Simple Recurrent Neural Networks Work!

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2866–2872 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics No Need to Pay Attention: Simple Recurrent Neural Networks Work! (for Answering “Simple” Questions) Ferhan Ture and Oliver Jojic Comcast Labs, Washington, DC 20005 {ferhan ture, oliver jojic}@cable.comcast.com Abstract First-order factoid question answering as- sumes that the question can be answered by a single fact in a knowledge base (KB). While this does not seem like a challeng- ing task, many recent attempts that ap- ply either complex linguistic reasoning or deep neural networks achieve 65%–76% accuracy on benchmark sets. Our ap- proach formulates the task as two machine learning problems: detecting the entities in the question, and classifying the question as one of the relation types in the KB. We train a recurrent neural network to solve each problem. On the SimpleQuestions dataset, our approach yields substantial improvements over previously published results — even neural networks based on much more complex architectures. The simplicity of our approach also has prac- tical advantages, such as efficiency and modularity, that are valuable especially in an industry setting. In fact, we present a preliminary analysis of the performance of our model on real queries from Comcast’s X1 entertainment platform with millions of users every day. 1 Introduction First-order factoid question answering (QA) as- sumes that the question can be answered by a single fact in a knowledge base (KB). For ex- ample, “How old is Tom Hanks” is about the [age] of [Tom Hanks]. Also referred to as simple questions by Bordes et al. (2015), recent attempts that apply either complex linguistic reasoning or attention-based complex neural network architec- tures achieve up to 76% accuracy on benchmark sets (Golub and He, 2016; Yin et al., 2016). While it is tempting to study QA systems that can handle more complicated questions, it is hard to reach rea- sonably high precision for unrestricted questions. For more than a decade, successful industry ap- plications of QA have focused on first-order ques- tions. This bears the question: are users even in- terested in asking questions beyond first-order (or are these use cases more suitable for interactive dialogue)? Based on voice logs from a major en- tertainment platform with millions of users every day, Comcast X1, we find that most existing use cases of QA fall into the first-order category. Our strategy is to tailor our approach to first- order QA by making strong assumptions about the problem structure. In particular, we assume that the answer to a first-order question is a sin- gle property of a single entity in the KB, and de- compose the task into two subproblems: (a) de- tecting entities in the question and (b) classify- ing the question as one of the relation types in the KB. We simply train a vanilla recurrent neu- ral network (RNN) to solve each subproblem (El- man, 1990). Despite its simplicity, our approach (RNN-QA) achieves the highest reported accu- racy on the …

Content truncated to 500 words