Task-Oriented Query Reformulation with Reinforcement Learning

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 574–583 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Task-Oriented Query Reformulation with Reinforcement Learning Rodrigo Nogueira Tandon School of Engineering New York University rodrigonogueira@nyu.edu Kyunghyun Cho Courant Institute of Mathematical Sciences Center for Data Science New York University kyunghyun.cho@nyu.edu Abstract Search engines play an important role in our everyday lives by assisting us in find- ing the information we need. When we input a complex query, however, results are often far from satisfactory. In this work, we introduce a query reformula- tion system based on a neural network that rewrites a query to maximize the number of relevant documents returned. We train this neural network with reinforcement learning. The actions correspond to select- ing terms to build a reformulated query, and the reward is the document recall. We evaluate our approach on three datasets against strong baselines and show a rel- ative improvement of 5-20% in terms of recall. Furthermore, we present a simple method to estimate a conservative upper- bound performance of a model in a partic- ular environment and verify that there is still large room for improvements. 1 Introduction Search engines help us find what we need among the vast array of available data. When we request some information using a long or inexact descrip- tion of it, these systems, however, often fail to de- liver relevant items. In this case, what typically follows is an iterative process in which we try to express our need differently in the hope that the system will return what we want. This is a major issue in information retrieval. For instance, Huang and Efthimiadis (2009) estimate that 28-52% of all the web queries are modifications of previous ones. To a certain extent, this problem occurs be- cause search engines rely on matching words in the query with words in relevant documents, to q' Scorer Reformulator Ground Truth D' Search Engine q' : cancer treatment state-of-the-art frontiers survey Reward q0 : What are the most promising directions to cure cancer and why? D0 Figure 1: A graphical illustration of the proposed framework for query reformulation. A set of doc- uments D0 is retrieved from a search engine using the initial query q0. Our reformulator selects terms from q0 and D0 to produce a reformulated query q′ which is then sent to the search engine. Docu- ments D′ are returned, and a reward is computed against the set of ground-truth documents. The re- formulator is trained with reinforcement learning to produce a query, or a series of queries, to maxi- mize the expected return. perform retrieval. If there is a mismatch between them, a relevant document may be missed. One way to address this problem is to automati- cally rewrite a query so that it becomes more likely to retrieve relevant documents. This technique is known as automatic query reformulation. It typi- cally expands the original query by adding terms from, for instance, dictionaries of synonyms …

Content truncated to 500 words