Learning a Neural Semantic Parser from User Feedback

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 963–973 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1089 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 963–973 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1089 Learning a Neural Semantic Parser from User Feedback Srinivasan Iyer†⋄, Ioannis Konstas†, Alvin Cheung† Jayant Krishnamurthy‡ and Luke Zettlemoyer†‡ †Paul G. Allen School of Computer Science & Engineering, Univ. of Washington, Seattle, WA {sviyer,ikonstas,akcheung,lsz}@cs.washington.edu ‡Allen Institute for Artificial Intelligence, Seattle, WA {jayantk,lukez}@allenai.org Abstract We present an approach to rapidly and easily build natural language interfaces to databases for new domains, whose perfor- mance improves over time based on user feedback, and requires minimal interven- tion. To achieve this, we adapt neural se- quence models to map utterances directly to SQL with its full expressivity, bypass- ing any intermediate meaning representa- tions. These models are immediately de- ployed online to solicit feedback from real users to flag incorrect queries. Finally, the popularity of SQL facilitates gathering annotations for incorrect predictions using the crowd, which is directly used to im- prove our models. This complete feedback loop, without intermediate representations or database specific engineering, opens up new ways of building high quality seman- tic parsers. Experiments suggest that this approach can be deployed quickly for any new target domain, as we show by learning a semantic parser for an online academic database from scratch. 1 Introduction Existing semantic parsing approaches for building natural language interfaces to databases (NLIDBs) either use special-purpose intermediate meaning representations that lack the full expressivity of database query languages or require extensive fea- ture engineering, making it difficult to deploy them in new domains. We present a robust ap- proach to quickly and easily learn and deploy se- mantic parsers from scratch, whose performance ⋄Work done partly during an internship at the Allen Insti- tute for Artificial Intelligence. Most recent papers of Michael I. Jordan SELECT paper.paperId , paper.year FROM paper , writes , author WHERE paper.paperId = writes.paperId AND writes.authorId = author.authorId AND author.authorName = "michael i. jordan" AND paper.year = (SELECT max(paper.year) FROM paper , writes , author WHERE paper.paperId = writes.paperId AND writes.authorId = author.authorId AND author.authorName = "michael i. jordan"); I’d like to book a flight from San Diego to Toronto SELECT DISTINCT f1.flight_id FROM flight f1 , airport_service a1 , city c1 , airport_service a2 , city c2 WHERE f1.from_airport = a1.airport_code AND a1.city_code = c1.city_code AND c1.city_name = 'san diego' AND f1.to_airport = a2.airport_code AND a2.city_code = c2.city_code AND c2.city_name = 'toronto '; Figure 1: Utterances with corresponding SQL queries to answer them for two domains, an aca- demic database and a flight reservation database. improves over time based on user feedback, and requires very little expert intervention. To learn these semantic parsers, we (1) adapt neural sequence models to map utterances directly to SQL thereby bypassing intermediate represen- tations and taking full advantage of SQL’s …

Content truncated to 500 words