Language to Logical Form with Neural Attention

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 33–43, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Language to Logical Form with Neural Attention Li Dong and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB li.dong@ed.ac.uk, mlap@inf.ed.ac.uk Abstract Semantic parsing aims at mapping nat- ural language to machine interpretable meaning representations. Traditional ap- proaches rely on high-quality lexicons, manually-built templates, and linguis- tic features which are either domain- or representation-specific. In this pa- per we present a general method based on an attention-enhanced encoder-decoder model. We encode input utterances into vector representations, and generate their logical forms by conditioning the output sequences or trees on the encoding vec- tors. Experimental results on four datasets show that our approach performs compet- itively without using hand-engineered fea- tures and is easy to adapt across domains and meaning representations. 1 Introduction Semantic parsing is the task of translating text to a formal meaning representation such as log- ical forms or structured queries. There has re- cently been a surge of interest in developing ma- chine learning methods for semantic parsing (see the references in Section 2), due in part to the existence of corpora containing utterances anno- tated with formal meaning representations. Fig- ure 1 shows an example of a question (left hand- side) and its annotated logical form (right hand- side), taken from JOBS (Tang and Mooney, 2001), a well-known semantic parsing benchmark. In or- der to predict the correct logical form for a given utterance, most previous systems rely on prede- fined templates and manually designed features, which often render the parsing model domain- or representation-specific. In this work, we aim to use a simple yet effective method to bridge the gap between natural language and logical form with minimal domain knowledge. Sequence Encoder Sequence/Tree Decoder LSTM answer(J,(compa ny(J,'microsoft'),j ob(J),not((req_de g(J,'bscs'))))) Attention Layer LSTM what microsoft jobs do not require a bscs? Input Utterance Logical Form Figure 1: Input utterances and their logical forms are encoded and decoded with neural networks. An attention layer is used to learn soft alignments. Encoder-decoder architectures based on recur- rent neural networks have been successfully ap- plied to a variety of NLP tasks ranging from syn- tactic parsing (Vinyals et al., 2015a), to machine translation (Kalchbrenner and Blunsom, 2013; Cho et al., 2014; Sutskever et al., 2014), and image description generation (Karpathy and Fei- Fei, 2015; Vinyals et al., 2015b). As shown in Figure 1, we adapt the general encoder-decoder paradigm to the semantic parsing task. Our model learns from natural language descriptions paired with meaning representations; it encodes sentences and decodes logical forms using recur- rent neural networks with long short-term memory (LSTM) units. We present two model variants, the first one treats semantic parsing as a vanilla sequence transduction task, whereas our second model is equipped with a hierarchical tree decoder which explicitly captures the compositional struc- ture of logical forms. We also introduce …

Content truncated to 500 words