AdaNSP: Uncertainty-driven Adaptive Decoding in Neural Semantic Parsing

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4265–4270 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4265 AdaNSP: Uncertainty-driven Adaptive Decoding in Neural Semantic Parsing Xiang Zhang1,2, Shizhu He2, Kang Liu1,2, Jun Zhao1,2 1University of Chinese Academy of Sciences, Beijing, 100049, China 2National Laboratory of Pattern Recognition (NLPR), Institute of Automation Chinese Academy of Sciences, Beijing, 100190, China {xiang.zhang, shizhu.he, kliu, jzhao}@nlpr.ia.ac.cn Abstract Neural semantic parsers utilize the encoder- decoder framework to learn an end-to-end model for semantic parsing that transduces a natural language sentence to the formal se- mantic representation. To keep the model aware of the underlying grammar in target se- quences, many constrained decoders were de- vised in a multi-stage paradigm, which decode to the sketches or abstract syntax trees first, and then decode to target semantic tokens. We instead to propose an adaptive decoding method to avoid such intermediate represen- tations. The decoder is guided by model un- certainty and automatically uses deeper com- putations when necessary. Thus it can pre- dict tokens adaptively. Our model outperforms the state-of-the-art neural models and does not need any expertise like predefined grammar or sketches in the meantime. 1 Introduction Semantic Parsing (SP) maps a natural language utterance into a formal language, which is cru- cial in abundant tasks, such as question answering (Zettlemoyer and Collins, 2005, 2007) and code generation (Yin and Neubig, 2017). The prevail- ing neural semantic parsers view semantic pars- ing as a sequence transduction task, and adopt the encoder-decoder framework similar to machine translation. The distinguishing difference of semantic pars- ing, however, is in its target sequences, which are token sequences of well-formed semantic rep- resentations. SQL language and lambda expres- sions are typical examples of SP targets. The “SELECT..FROM..WHERE” pattern in SQL and the paired parentheses in lambda expressions are consequences of underlying grammars. However, standard Seq2Seq models ignore the patterns and may give ill-formed results. To better model the grammatical and semanti- cal constraints, many decoding methods were de- vised. Dong and Lapata (2018) proposed to gen- erate tokens of an intermediate sketch first, fol- lowed by decoding into final formal targets. Oth- ers chose to gradually build abstract syntax trees using a transition-based paradigm, and tokens are generated at the tree leaves or in the middle of the transitions (Krishnamurthy et al., 2017; Chen et al., 2018; Yin and Neubig, 2018). There are also some decoders comprised of several submod- ules which are intended to generate different parts of the semantic output, respectively (Yu et al., 2018a,b). However, the aforementioned methods still have the following key issue. They explicitly require the expertise to design intermediate repre- sentations or model structures, which is not ideal or acceptable for scenarios with Domain Specific Languages (DSL) or new representations because of domain alterations and the incompleteness of the expert knowledge. To follow the successful idea and overcome the above issue, we introduce a novel adaptive decod- ing mechanism. Inspired by adaptive computing (Graves, …

Content truncated to 500 words