Don’t Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 496–506 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 496 Don’t Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding Qile Zhu1∗, Haidar Khan2, Saleh Soltan2, Stephen Rawls2, and Wael Hamza2 1University of Florida, 2Amazon Alexa AI valder@ufl.edu {khhaida,ssoltan,sterawls,waelhamz}@amazon.com Abstract Semantic parsing is one of the key components of natural language understanding systems. A successful parse transforms an input utterance to an action that is easily understood by the system. Many algorithms have been proposed to solve this problem, from conventional rule- based or statistical slot-filling systems to shift- reduce based neural parsers. For complex pars- ing tasks, the state-of-the-art method is based on autoregressive sequence to sequence mod- els to generate the parse directly. This model is slow at inference time, generating parses in O(n) decoding steps (n is the length of the target sequence). In addition, we demonstrate that this method performs poorly in zero-shot cross-lingual transfer learning settings. In this paper, we propose a non-autoregressive parser which is based on the insertion transformer to overcome these two issues. Our approach 1) speeds up decoding by 3x while outperform- ing the autoregressive model and 2) signifi- cantly improves cross-lingual transfer in the low-resource setting by 37% compared to au- toregressive baseline. We test our approach on three well-known monolingual datasets: ATIS, SNIPS and TOP. For cross lingual semantic parsing, we use the MultiATIS++ and the mul- tilingual TOP datasets. 1 Introduction Given a query, a semantic parsing module identifies not only the intent (play music, book a flight) of the query but also extracts necessary slots (entities) that further refines the action to perform (which song to play? Where or when to go?). A traditional rule- based or slot-filling system classifies a query with one intent and tags each input token (Mesnil et al., 2013). However, supporting more complex queries that are composed of multiple intents and nested slots is a challenging problem (Gupta et al., 2018). ∗Work done while interning at Amazon Alexa Gupta et al. (2018) and Einolghozati et al. (2019) propose to use a Shift-Reduce parser based on Re- current Neural Network for these complex queries. Recently, Rongali et al. (2020) propose directly generating the parse as a formatted sequence and design a unified model based on sequence to se- quence generation and pointer networks. Their approach formulates the tagging problem into a generation task in which the target is constructed by combining all the necessary intents and slots in a flat sequence with no restriction on the semantic parse schema. A relatively unexplored direction is the cross- lingual transfer problem (Duong et al., 2017; Su- santo and Lu, 2017), where the parsing system is trained in a high-resource language and transfered directly to a low-resource language (zero-shot). The state-of-the-art model leverages the autore- gressive decoder such as Transformer (Vaswani et al., 2017) and Long-Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) to generate the target sequence (representing the parse) from left to …

Content truncated to 500 words