Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2969–2978 June 6–11, 2021. ©2021 Association for Computational Linguistics 2969 Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog Arun Babu Akshat Shrivastava Armen Aghajanyan Ahmed Aly Angela Fan FacebookMarjan Ghazvininejad {arbabu, akshats, armenag, ahhegazy, angelafan, ghazvini}@fb.com Abstract Semantic parsing using sequence-to-sequence models allows parsing of deeper representa- tions compared to traditional word tagging based models. In spite of these advan- tages, widespread adoption of these models for real-time conversational use cases has been stymied by higher compute requirements and thus higher latency. In this work, we propose a non-autoregressive approach to predict se- mantic parse trees with an efficient seq2seq model architecture. By combining non- autoregressive prediction with convolutional neural networks, we achieve significant la- tency gains and parameter size reduction com- pared to traditional RNN models. Our novel architecture achieves up to an 81% reduction in latency on TOP dataset and retains com- petitive performance to non-pretrained mod- els on three different semantic parsing datasets. Our code is available at https://github. com/facebookresearch/pytext. 1 Introduction Advances in conversational assistants have helped to improve the usability of smart speakers and con- sumer wearables for different tasks. Semantic pars- ing is one of the fundamental components of these assistants and it helps to convert the user input in natural language to a structure representation that can be understood by downstream systems. Ma- jority of the semantic parsing systems deployed on various devices, rely on server-side inference because of the lower compute/memory available on these edge devices. This poses a few drawbacks such as flaky user experience with spotty internet connectivity and compromised user data privacy due to the dependence on a centralized server to which all user interactions are sent to. Thus, seman- tic parsing on-device has numerous advantages. For the semantic parsing task, the meaning rep- resentation used decides the capabilities of the sys- tem built. Limitations of the representation with one intent and slot labels were studied in the con- text of nested queries and multi turn utterances in Aghajanyan et al. (2020) and Gupta et al. (2018). New representations were proposed to overcome these limitations and sequence-to-sequence mod- els were proposed as the solution to model these complex forms. But using these new models in real- time conversational assistants still remains a chal- lenge due to higher latency requirements. In our work, we propose a novel architecture and genera- tion scheme to significantly improve the end2end latency of sequence-to-sequence models for the semantic parsing task. Due to the autoregressive nature of generation in sequence-to-sequence semantic parsing models, the recurrence relationship between target tokens creates a limitation that decoding cannot be paral- lelized. There are multiple works in machine translation which try to solve this problem. These approaches relax the decoder token-by-token generation by al- lowing multiple target tokens to be generated at once. Fully non-autoregressive models (Gu et al., 2017; Ma et al., 2019; Ghazvininejad et al., 2020a; Saharia et …

Content truncated to 500 words