Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1554 - 1576 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue Jiawei Zhoum∗ Jason Eisner3 Michael Newman3 Emmanouil Antonios Platanios3 Sam Thomson3 mHarvard University 3Microsoft Semantic Machines mjzhou02@g.harvard.edu 3{jason.eisner,mike.newman, anthony.platanios,samuel.thomson}@microsoft.com Abstract Standard conversational semantic parsing maps a complete user utterance into an exe- cutable program, after which the program is executed to respond to the user. This could be slow when the program contains expensive function calls. We investigate the opportunity to reduce latency by predicting and executing function calls while the user is still speaking. We introduce the task of online semantic pars- ing for this purpose, with a formal latency re- duction metric inspired by simultaneous ma- chine translation. We propose a general frame- work with first a learned prefix-to-program pre- diction module, and then a simple yet effec- tive thresholding heuristic for subprogram se- lection for early execution. Experiments on the SMCalFlow and TreeDST datasets show our approach achieves large latency reduction with good parsing quality, with a 30%–63% la- tency reduction depending on function execu- tion time and allowed cost. 1 Introduction In task-oriented dialogue systems, a software agent typically translates a user’s intent into a program, executes it to query information sources (e.g., find a person in the user’s contact list) or effect external actions (e.g., schedule a meeting or send an email), then communicates the results back to the user. If an agent waits to begin this process until the user finishes speaking, there is a noticeable lag before the user receives a response. The complex intents in the datasets SMCalFlow (Semantic Machines et al., 2020) and TreeDST (Cheng et al., 2020), for example, can be slow to execute, nesting up to 7 slow function calls that cannot be parallelized. Inspired by simultaneous machine translation, we ask: How much can latency be reduced by inter- preting and executing early, before the user finishes speaking? ∗ Work performed during a research internship at Mi- crosoft Semantic Machines. FindPerson “Barack Obama” Create Event subject “pool party” start DateAt Time Tomorrow AM 9 attendees contains FindPerson… Yield Create Event… Utterance End Add a pool party with Barack Obama for tomorrow at 9 : 00 AM Execution End Utterance Start Figure 1: A possible execution timeline for “Add a pool party with Barack Obama for tomorrow at 9:00 AM”. FindPerson was predicted and executed af- ter the token “Obama.” CreateEvent was pre- dicted and executed after the token “9” (AM was guessed). Yield was also predicted after the to- ken “9”, but could not begin until CreateEvent fin- ished. Because FindPerson and CreateEvent started execution before the end of the utterance, the top-level node Yield was able to finish early, and overall latency was reduced. Gray nodes have near- instantaneous execution, and do not contribute to the timeline. Edge labels were omitted to save space. In general, an agent could begin speculatively …

Content truncated to 500 words