Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systems

Last Words Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systems Manaal Faruqui Google Assistant mfaruqui@google.com Dilek Hakkani-T¨ur Amazon Alexa AI hakkanit@amazon.com As more users across the world are interacting with dialog agents in their daily life, there is a need for better speech understanding that calls for renewed attention to the dynamics between research in automatic speech recognition (ASR) and natural language understanding (NLU). We briefly review these research areas and lay out the current relationship between them. In light of the observations we make in this article, we argue that (1) NLU should be cognizant of the presence of ASR models being used upstream in a dialog system’s pipeline, (2) ASR should be able to learn from errors found in NLU, (3) there is a need for end-to-end data sets that provide semantic annotations on spoken input, (4) there should be stronger collaboration between ASR and NLU research communities. 1. Introduction More and more users every day are communicating with conversational dialog systems present around them like Apple Siri, Amazon Alexa, and Google Assistant. As of 2019, 31% of the broadband households in the United States have a digital assistant.1 Henceforth, we refer to these systems as dialog agents or simply agents. A majority of queries issued to these dialog agents are in the form of speech as the users are directly talking to these agents hands-free. This is in contrast to a few years ago, when most of the traffic to search engines like Google Search, Yahoo!, or Microsoft Bing was in the form of text queries. The natural language understanding (NLU) models that underlie these search engines were tuned 1 https://www.statista.com/statistics/791575/us-smart-speaker-household-ownership/. Submission received: 26 April 2021; revised version received: 20 August 2021; accepted for publication: 4 December 2021. https://doi.org/10.1162/COLI a 00430 © 2022 Association for Computational Linguistics Published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license Computational Linguistics Volume 48, Number 1 Figure 1 The number of submitted papers in the speech processing (+ multimodal) track vs. total in ACL conference from 2018–2020. to handle textual queries typed by users. However, with the changing nature of query stream from text to speech, these NLU models also need to adapt in order to better understand the user. This is an opportune moment to bring our attention to the current state of automatic speech recognition (ASR) and NLU, and the interface between them. Traditionally, the ASR and NLU research communities have operated independently, with little cross- pollination. Although there is a long history of efforts to get ASR and NLU researchers to collaborate, for example, through conferences like HLT and DARPA programs (Liu et al. 2006; Ostendorf et al. 2008), the two communities are diverging again. This is reflected in their disjoint set of conference publication venues: ICASSP, ASRU/SLT, and Interspeech are the major conferences for speech processing, whereas ACL, NAACL, and EMNLP are the major venues for NLU. Figure 1 shows the total number of submitted papers to the …

Content truncated to 500 words