Contextual Rephrase Detection for Reducing Friction in Dialogue Systems

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1899–1905 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1899 Contextual Rephrase Detection for Reducing Friction in Dialogue Systems Zhuoyi Wang1∗ † Saurabh Gupta2∗ Jie Hao2∗ Xing Fan2 Dingcheng Li2 Alexander Hanbo Li3 Chenlei Guo2 1University of Texas at Dallas 2Amazon Alexa AI 3Amazon AWS AI zxw151030@utdallas.edu {gsaur,jieha,fanxing,lidingch,hanboli,guochenl}@amazon.com Abstract For voice assistants like Alexa, Google Assis- tant and Siri, correctly interpreting users’ in- tentions is of utmost importance. However, users sometimes experience friction with these assistants, caused by errors from different sys- tem components or user errors such as slips of the tongue. Users tend to rephrase their query until they get a satisfactory response. Rephrase detection is used to identify the rephrases and has long been treated as a task with pairwise input, which does not fully uti- lize the contextual information (e.g. users’ im- plicit feedback). To this end, we propose a con- textual rephrase detection model ContReph to automatically identify rephrases from multi- turn dialogues. We showcase how to leverage the dialogue context and user-agent interaction signals, including user’s implicit feedback and the time gap between different turns, which can help significantly outperform the pairwise rephrase detection models. 1 Introduction Large-scale conversational AI based dialogue sys- tems like Alexa, Siri, and Google Assistant, are getting more and more prevalent in real-world ap- plications to help users across the globe. Natu- ral Language Understanding (NLU) technology is an established component that produces seman- tic interpretations of a user request. Improving the accuracy of the NLU component is a key con- sideration for satisfactory end-to-end user experi- ence, especially when the NLU component misin- terprets the semantics due to ambiguity or errors that come from the previous component (e.g., Auto- matic Speech Recognition). For instance, the ASR system may incorrectly recognize “play jacking the ball” as “ play jack in the fall”. These errors accumulate and introduce friction in the dialogue ∗Equal contribution. †Work done when Zhuoyi Wang was interning at Amazon Alexa AI. Figure 1: Difference between the contextual rephrase detection and pairwise approach. Pair-wise rephrase detection model computes the similarity score for each pair that appears in the multi-turn dialogue, and selects the maximum score among them for the rephrase pre- diction. In this case, the pair-wise model without con- sidering context information incorrectly predicts “play tyler hero explicit by jack harlow” as the rephrase of user’s defective request “play tyler hero explicit” since it has highest similarity score. conversation. Fixing these frictions would help users to have a better experience, and engage more with the AI agents. Previous works (Yuan et al., 2021; Chen et al., 2020; Park et al., 2020) focus on friction reduction in the ASR and NLU components using Query Rewriting (QR) (Grbovic et al., 2015). These approaches reformulate the ASR transcription of user’s query, such that it conveys the same mean- ing/intent, to minimize user dissatisfaction. An important aspect of the QR approaches is to detect user rephrase of …

Content truncated to 500 words