A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6054–6063 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6054 A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems Sunghyun Park∗, Han Li*, Ameen Patel, Sidharth Mudgal, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, Ruhi Sarikaya Amazon Alexa AI {sunghyu, lahl, paameen, sidmsk, sungjinl, youngbum, matsouka, rsarikay}@amazon.com Abstract Natural Language Understanding (NLU) is an established component within a conversational AI or digital assistant system, and it is respon- sible for producing semantic understanding of a user request. We propose a scalable and automatic approach for improving NLU in a large-scale conversational AI system by lever- aging implicit user feedback, with an insight that user interaction data and dialog context have rich information embedded from which user satisfaction and intention can be inferred. In particular, we propose a domain-agnostic framework for curating new supervision data for improving NLU from live production traf- fic. With an extensive set of experiments, we show the results of applying the framework and improving NLU for a large-scale produc- tion system across 10 domains. 1 Introduction For a conversational AI or digital assistant system (Kepuska and Bohouta, 2018), Natural Language Understanding (NLU) is an established component that produces semantic interpretations of a user request, which typically involves analysis in terms of domain, intent, and slot (El-Kahky et al., 2014). For instance, the request “Play a song by Taylor Swift" can be interpreted as falling within the scope of Music domain with Play Song intent and Taylor Swift identified for Artist slot. Without an accurate semantic understanding of the user request, a conversational AI system cannot fulfill the request with a satisfactory response or action. As one of the most upstream components in the runtime workflow (Sarikaya, 2017), NLU’s errors also have a wider blast radius that propagate to all subsequent downstream components, such as dialog management, routing logic to back-end applications, and language generation. ∗ Equal contribution. User Agent Domain: Shopping Intent: Remove Item from List Slot: Item Name = “lights” Take off the lights. Turn off the lights in the room. User Agent Domain: Smart Home Intent: Turn Off Device Slot: Device Name = “lights” Correction Rephrase Domain: Smart Home Intent: Turn Off Device Slot: Device Name = “lights” Conversation NLU Interpretations … Top-1 Identical Top-1 … Interruption … Turning off the lights now. I didn't find the item called “lights” on your shopping list. Did you want me to … … Figure 1: An example of implicit user feedback, specif- ically an indication of user dissatisfaction and user rephrase behavior, that can be used to create new super- vision data to correct NLU errors. The left side shows the dialog history and the right side shows the ranked NLU interpretations for each user request. A straight-forward way to improve NLU is through human annotations, but they are labor- intensive and expensive. Such annotations require at least multiple tiers of annotations (e.g., end …

Content truncated to 500 words