Novel Feature Discovery for Task-Oriented Dialog Systems

Findings of the Association for Computational Linguistics: EACL 2023, pages 782–792 May 2-6, 2023 ©2023 Association for Computational Linguistics Novel Feature Discovery for Task-Oriented Dialog Systems Vinh Thinh Ho Amazon Alexa AI Berlin, Germany hovnh@amazon.com Mohamed Soliman Amazon Alexa AI Aachen, Germany mohsol@amazon.com Abdalghani Abujabal Amazon Alexa AI Aachen, Germany abujabaa@amazon.com Abstract A novel feature represents a cluster of semanti- cally equivalent novel user requests e.g., re- quests to play a song on a service or read- ing user’s messages. Detecting and supporting novel features is crucial towards wider adop- tion of dialog systems by end users. Intuitively, features are represented by a combination of intents, slot types and/or their values. For ex- ample, while playing a song is a feature rep- resented by a single intent (PlayMusic) only, playing a song on a service is another feature represented by the combination of PlayMusic intent and ServiceName slot type. Prior work on novelty detection limits the scope of features to those represented by novel single intents, leading to (1) giant clusters spanning several user-perceived fine-grained features belonging to the same intent, (2) incoherent interpreta- tion of clusters from users’ perspective (no di- rect connection to some user-perceived feature), and (3) missing those features spanning several intents. In this work, we introduce feature dis- covery as opposed to single intent discovery, which aims at discovering novel features span- ning a combination of intents and slots, and present a technique for discovering novel fea- tures from user utterances. Experiments on two datasets demonstrate the effectiveness of our approach and consistently show its ability to detect novel features. 1 Introduction Advances in Natural Language Understanding (NLU) have led to accelerated adoption of dialog systems such as Apple Siri and Amazon Alexa by end users. Standing at the core of a dialog system is an NLU model for parsing and understanding user utterances. Two of the key tasks of an NLU model are (1) Intent Classification, which classifies an utterance into a fixed set of intent labels, and (2) Slot Labeling, which classifies slot values into a predefined set of slot types (Weld et al., 2023). Determining the intent guides the dialog system to perform the proper actions as response to user’s utterance. For example, user utterances “play some music” and “play despacito” express the intent PlayMusic, while “how is the weather?” and “is it raining today” express the intent GetWeather. Intents can be further grouped into domains, for instance, PlayMusic and RateSong intents belong to Music domain. Detecting slots and their corre- sponding values within an utterance gives informa- tion about objects upon which the actions should be performed. For example, ‘despacito’ in “play despacito” is of type SongName. A feature represents a user experience with the dialog system, for example, playing a song on a service or reading user’s messages. Over time, users build up expectations about the fea- tures/experiences that the dialog system offers. Un- supported features cause friction and degrade user’s experience. In terms of NLU, a novel feature could …

Content truncated to 500 words