Intent Detection with WikiHow

Intent Detection with WikiHow Li Zhang Qing Lyu University of Pennsylvania {zharry,lyuqing,ccb}@seas.upenn.edu Chris Callison-Burch Abstract Modern task-oriented dialog systems need to reliably understand users’ intents. Intent detec- tion is even more challenging when moving to new domains or new languages, since there is little annotated data. To address this challenge, we present a suite of pretrained intent detec- tion models which can predict a broad range of intended goals from many actions because they are trained on wikiHow, a comprehen- sive instructional website. Our models achieve state-of-the-art results on the Snips dataset, the Schema-Guided Dialogue dataset, and all 3 languages of the Facebook multilingual dialog datasets. Our models also demonstrate strong zero- and few-shot performance, reaching over 75% accuracy using only 100 training exam- ples in all datasets.1 1 Introduction Task-oriented dialog systems like Apple’s Siri, Amazon Alexa, and Google Assistant have become pervasive in smartphones and smart speakers. To support a wide range of functions, dialog systems must be able to map a user’s natural language in- struction onto the desired skill or API. Performing this mapping is called intent detection. Intent detection is usually formulated as a sen- tence classification task. Given an utterance (e.g. “wake me up at 8”), a system needs to predict its intent (e.g. “Set an Alarm”). Most modern ap- proaches use neural networks to jointly model in- tent detection and slot filling (Xu and Sarikaya, 2013; Liu and Lane, 2016; Goo et al., 2018; Zhang et al., 2019). In response to a rapidly grow- ing range of services, more attention has been given to zero-shot intent detection (Ferreira et al., 2015a,b; Yazdani and Henderson, 2015; Chen et al., 2016; Kumar et al., 2017; Gangadharaiah and 1The data and models are available at https:// github.com/zharry29/wikihow-intent. Narayanaswamy, 2019). While most existing re- search on intent detection proposed novel model architectures, few have attempted data augmenta- tion. One such work (Hu et al., 2009) showed that models can learn much knowledge that is important for intent detection from massive online resources such as Wikipedia. We propose a pretraining task based on wiki- How, a comprehensive instructional website with over 110,000 professionally edited articles. Their topics span from common sense such as “How to Download Music” to more niche tasks like “How to Crochet a Teddy Bear.” We observe that the header of each step in a wikiHow article describes an action and can be approximated as an utterance, while the title describes a goal and can be seen as an intent. For example, “find good gas prices” in the article “How to Save Money on Gas” is similar to the utterance “where can I find cheap gas?” with the intent “Save Money on Gas.” Hence, we intro- duce a dataset based on wikiHow, where a model predicts the goal of an action given some candi- dates. Although most of wikiHow’s domains are far beyond the scope of any present dialog system, models pretrained on our dataset would be robust to emerging services and scenarios. Also, as …

Content truncated to 500 words