A Model of Zero-Shot Learning of Spoken Language Understanding

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 244–249, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. A Model of Zero-Shot Learning of Spoken Language Understanding Majid Yazdani Computer Science Department University of Geneva majid.yazdani@unige.ch James Henderson Xerox Research Center Europe james.henderson@xrce.xerox.com Abstract When building spoken dialogue systems for a new domain, a major bottleneck is developing a spoken language understand- ing (SLU) module that handles the new domain’s terminology and semantic con- cepts. We propose a statistical SLU model that generalises to both previously unseen input words and previously unseen out- put classes by leveraging unlabelled data. After mapping the utterance into a vector space, the model exploits the structure of the output labels by mapping each label to a hyperplane that separates utterances with and without that label. Both these mappings are initialised with unsupervised word embeddings, so they can be com- puted even for words or concepts which were not in the SLU training data. 1 Introduction Spoken Language Understanding (SLU) in dia- logue systems is the task of taking the utterance output by a speech recognizer and assigning it a semantic label that represents the dialogue actions of that utterance accompanied with their associ- ated attributes and values. For example, the utter- ance ”I would like Chinese food” is labelled with inform(food=Chinese), in which inform is the dia- logue action that provides the value of the attribute food that is Chinese. Dialogue systems often use hand-crafted gram- mars for SLU, such as Phoenix (Ward, 1994), which are expensive to develop, and expensive to extend or adapt to new attributes and values. Statistical SLU models are usually trained on the data obtained from a specific domain and loca- tion, using a structured output classifier that can be discriminative (Pradhan et al., 2004; Kate and Mooney, 2006; Henderson et al., 2012) or genera- tive (Schwartz et al., 1996; He and Young, 2005). Gathering and annotating SLU data is costly and time consuming and therefore SLU datasets are small compare to the number of possible labels. Because training sets for a new domain are small, or non-existent, learning is often an in- stance of Zero-shot or One-shot learning prob- lems (Palatucci et al., 2009; L. Fei-Fei; Fergus, 2006), in which zero or few examples of some output classes are available during the training. For example, in the restaurant reservation domain, not all possible combinations of foods and dia- logue actions may be included in the training set. The general idea to solve this type of problems is to map the input and class labels to a semantic space of usually lower dimension in which simi- lar classes are represented by closer points in the space (Palatucci et al., 2009; Weston et al., 2011; Weston et al., 2010). Usually unsupervised knowl- edge sources are used to form semantic codes of the labels that helps us to generalize to unseen la- bels. On the other hand, there are also different ways to express the …

Content truncated to 500 words