Nonparametric Bayesian Models for Spoken Language Understanding

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2144–2152, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Nonparametric Bayesian Models for Spoken Language Understanding Kei Wakabayashi Tsukuba University, 1-2 Kasuga, Tsukuba, Ibaraki 305-8550, Japan kwakaba@slis.tsukuba.ac.jp Johane Takeuchi, Kotaro Funakoshi and Mikio Nakano Honda Research Institute Japan Co., Ltd. 8-1 Honcho, Wako, Saitama 351-0188, Japan {johane.takeuchi,funakoshi,nakano} @jp.honda-ri.com Abstract In this paper, we propose a new generative ap- proach for semantic slot filling task in spoken language understanding using a nonparamet- ric Bayesian formalism. Slot filling is typi- cally formulated as a sequential labeling prob- lem, which does not directly deal with the pos- terior distribution of possible slot values. We present a nonparametric Bayesian model in- volving the generation of arbitrary natural lan- guage phrases, which allows an explicit cal- culation of the distribution over an infinite set of slot values. We demonstrate that this ap- proach significantly improves slot estimation accuracy compared to the existing sequential labeling algorithm. 1 Introduction Spoken language understanding (SLU) refers to the challenge of recognizing a speaker’s intent from a natural language utterance, which is typically de- fined as a slot filling task. For example, in the ut- terance “Remind me to call John at 9am tomorrow”, the specified information {“time”: “9am tomor- row”} and {“subject”: “to call John”} should be extracted. The term slot refers to a variable such as the time or subject that is expected to be filled with a value provided through the user’s utterance. The slot filling task is typically formulated as a sequential labeling problem as shown in Figure 1. This labeling scheme naturally represents the recog- nition of arbitrary phrases that appear in the tran- scription of an utterance. Formally speaking, when we assume a given set of slots {s1, ..., sM} and de- note the corresponding slot values by {vs1, ..., vsM } where vsi ∈ Vsi, the domain of each slot value Vsi is an infinite set of word sequences. In this paper, we use the term arbitrary slot filling task to refer to this implicit problem statement, which inherently under- lies the sequential labeling formulation. In contrast, a different line of work has explored the case where Vsi is provided as a finite set of possi- ble values that can be handled by a backend system (Henderson, 2015). We refer to this type of task as a categorical slot filling task. In this case, the slot filling task is regarded as a classification problem that explicitly considers a value-based prediction, as shown in Figure 2. From this point of view, we can say that a distribution of slot values is actually con- centrated in a small set of typical phrases, even in the arbitrary slot filling task, because users basically know what kind of function is offered by the system. To reflect this observation, in this paper we ex- plore the value-based formulation approach for arbi- trary slot filling tasks. Unlike the sequential labeling formulation, which is basically …

Content truncated to 500 words