Semi-Supervised Semantic Tagging of Conversational Understanding using Markov Topic Regression

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 914–923, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Semi-Supervised Semantic Tagging of Conversational Understanding using Markov Topic Regression Asli Celikyilmaz Microsoft Mountain View, CA, USA asli@ieee.org Dilek Hakkani-Tur, Gokhan Tur Microsoft Research Mountain View, CA, USA dilek@ieee.org gokhan.tur@ieee.org Ruhi Sarikaya Microsoft Redmond, WA, USA rusarika@microsoft.com Abstract Finding concepts in natural language ut- terances is a challenging task, especially given the scarcity of labeled data for learn- ing semantic ambiguity. Furthermore, data mismatch issues, which arise when the expected test (target) data does not exactly match the training data, aggra- vate this scarcity problem. To deal with these issues, we describe an efficient semi- supervised learning (SSL) approach which has two components: (i) Markov Topic Regression is a new probabilistic model to cluster words into semantic tags (con- cepts). It can efficiently handle seman- tic ambiguity by extending standard topic models with two new features. First, it en- codes word n-gram features from labeled source and unlabeled target data. Sec- ond, by going beyond a bag-of-words ap- proach, it takes into account the inherent sequential nature of utterances to learn se- mantic classes based on context. (ii) Ret- rospective Learner is a new learning tech- nique that adapts to the unlabeled target data. Our new SSL approach improves semantic tagging performance by 3% ab- solute over the baseline models, and also compares favorably on semi-supervised syntactic tagging. 1 Introduction Semantic tagging is used in natural language un- derstanding (NLU) to recognize words of seman- tic importance in an utterance, such as entities. Typically, a semantic tagging model require large amount of domain specific data to achieve good performance (Tur and DeMori, 2011). This re- quires a tedious and time intensive data collection and labeling process. In the absence of large la- beled training data, the tagging model can behave poorly on test data (target domain). This is usually caused by data mismatch issues and lack of cover- age that arise when the target data does not match the training data. To deal with these issues, we present a new semi-supervised learning (SSL) approach, which mainly has two components. It initially starts with training supervised Conditional Random Fields (CRF) (Lafferty et al., 2001) on the source train- ing data which has been semantically tagged. Us- ing the trained model, it decodes unlabeled dataset from the target domain. With the data mismatch issues in mind, to correct errors that the supervised model make on the target data, the SSL model leverages the additional information by way of a new clustering method. Our first contribution is a new probabilistic topic model, Markov Topic Re- gression (MTR), which uses rich features to cap- ture the degree of association between words and semantic tags. First, it encodes the n-gram context features from the labeled source data and the unla- beled target data as prior information to learn se- mantic classes based on context. Thus, each latent semantic class corresponds to …

Content truncated to 500 words