Graph-Based Semi-Supervised Conditional Random Fields For Spoken Language Understanding Using Unaligned Data

Graph-Based Semi-Supervised Conditional Random Fields For Spoken Language Understanding Using Unaligned Data Mohammad Aliannejadi Amirkabir University of Technology (Tehran Polytechnic) m.aliannejadi@aut.ac.ir Masoud Kiaeeha Sharif University of Technology kiaeeha@ce.sharif.edu Shahram Khadivi & Saeed Shiry Ghidary Amirkabir University of Technology (Tehran Polytechnic) {khadivi, shiry}@aut.ac.ir Abstract We experiment graph-based Semi- Supervised Learning (SSL) of Conditional Random Fields (CRF) for the application of Spoken Language (SLU) on unaligned data.Understanding The aligned labels for examples are obtained using IBM Model. We adapt a baseline semi- supervised CRF by defining new feature set and altering the label propagation al- gorithm. Our results demonstrate that our proposed approach significantly improves the performance of the supervised model by utilizing the knowledge gained from the graph. 1 Introduction The aim of Spoken Language Understanding (SLU) is to interpret the intention of the user’s ut- terance. More specifically, a SLU system attempts to find a mapping from user’s utterance in natu- ral language, to the limited set of concepts that is structured and meaningful for the computer. As an example, for the sample utterance: I want to return to Dallas on Thursday It’s corresponding output would be: GOAL : RETURN TOLOC.CITY = Dallas RETURN.DATE = Thursday. SLU can be widely used in many real world appli- cations; however, data processing costs may im- pede practicability of it. Thus, attempting to train a SLU model using less training data is a key issue. The first statistical SLU system was based on hidden Markov model and modeled using a fi- nite state semantic tagger employed in AT&T’s CHRONUS system (Pieraccini et al., 1992). Their semantic representation was flat-concept; but, later He and Young (2005) extended the represen- tation to a hierarchical structure and modeled the problem using a push-down automaton. There are other works which have dealt with SLU as a se- quential labeling problem. Raymond and Riccardi (2007) and Wang and Acero (2006) have fully an- notated the data and trained the model in discrim- inative frameworks such as CRF. CRF captures many complex dependencies and models the se- quential relations between the labels; therefore, it is a powerful framework for SLU. The Semi-Supervised Learning (SSL) approach has drawn a raft of interest among the machine learning community basically because of its prac- tical application. Manual tagging of data can take considerable effort and time; however, in the train- ing phase of SSL, a large amount of unlabeled data along with a small amount of labeled data is pro- vided. This makes it more practicable and cost ef- fective than providing a fully labeled set of train- ing data; thus, SSL is more favorable. Graph-based SSL, the most active area of re- search in SSL in the recent years, has shown to outperform other SSL methods (Chapelle et al., 2006). Graph-based SSL algorithms are gener- ally run in two steps: graph construction and label propagation. Graph construction is the most im- portant step in graph-based SSL; and, the funda- mental approach is to assign labeled and unlabeled examples to nodes of …

Content truncated to 500 words