Re-Ranking Models Based-on Small Training Data for Spoken Language Understanding

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1076–1085, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Re-Ranking Models Based-on Small Training Data for Spoken Language Understanding Marco Dinarelli University of Trento Italy dinarelli@disi.unitn.it Alessandro Moschitti University of Trento Italy moschitti@disi.unitn.it Giuseppe Riccardi University of Trento Italy riccardi@disi.unitn.it Abstract The design of practical language applica- tions by means of statistical approaches requires annotated data, which is one of the most critical constraint. This is par- ticularly true for Spoken Dialog Systems since considerably domain-specific con- ceptual annotation is needed to obtain ac- curate Language Understanding models. Since data annotation is usually costly, methods to reduce the amount of data are needed. In this paper, we show that bet- ter feature representations serve the above purpose and that structure kernels pro- vide the needed improved representation. Given the relatively high computational cost of kernel methods, we apply them to just re-rank the list of hypotheses provided by a fast generative model. Experiments with Support Vector Machines and differ- ent kernels on two different dialog cor- pora show that our re-ranking models can achieve better results than state-of-the-art approaches when small data is available. 1 Introduction Spoken Dialog Systems carry out automatic speech recognition and shallow natural language understanding by heavily relying on statistical models. These in turn need annotated data de- scribing the application domain. Such annotation is far the most expensive part of the system de- sign. Therefore, methods reducing the amount of labeled data can speed up and lower the overall amount of work. Among others, Spoken Language Understand- ing (SLU) is an important component of the sys- tems above, which requires training data to trans- late a spoken sentence into its meaning repre- sentation based on semantic constituents. These are conceptual units instantiated by sequences of words. In the last decade two major approaches have been proposed to automatically map words in con- cepts: (i) generative models, whose parameters re- fer to the joint probability of concepts and con- stituents; and (ii) discriminative models, which learn a classification function based on conditional probabilities of concepts given words. A simple but effective generative model is the one based on Finite State Transducers. It performs SLU as a translation process from words to con- cepts using Finite State Transducers (FST). An ex- ample of discriminative model used for SLU is the one based on Support Vector Machines (SVMs) (Vapnik, 1995), as shown in (Raymond and Ric- cardi, 2007). In this approach, data is mapped into a vector space and SLU is performed as a clas- sification problem using Maximal Margin Clas- sifiers (Vapnik, 1995). A relatively more recent approach for SLU is based on Conditional Ran- dom Fields (CRF) (Lafferty et al., 2001). CRFs are undirected graphical and discriminative mod- els. They use conditional probabilities to account for many feature dependencies without the need of explicitly representing such dependencies. Generative models have the advantage to be more robust to overfitting on training data, while discriminative …

Content truncated to 500 words