A Hybrid Generative/Discriminative Framework to Train a Semantic Parser from an Un-annotated Corpus

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 1113–1120 Manchester, August 2008 A Hybrid Generative/Discriminative Framework to Train a Semantic Parser from an Un-annotated Corpus Deyu Zhou and Yulan He Information Research Centre The University of Reading Reading, RG6 6BX, UK d.zhou@rdg.ac.uk, y.he@rdg.ac.uk Abstract We propose a hybrid genera- tive/discriminative framework for se- mantic parsing which combines the hidden vector state (HVS) model and the hidden Markov support vector machines (HM- SVMs). The HVS model is an extension of the basic discrete Markov model in which context is encoded as a stack-oriented state vector. The HM-SVMs combine the advantages of the hidden Markov models and the support vector machines. By employing a modified K-means clustering method, a small set of most representative sentences can be automatically selected from an un-annotated corpus. These sentences together with their abstract an- notations are used to train an HVS model which could be subsequently applied on the whole corpus to generate semantic parsing results. The most confident semantic parsing results are selected to generate a fully-annotated corpus which is used to train the HM-SVMs. The proposed framework has been tested on the DARPA Communicator Data. Experimental results show that an improvement over the base- line HVS parser has been observed using the hybrid framework. When compared with the HM-SVMs trained from the fully- annotated corpus, the hybrid framework gave a comparable performance with only a small set of lightly annotated sentences. c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. 1 Introduction Semantic parsing maps the natural language sen- tences to complete formal meaning representa- tions. Traditionally, research in the field of se- mantic parsing can be divided into two categories: rule-based approaches and statistical approaches. Based on hand-crafted semantic grammar rules, rule-based approaches fill slots in semantic frames using word pattern and semantic tokens (Dowding et al., 1994; Ward and Issar, 1994). Such rule- based approaches are typically domain-specific and often fragile. Statistical approaches are gen- erally based on stochastic models. They can be further categorized into three types: generative approaches, discriminative approaches and a hy- brid of the two. Generative approaches learn the joint probability model, P(W, C), of input sen- tence W and its semantic tag sequence C, com- pute P(C|W) using the Bayes rule, and then take the most probable tag sequence C. The hidden Morkov model (HMM), being a generative model, has been predominantly used in statistical seman- tic parsing. It models sequential dependencies by treating a semantic parse sequence as a Markov chain, which leads to an efficient dynamic pro- gramming formulation for inference and learning. The hidden vector state (HVS) model (He and Young, 2005) is a discrete HMM model in which each HMM state represents the state of a push- down automaton with a finite stack size. State transitions are factored into separate stack pop and push operations constrained to give a tractable search space. The result is a model which is complex enough …

Content truncated to 500 words