A model of syntactic disambiguation based on lexicalized grammars

A model of syntactic disambiguation based on lexicalized grammars Yusuke Miyao Department of Computer Science, University of Tokyo yusuke@is.s.u-tokyo.ac.jp Jun’ichi Tsujii Department of Computer Science, University of Tokyo CREST, JST (Japan Science and Technology Corporation) tsujii@is.s.u-tokyo.ac.jp Abstract This paper presents a new approach to syntac- tic disambiguation based on lexicalized gram- mars. While existing disambiguation mod- els decompose the probability of parsing re- sults into that of primitive dependencies of two words, our model selects the most probable parsing result from a set of candidates allowed by a lexicalized grammar. Since parsing re- sults given by the lexicalized grammar cannot be decomposed into independent sub-events, we apply a maximum entropy model for fea- ture forests, which allows probabilistic model- ing without the independence assumption. Our approach provides a general method of produc- ing a consistent probabilistic model of parsing results given by lexicalized grammars. 1 Introduction Recent studies on the automatic extraction of lexicalized grammars (Xia, 1999; Chen and Vijay-Shanker, 2000; Hockenmaier and Steedman, 2002a) allow the modeling of syntactic disambiguation based on linguistically moti- vated grammar theories including LTAG (Chiang, 2000) and CCG (Clark et al., 2002; Hockenmaier and Steed- man, 2002b). However, existing models of disambigua- tion with lexicalized grammars are a mere extension of lexicalized probabilistic context-free grammars (LPCFG) (Collins, 1996; Collins, 1997; Charniak, 1997), which are based on the decomposition of parsing results into the syntactic/semantic dependencies of two words in a sen- tence under the assumption of independence of the de- pendencies. While LPCFG models have proved that the incorporation of lexical associations (i.e., dependencies of words) significantly improves the accuracy of parsing, this idea has been naively inherited in the recent studies on disambiguation models of lexicalized grammars. However, the disambiguation models of lexicalized grammars should be totally different from that of LPCFG, because the grammars define the relation of syntax and semantics, and can restrict the possible structure of pars- ing results. Parsing results cannot simply be decomposed into primitive dependencies, because the complete struc- ture is determined by solving the syntactic constraints of a complete sentence. For example, when we apply a unification-based grammar, LPCFG-like modeling re- sults in an inconsistent probability model because the model assigns probabilities to parsing results not allowed by the grammar (Abney, 1997). We have only two ways of adhering to LPCFG models: preserve the consistency of probability models by abandoning improvements to the lexicalized grammars using complex constraints (Chi- ang, 2000), or ignore the inconsistency in probability models (Clark et al., 2002). This paper provides a new model of syntactic disam- biguation in which lexicalized grammars can restrict the possible structures of parsing results. Our modeling aims at providing grounds for i) producing a consistent proba- bilistic model of lexicalized grammars, as well as ii) eval- uating the contributions of syntactic and semantic prefer- ences to syntactic disambiguation. The model is com- posed of the syntax and semantics probabilities, which represent syntactic and semantic preferences respectively. The syntax probability is responsible for determining the syntactic categories …

Content truncated to 500 words