Stochastic HPSG

Stochastic HPSG Chris Brew Language Technology Group HCRC, University of Edinburgh 2 Buccleuch Place • Edinburgh EH8 9LW Scotland, UK email: Chris. Brew0edinburgh. ac. uk Abstract In this paper we provide a probabilis- tic interpretation for typed feature struc- tures very similar to those used by Pol- lard and Sag. We begin with a ver- sion of the interpretation which lacks a treatment of re-entrant feature struc- tures, then provide an extended interpre- tation which allows them. We sketch al- gorithms allowing the numerical param- eters of our probabilistic interpretations of HPSG to be estimated from corpora. 1 Introduction The purpose of our paper is to develop a princi- pled technique for attaching a probabilistic inter- pretation to feature structures. Our techniques apply to the feature structures described by Car- penter (Carpenter, 1992). Since these structures are the ones which are used in by Pollard and Sag (Pollard and Sag, 1994) their relevance to computational grammars is apparent. On the ba- sis of the usefulness of probabilistic context-free grammars (Charniak, 1993, ch. 5), it is plausible to assume that that the extension of probabilistic techniques to such structures will allow the ap- plication of known and new techniques of parse ranking and grammar induction to more interest- ing grammars than has hitherto been the case. The paper is structured as follows. We start by reviewing the training and use of probabilis- tic context-free grammars (PCFGs). We then de: velop a technique to allow analogous probabilistic annotations on type hierarchies. This gives us a clear account of the relationship between a large class of feature structures and their probabilities, but does not treat re-entrancy. We conclude by sketching a technique which does treat such struc- tures. While we know of previous work which as- sociates scores with feature structures (Kim, 1994) are not aware of any previous treatment which makes explicit the link to classical probability the- ory. We take a slightly unconventional perspective on feature structures, because it is easier to cast our theory within the more general framework of incremental description refinement (Mellish, 1988) than to exploit the usual metaphors of constraint-based grammar. In fact we can afford to remain entirely agnostic about the means by which the HPSG grammar associates signs with linguistic strings, because all that we need in or- der to train our stochastic procedures is a corpus of signs which are known to be valid descriptions of strings. 2 Probabilistic interpretation of PCFGs We review the standard probabilistic interpreta- tion of PCFGs 1 A PCFG is a four-tuple < W,N, N1,R > , where W is a Set of terminal symbols {wl,..., w~}, N is a set of non-terminal symbols {N1,...,N~}, N1 is the starting symbol and R is a set of rules of the form N ~ ~ (J, where (J is a string of terminals and non-terminals. Each rule has a probability P(N i --~ ~J) and the prob- abilities for all the rules that expand a given non- terminal must sum to …

Content truncated to 500 words