A Distributional Analysis of a Lexicalized Statistical Parsing Model

A Distributional Analysis of a Lexicalized Statistical Parsing Model Daniel M. Bikel Department of Computer and Information Science University of Pennsylvania 3330 Walnut Street Philadelphia, PA 19104 dbikel@cis.upenn.edu Abstract This paper presents some of the first data visualiza- tions and analysis of distributions for a lexicalized statistical parsing model, in order to better under- stand their nature. In the course of this analysis, we have paid particular attention to parameters that include bilexical dependencies. The prevailing view has been that such statistics are very informative but suffer greatly from sparse data problems. By using a parser to constrain-parse its own output, and by hy- pothesizing and testing for distributional similarity with back-off distributions, we have evidence that finally explains that (a) bilexical statistics are actu- ally getting used quite often but that (b) the distri- butions are so similar to those that do not include head words as to be nearly indistinguishable inso- far as making parse decisions. Finally, our analysis has provided for the first time an effective way to do parameter selection for a generative lexicalized statistical parsing model. 1 Introduction Lexicalized statistical parsing models, such as those built by Black et al. (1992a), Magerman (1994), Collins (1999) and Charniak (2000), have been enormously successful, but they also have an enor- mous complexity. Their success has often been attributed to their sensitivity to individual lexical items, and it is precisely this incorporation of lexical items into features or parameter schemata that gives rise to their complexity. In order to help determine which features are helpful, the somewhat crude-but- effective method has been to compare a model’s overall parsing performance with and without a fea- ture. Often, it has seemed that features that are derived from linguistic principles result in higher- performing models (cf. (Collins, 1999)). While this may be true, it is clearly inappropriate to high- light ex post facto the linguistically-motivated fea- tures and rationalize their inclusion and state how effective they are. A rigorous analysis of features or parameters in relation to the entire model is called for. Accordingly, this work aims to provide a thor- ough analysis of the nature of the parameters in a Collins-style parsing model, with particular focus on the two parameter classes that generate lexical- ized modifying nonterminals, for these are where all a sentence’s words are generated except for the head word of the entire sentence; also, these two parameter classes have by far the most parameters and suffer the most from sparse data problems. In spite of using a Collins-style model as the basis for analysis, throughout this paper, we will attempt to present information that is widely applicable be- cause it pertains to properties of the widely-used Treebank (Marcus et al., 1993) and lexicalized pars- ing models in general. This work also sheds light on the much-discussed “bilexical dependencies” of statistical parsing mod- els. Beginning with the seminal work at IBM (Black et al., 1991; Black et al., 1992b; Black et al., 1992a), and continuing with such lexicalist …

Content truncated to 500 words