Faster parsing and supertagging model estimation

Faster parsing and supertagging model estimation Jonathan K. Kummerfelda James R. Currana Jessika Roesnerb School of Information Technologiesa Department of Computer Scienceb University of Sydney University of Texas at Austin NSW 2006, Australia Austin, TX, USA {jkum0593,james}@it.usyd.edu.au jessi@mail.utexas.edu Abstract Parsers are often the bottleneck for data ac- quisition, processing text too slowly to be widely applied. One way to improve the efficiency of parsers is to construct more confident statistical models. More training data would enable the use of more sophis- ticated features and also provide more ev- idence for current features, but gold stan- dard annotated data is limited and expen- sive to produce. We demonstrate faster methods for training a supertagger using hundreds of millions of automatically annotated words, construct- ing statistical models that further constrain the number of derivations the parser must consider. By introducing new features and using an automatically annotated corpus we are able to double parsing speed on Wikipedia and the Wall Street Journal, and gain accuracy slightly when parsing Sec- tion 00 of the Wall Street Journal. 1 Introduction Many systems in NLP for tasks such as Question– Answering rely on large volumes of data. Parsers are a useful means of extracting extra informa- tion about text, providing the syntactic structure of sentences. However, when they are the bot- tleneck in the data acquisition phase of a system simple solutions are to use less data, or not use a parser at all. If we can improve the speed of parsers this will be unnecessary. For lexicalised grammars such as Combinatory Categorial Grammar (CCG) (Steedman, 2000) the step in which words are labelled with lexical cat- egories has great influence on parsing speed and accuracy. In these formalisms, the labels chosen constrain the set of possible derivations so much that the process of choosing them, supertagging, is described as ‘almost parsing’ (Joshi and Ban- galore, 1994). If the supertagger is more accurate it can further constrain the set of possible deriva- tions by supplying fewer categories, leaving the parser with less to do. One means of improving the supertagger’s sta- tistical model of language is to provide more evi- dence, in this case, more annotated text. However, creating a significant amount of extra gold stan- dard annotated text is not feasible. An alternative approach is ‘semi-supervised training’, in which a small set of annotated data and a much larger set of unannotated data is used. Training a sys- tem directly on its own output, ‘self-training’, is not normally effective (Clark et al., 2003), but re- cently McClosky et al. (2006) demonstrated that parser output can be made useful for retraining by the application of a reranker. To enable the use of more training data we have parallelised the C&C parser’s supertagger train- ing process and implemented perceptron–based algorithms for parameter estimation. In the pro- cess of this work we also modified the C&C parser’s use of a particular CCG rule, based on observations of its behaviour. Our unlabeled training data was part of the …

Content truncated to 500 words