Parsing Internal Noun Phrase Structure with Collins’ Models

Parsing Internal Noun Phrase Structure with Collins’ Models David Vadas and James R. Curran School of Information Technologies University of Sydney NSW 2006, Australia � dvadas1, james � @it.usyd.edu.au Abstract Collins’ widely-used parsing models treat noun phrases (NPs) in a different manner to other constituents. We investigate these differences, using the recently released in- ternal NP bracketing data (Vadas and Cur- ran, 2007a). Altering the structure of the Treebank, as this data does, has a num- ber of consequences, as parsers built using Collins’ models assume that their training and test data will have structure similar to the Penn Treebank’s. Our results demonstrate that it is difficult for Collins’ models to adapt to this new NP structure, and that parsers using these models make mistakes as a result. This emphasises how important treebank struc- ture itself is, and the large amount of in- fluence it can have. 1 Introduction Collins’ parsing models (Collins, 1999) are widely used in natural language processing (NLP), as they are robust and accurate for recovering syntactic structure. These models can be trained on a wide variety of domains and languages, such as biological text, Chinese and Arabic. However, Collins’ models were originally built for the Penn Treebank (Marcus et al., 1993), and as such, are predisposed to parsing not only similar text, but also similar structure. This paper deals with the effects of assuming such a structure, after the Treebank has been altered. We focus on noun phrases (NPs) in particular, for two reasons. Firstly, there are a number of intricacies as part of Collins’ model in this area. Indeed, a Collins-style parser uses a different model for gen- erating NPs, compared to all other structures. Sec- ondly, we can make use of our previous work in annotating internal NP structure (Vadas and Curran, 2007a), which gives us a ready source of Penn Tree- bank data with an altered structure. Using this extended corpus, we make a number of alterations to the model itself, in an attempt to im- prove the parser’s performance. We also examine the errors made, focusing on the altered data in par- ticular, and suggest ways that performance can be improved in the future. Next, we look at what effect the NP bracketing structure has. The existing Penn Treebank uses a generally flat structure (especially in NPs), but when internal NP brackets are introduced this is no longer the case. We determine what is the best representa- tion to use for these new annotations, and also ex- amine why that is. Finally, we experiment with the task of identify- ing apposition within NPs, using manually annotated data as a gold standard. Although we find that this is a relatively easy task, the parser’s performance is very low. The reasons for why Collins’ models are unable to recover this structure are then explored. The work described here raises questions about how researchers create NLP models, which may be for any task and not just parsing. Implicitly assum- ing that data will always …

Content truncated to 500 words