Correcting Dependency Annotation Errors

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 193–201, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Correcting Dependency Annotation Errors Markus Dickinson Indiana University Bloomington, IN, USA md7@indiana.edu Abstract Building on work detecting errors in de- pendency annotation, we set out to correct local dependency errors. To do this, we outline the properties of annotation errors that make the task challenging and their existence problematic for learning. For the task, we define a feature-based model that explicitly accounts for non-relations between words, and then use ambiguities from one model to constrain a second, more relaxed model. In this way, we are successfully able to correct many errors, in a way which is potentially applicable to dependency parsing more generally. 1 Introduction and Motivation Annotation error detection has been explored for part-of-speech (POS), syntactic constituency, se- mantic role, and syntactic dependency annotation (see Boyd et al., 2008, and references therein). Such work is extremely useful, given the harm- fulness of annotation errors for training, including the learning of noise (e.g., Hogan, 2007; Habash et al., 2007), and for evaluation (e.g., Padro and Marquez, 1998). But little work has been done to show the full impact of errors, or what types of cases are the most damaging, important since noise can sometimes be overcome (cf. Osborne, 2002). Likewise, it is not clear how to learn from consistently misannotated data; studies often only note the presence of errors or eliminate them from evaluation (e.g., Hogan, 2007), and a previous at- tempt at correction was limited to POS annotation (Dickinson, 2006). By moving from annotation error detection to error correction, we can more fully elucidate ways in which noise can be over- come and ways it cannot. We thus explore annotation error correction and its feasibility for dependency annotation, a form of annotation that provides argument relations among words and is useful for training and testing dependency parsers (e.g., Nivre, 2006; McDonald and Pereira, 2006). A recent innovation in depen- dency parsing, relevant here, is to use the predic- tions made by one model to refine another (Nivre and McDonald, 2008; Torres Martins et al., 2008). This general notion can be employed here, as dif- ferent models of the data have different predictions about whch parts are erroneous and can highlight the contributions of different features. Using dif- ferences that complement one another, we can be- gin to sort accurate from inaccurate patterns, by integrating models in such a way as to learn the true patterns and not the errors. Although we focus on dependency annotation, the methods are poten- tially applicable for different types of annotation, given that they are based on the similar data repre- sentations (see sections 2.1 and 3.2). In order to examine the effects of errors and to refine one model with another’s information, we need to isolate the problematic cases. The data representation must therefore be such that it clearly allows for the specific identification of er- …

Content truncated to 500 words