Correcting a POS-Tagged Corpus Using Three Complementary Methods

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 523–531, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Correcting a PoS-tagged corpus using three complementary methods Hrafn Loftsson School of Computer Science Reykjavik University Reykjavik, Iceland hrafn@ru.is Abstract The quality of the part-of-speech (PoS) annotation in a corpus is crucial for the development of PoS taggers. In this pa- per, we experiment with three complemen- tary methods for automatically detecting errors in the PoS annotation for the Ice- landic Frequency Dictionary corpus. The first two methods are language indepen- dent and we argue that the third method can be adapted to other morphologically complex languages. Once possible errors have been detected, we examine each er- ror candidate and hand-correct the cor- responding PoS tag if necessary. Over- all, based on the three methods, we hand- correct the PoS tagging of 1,334 tokens (0.23% of the tokens) in the corpus. Fur- thermore, we re-evaluate existing state-of- the-art PoS taggers on Icelandic text using the corrected corpus. 1 Introduction Part-of-speech (PoS) tagged corpora are valuable resources for developing PoS taggers, i.e. pro- grams which automatically tag each word in run- ning text with morphosyntactic information. Cor- pora in various languages, such as the English Penn Treebank corpus (Marcus et al., 1993), the Swedish Stockholm-Umeå corpus (Ejerhed et al., 1992), and the Icelandic Frequency Dictionary (IFD) corpus (Pind et al., 1991), have been used to train (in the case of data-driven methods) and develop (in the case of linguistic rule-based meth- ods) different taggers, and to evaluate their accu- racy, e.g. (van Halteren et al., 2001; Megyesi, 2001; Loftsson, 2006). Consequently, the quality of the PoS annotation in a corpus (the gold stan- dard annotation) is crucial. Many corpora are annotated semi- automatically. First, a PoS tagger is run on the corpus text, and, then, the text is hand-corrected by humans. Despite human post-editing, (large) tagged corpora are almost certain to contain errors, because humans make mistakes. Thus, it is important to apply known methods and/or develop new methods for automatically detecting tagging errors in corpora. Once an error has been detected it can be corrected by humans or an automatic method. In this paper, we experiment with three differ- ent methods of PoS error detection using the IFD corpus. First, we use the variation n-gram method proposed by Dickinson and Meurers (2003). Sec- ondly, we run five different taggers on the cor- pus and examine those cases where all the tag- gers agree on a tag, but, at the same time, disagree with the gold standard annotation. Lastly, we use IceParser (Loftsson and Rögnvaldsson, 2007) to generate shallow parses of sentences in the corpus and then develop various patterns, based on fea- ture agreement, for finding candidates for annota- tion errors. Once error candidates have been detected by each method, we examine the candidates man- ually and correct the errors. Overall, based on these methods, we hand-correct the PoS tagging of 1,334 tokens …

Content truncated to 500 words