Phrase Structure Annotation and Parsing for Learner English

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1837–1847, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Phrase Structure Annotation and Parsing for Learner English Ryo Nagata Konan University 8-9-1 Okamoto, Higashinada Kobe, Hyogo 658-8501, Japan nagata-acl @hyogo-u.ac.jp. Keisuke Sakaguchi Johns Hopkins University 3400 North Charles Street Baltimore, MD, 21218, USA keisuke@cs.jhu.edu Abstract There has been almost no work on phrase structure annotation and parsing specially designed for learner English despite the fact that they are useful for representing the structural characteristics of learner En- glish. To address this problem, in this pa- per, we first propose a phrase structure an- notation scheme for learner English and annotate two different learner corpora us- ing it. Second, we show their usefulness, reporting on (a) inter-annotator agreement rate, (b) characteristic CFG rules in the corpora, and (c) parsing performance on them. In addition, we explore methods to improve phrase structure parsing for learner English (achieving an F-measure of 0.878). Finally, we release the full annotation guidelines, the annotated data, and the improved parser model for learner English to the public. 1 Introduction Learner corpora have been essential for NLP tasks related to learner language such as grammatical error correction. They are normally annotated with linguistic properties. In the beginning, at- tention was mainly focused on grammatical error annotation (Izumi et al., 2004; D´ıaz-Negrillo et al., 2009; Dale and Kilgarriff, 2011; Ng et al., 2013). Recently, it has been expanded to gram- matical annotation — first, Part-Of-Speech (POS) tagging (D´ıaz-Negrillo et al., 2009; Nagata et al., 2011) and then syntactic annotation (Kepser et al., 2004; Dickinson and Ragheb, 2009; Ragheb and Dickinson, 2012; Ragheb and Dickinson, 2013); syntactic annotation for learner corpora is now in- tensively studied. Among a variety of studies, a series of work by Ragheb and Dickinson (Dick- inson and Ragheb, 2009; Ragheb and Dickinson, 2012; Ragheb and Dickinson, 2013) is important in that they proposed a dependency annotation scheme, theoretically and empirically evaluated it, and revealed its theoretical problems, which gives a good starting point to those who wish to develop a new annotation scheme for learner corpora. Re- searchers including Foster (2004) and Ott and Ziai (2010) have even started using dependency- annotated learner corpora to develop dependency parsers for learner language. Although research on syntactic analysis for learner corpora has been making great progress as noted above, it is not yet complete. There are at least three limitations in the previous work: (i) as far as we are aware, there has been almost no work on phrase structure annotation specially designed for learner corpora; (ii) there are no publicly avail- able learner corpora annotated with syntax; (iii) phrase structure parsing performance on learner English has not yet been reported. The first limitation is that there exists no phrase structure annotation scheme specially designed for learner English. As related work, Foster (2007a; 2007b) and Foster and Andersen (2009) propose a method for creating a pseudo-learner corpus by ar- tificially generating errors …

Content truncated to 500 words