Universal Dependencies for Learner English

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 737–746, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Universal Dependencies for Learner English Yevgeni Berzak CSAIL MIT berzak@mit.edu Jessica Kenney EECS & Linguistics MIT jessk@mit.edu Carolyn Spadine Linguistics MIT cspadine@mit.edu Jing Xian Wang EECS MIT jxwang@mit.edu Lucia Lam MECHE MIT lucci@mit.edu Keiko Sophie Mori Linguistics MIT ksmori@mit.edu Sebastian Garza Linguistics MIT sjgarza@mit.edu Boris Katz CSAIL MIT boris@mit.edu Abstract We introduce the Treebank of Learner En- glish (TLE), the first publicly available syntactic treebank for English as a Sec- ond Language (ESL). The TLE provides manually annotated POS tags and Univer- sal Dependency (UD) trees for 5,124 sen- tences from the Cambridge First Certifi- cate in English (FCE) corpus. The UD annotations are tied to a pre-existing er- ror annotation of the FCE, whereby full syntactic analyses are provided for both the original and error corrected versions of each sentence. Further on, we delineate ESL annotation guidelines that allow for consistent syntactic treatment of ungram- matical English. Finally, we benchmark POS tagging and dependency parsing per- formance on the TLE dataset and measure the effect of grammatical errors on parsing accuracy. We envision the treebank to sup- port a wide range of linguistic and compu- tational research on second language ac- quisition as well as automatic processing of ungrammatical language1. 1 Introduction The majority of the English text available world- wide is generated by non-native speakers (Crys- tal, 2003). Such texts introduce a variety of chal- lenges, most notably grammatical errors, and are of paramount importance for the scientific study of language acquisition as well as for NLP. De- spite the ubiquity of non-native English, there is 1The treebank is available at universaldependencies.org. The annotation manual used in this project and a graphical query engine are available at esltreebank.org. currently no publicly available syntactic treebank for English as a Second Language (ESL). To address this shortcoming, we present the Treebank of Learner English (TLE), a first of its kind resource for non-native English, contain- ing 5,124 sentences manually annotated with POS tags and dependency trees. The TLE sentences are drawn from the FCE dataset (Yannakoudakis et al., 2011), and authored by English learners from 10 different native language backgrounds. The tree- bank uses the Universal Dependencies (UD) for- malism (De Marneffe et al., 2014; Nivre et al., 2016), which provides a unified annotation frame- work across different languages and is geared to- wards multilingual NLP (McDonald et al., 2013). This characteristic allows our treebank to sup- port computational analysis of ESL using not only English based but also multilingual approaches which seek to relate ESL phenomena to native lan- guage syntax. While the annotation inventory and guidelines are defined by the English UD formalism, we build on previous work in learner language anal- ysis (Dıaz-Negrillo et al., 2010; Dickinson and Ragheb, 2013) to formulate an additional set of annotation conventions aiming at a uniform treat- ment of ungrammatical learner language. Our annotation scheme uses a …

Content truncated to 500 words