The Tembusu Treebank: An English Learner Treebank

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4817–4826 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4817 The Tembusu Treebank: An English Learner Treebank Luis Morgado da Costa ♠ , Francis Bond ♠ , Roger V P Winder ♡ ♠Asian Studies, Palack´y University Olomouc, ♡LCC, Nanyang Technological University lmorgado.dacosta@gmail.com Abstract This paper reports on the creation and development of the Tembusu Learner Treebank — an open treebank created from the NTU Corpus of Learner English, unique for incorporating mal-rules in the annotation of ungrammatical sentences. It describes the motivation and development of the treebank, as well as its exploitation to build a new parse-ranking model for the English Resource Grammar, designed to help improve the parse selection of ungrammatical sentences and diagnose these sentences through mal-rules. The corpus contains 25,000 sentences, of which 4,900 are treebanked. The paper concludes with an evaluation experiment that shows the usefulness of this new treebank in the tasks of grammatical error detection and diagnosis. Keywords: treebank, learner corpus, error detection, error diagnosis, parsing 1. Introduction Treebanks are valuable language resources, where in- formation pertaining to syntactic and/or semantic struc- ture is provided alongside a corpus. Treebanks have many usages, both in empirical linguistic research and in NLP. They have proved to be invaluable resources for tasks such as parsing, machine translation, infor- mation retrieval, and others. In general, most NLP tasks have something to gain from richer annotation schemas, and this is why tree- banks are generally seen as worthwhile endeavors, de- spite requiring a large investment of resources to create. Current attempts to address the profoundly linguistic problems of grammatical error detection and diagnosis rely on fairly shallow annotations. Mainstream shared- tasks generally bypass any notion of grammatical struc- ture when providing labeled training data. We believe this information is important for these tasks, and aim to fill this gap. The Tembusu Treebank is new kind of data resource, designed to aid the tasks of grammati- cal error detection and diagnosis with deep structural annotations of syntax and semantics, along with gram- matical error annotations in the form of mal-rules. 2. Related Work 2.1. The NTU Corpus of Learner English The Tembusu Treebank is based on the NTU Corpus of Learner English (Winder et al., 2017, NTUCLE). The NTUCLE is an open corpus of learner English, made up of assignments submitted by first year under- graduate engineering students from a major university in Singapore (NTU). It was hand-tagged by six pro- fessional English lecturers, and uses a new annotation schema largely based on pre-existing tagsets such as the ones used by the NUS Corpus of Learner English (Dahlmeier et al., 2013) and the Cambridge Learner Corpus (Nicholls, 2003). NTUCLE’s first release con- tained 180 tagged documents, containing 9,571 sen- tences out of which 2,751 were considered problem- atic. 2.2. The English Resource Grammar The Tembusu Treebank is built from parses generated by the English Resource Grammar (Flickinger, 2000; Copestake and Flickinger, 2000, …

Content truncated to 500 words