Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer

Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer Djamé Seddah1,2 Eric de la Clergerie2 Benoˆıt Sagot2 Héctor Martínez Alonso 4 Marie Candito3 1Université Paris-Sorbonne 2Inria 3Université Denis Diderot & CNRS 4 Thomson Reuters Lab djame.seddah@paris-sorbonne.fr, {eric.de_la_clergerie, benoˆıt.sagot}@inria.fr, hector.martinezalonso@thomsonreuters.com, marie.candito@linguist.univ-paris-diderot.fr Abstract We present an efficient and accurate method for transferring annotations between two different treebanks of the same language. This method led to the creation of a new instance of the French Treebank (Abeillé et al., 2003), which follows the Universal Dependency annotation scheme and which was proposed to the participants of the CoNLL 2017 Universal Dependency parsing shared task (Zeman et al., 2017). Strong results from an evaluation on our gold standard (94.75% of LAS, 99.40% UAS on the test set) demonstrate the quality of this new annotated data set and validate our approach. Keywords: Treebanking, Universal Dependencies, Syntax, Automatic Correction, Cross-annotation Transfer 1. Introduction After many decades of treebanking initiatives (Einarsson, 1976; Marcus et al., 1993), the interest in developing an- notated corpora no longer needs to be justified. Although a distinction can be noted between treebanks created for linguistic purposes and those only conceived in a natural language processing perspective, it tends to fade away in the face of the ever growing machine learning addiction to new sources of labeled data. In fact, not only can any an- notated corpus be used as a primary or secondary source of training data within more or less complex systems, but hand-crafted syntactic resources such as grammars and lex- icons can be used as sources of features to guide data driven systems (Øvrelid et al., 2009; Villemonte De La Clergerie, 2014a). The crucial point here lies in the interopability of such heterogenous sources of information. Before the rise of the Universal Dependency initiative (Nivre et al., 2017) and its eponymous scheme, henceforth UD, which resulted in the release of 81 treebanks on more than 50 language, the situation was at best complicated. Nevertheless, the pre- UD multitude of annotation schemes allowed many to use stacking methodologies for predicting syntactic annotations of a certain type and following specific guidelines (e.g. UD dependencies) with the help of other types of annotations that follow different schemes, sometimes even of a different topological nature (Farkas and Bohnet, 2012; Bj¨orkelund et al., 2013; Ambati et al., 2013; Ribeyre et al., 2015). In most cases, taking into account such heterogenous syntactic in- formation in the form of additional features does improve parsing accuracy. Unsurprisingly, the performance gain is generally outstand- ing whenever such features are extracted from gold anno- tations. When the goal is to produce new reference anno- tated data, such an performance gain results in fewer post- annotation corrections. In case of converting one treebank to another annotation scheme, such gold information is of course readily available and has the potential to consider- ably ease this process. In this paper, we describe such a conversion effort, for which we had to meet with another drastic constraint; in the context of the preparation of the …

Content truncated to 500 words