When Collaborative Treebank Curation Meets Graph Grammars

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5291–5300 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5291 When Collaborative Treebank Curation Meets Graph Grammars Arborator With a Grew Back-End Ga¨el Guibon, Marine Courtin, Kim Gerdes, Bruno Guillaume LLF (CNRS) – Universit´e Paris Diderot & Almanach (Inria), Almanach (Inria) & LPP (CNRS) – Sorbonne Nouvelle, Almanach (Inria) & LPP (CNRS) – Sorbonne Nouvelle, Loria (Inria) Paris & Nancy, France gael.guibon@inria.fr, marine.courtin@sorbonne-nouvelle.fr, kim@gerdes.fr, bruno.guillaume@inria.fr Abstract In this paper we present Arborator-Grew, a collaborative annotation tool for treebank development. Arborator-Grew combines the fea- tures of two preexisting tools: Arborator and Grew. Arborator is a widely used collaborative graphical online dependency treebank annotation tool. Grew is a tool for graph querying and rewriting specialized in structures needed in NLP, i.e. syntactic and semantic dependency trees and graphs. Grew also has an online version, Grew-match, where all Universal Dependencies treebanks in their clas- sical, deep and surface-syntactic flavors can be queried. Arborator-Grew is a complete redevelopment and modernization of Arborator, replacing its own internal database storage by a new Grew API, which adds a powerful query tool to Arborator’s existing treebank cre- ation and correction features. This includes complex access control for parallel expert and crowd-sourced annotation, tree comparison visualization, and various exercise modes for teaching and training of annotators. Arborator-Grew opens up new paths of collectively creating, updating, maintaining, and curating syntactic treebanks and semantic graph banks. Keywords: dependency treebanks, annotation tools, crowd-sourcing, class-sourcing, error-mining, graph banks 1. Introduction Dependency treebanks have become the standard resource for training syntactic parsers, and substantial efforts have been undertaken to develop large scale and multi-lingual treebanks. The flagship project is certainly Universal Dependencies (UD) (McDonald et al., 2013), which has served as the input to numerous parsers, text generators, and morphological taggers around various shared tasks (Ze- man et al., 2018; Mille et al., 2019; McCarthy et al., 2019). The impressive project with more than a hundred treebanks in the same annotation scheme for 90+ languages, com- bined with great online viewers and query tools have given increased visibility to the project also inside the syntax and typology communities (Croft et al., 2017; Gerdes et al., 2019b). Yet, for UD as well as for other treebank creation projects, many of the treebanks contain substantial errors and incon- sistencies, which can be attributed to three main causes: 1. Many of the UD treebanks are converted from other formats that do not contain all the information needed for a transfer into UD, or the converters are incom- plete. 2. Some descriptions in the UD guidelines are under- specified and leave room for different analyses of the same construction, inside a language, a language group, or generally among languages, cf. the con- stantly active UD discussion group on GitHub and also Gerdes and Kahane (2016). Moreover, some UD rules are not well-adapted for specific languages – e.g. the discussion of the direction of coordination in Japanese (Kanayama et al., …

Content truncated to 500 words