Adapting the TANL tool suite to Universal Dependencies

Adapting the TANL tool suite to Universal Dependencies Giuseppe Attardi, Maria Simi Dipartimento di Informatica Largo B. Pontecorvo 3, 56127 Pisa, Italy E-mail: attardi@di.unipi.it, simi@di.unipi.it Abstract TANL is a suite of tools for text analytics based on the software architecture paradigm of data driven pipelines. The strategies for upgrading TANL to the use of Universal Dependencies range from a minimalistic approach consisting of introducing pre/post-processing steps into the native pipeline to revising the whole pipeline. We explore the issue in the context of the Italian Treebank, considering both the efforts involved, how to avoid losing linguistically relevant information and the loss of accuracy in the process. In particular we compare different strategies for parsing and discuss the implications of simplifying the pipeline when detailed part-of-speech and morphological annotations are not available, as it is the case for less resourceful languages. The experiments are relative to the Italian linguistic pipeline, but the use of different parsers in our evaluations and the avoidance of language specific tagging make the results general enough to be useful in helping the transition to UD for other languages. Keywords: Linguistic pipeline, dependency parsing, Universal Dependencies 1. Introduction TANL (Natural Language Text Analytics) is a suite of tools for text analytics based on the software architecture paradigm of data pipelines. TANL pipelines are data driven, i.e. each stage pulls data from the preceding stage and transforms them for use by the next stage. Since data is processed as soon as it becomes available, processing delay is minimized improving data throughput. The processing modules can be written in either C++, Python or PHP and can be combined using few lines of scripts to produce full NLP applications. TANL provides a set of modules, ranging from tokenization to POS tagging, from parsing to NE recognition. A TANL pipeline can be processed in parallel on a cluster of computers by means of a map/reduce streaming framework. The architecture of the TANL pipeline, its modules and some sample applications where presented at LREC 2010 (Attardi et al. 2010). Given that most of the tools are based on machine learning techniques, the pipeline can be used to process texts in several languages: English, Italian, Spanish. For Italian in particular we have been contributing to the development of training resources for the various tools: an Italian lexicon, a resource for training the POS tagger, and, along the years, several treebanks of increasing size for training dependency parsers: ISST-CoNLL (released for CoNLL-2007), MIDT (the Merged Italian Dependency Treebank), ISDT (the Italian version of the Stanford Dependencies). Our team joined since the beginning the Universal Dependencies (UD) project, a recent initiative to develop cross-linguistically consistent treebank annotations for several languages, that aims to facilitate multilingual parser development and cross-language parsing (Nivre, 2015). As part of this effort, we released in May 2015 the first version of the Italian treebank adhering to the UD guidelines: UD-it 1.1. In this paper we analyze how to adapt the TANL linguistic pipeline in order to fully support the Universal …

Content truncated to 500 words