ZAP: An Open-Source Multilingual Annotation Projection Framework

ZAP: An Open-Source Multilingual Annotation Projection Framework Alan Akbik and Roland Vollgraf Zalando Research M¨uhlenstraße 25, 10243 Berlin {firstname.lastname}@zalando.de Abstract Previous work leveraged annotation projection as a convenient method to automatically generate linguistic resources such as treebanks or propbanks for new languages. This approach automatically transfers linguistic annotation from a resource-rich source language (SL) to translations in a target language (TL). However, to the best of our knowledge, no publicly available framework for this approach currently exists, limiting researchers’ ability to reproduce and compare experiments. In this paper, we present ZAP, the first open-source framework for annotation projection in parallel corpora. Our framework is Java-based and includes methods for preprocessing corpora, computing word-alignments between sentence pairs, transferring different layers of linguistic annotation, and visualization. The framework was designed for ease-of-use with lightweight APIs. We give an overview of ZAP and illustrate its usage. The framework is available on github at https://github.com/zalandoresearch/zap Keywords: Annotation projection, corpora, multilingual data 1. Introduction Linguistically annotated corpora, such as treebanks (Mar- cus et al., 1993) or propbanks (Palmer et al., 2005), are a crucial driver of progress in natural language processing research. As a cost-effective alternative to manual anno- tation, previous work explored the use of annotation pro- jection (Yarowsky et al., 2001) in parallel corpora to au- tomatically create linguistically annotated corpora for new languages. This approach requires only a parallel corpus consisting of sentences in a resource-rich source language (SL) and their translations in a target language (TL), as well as existing parsers for the SL. It leverages the hypothesis that translated sentences share a degree of syntactic and, in particular, semantic parallelism (Pad´o and Lapata, 2009), thus allowing us to automatically transfer linguistic annota- tions from SL to TL. Refer to Figure 1 for an illustration of this approach, and Section 2 for more details. Annotation projection resources. Previous works lever- aged annotation projection for various layers of syntactic and semantic annotation (Yarowsky et al., 2001; Hwa et al., 2005; Van der Plas et al., 2011; Akbik et al., 2015). How- ever, to the best of our knowledge, no publicly available framework exists for this approach. This limits the ability of researchers to quickly set up experiments, discuss and compare approaches against previous work, and analyze the viability of annotation projection for a specific type of lin- guistic annotation or language pair. ZAP framework. For these reasons, we present ZAP, the first open-source framework for annotation projection. The framework contains a number of components callable through lightweight APIs: 1. A set of preprocessing tools that wrap popular NLP libraries to facilitate parsing of sentences in all sup- ported languages. 2. An alignment module for word-aligning translated sentence pairs in a parallel corpus. 3. An annotation projection module for transferring dif- ferent types of linguistic annotation between word- aligned sentences. 4. A visualization module for rendering word-aligned sentence pairs and projected annotations, enabling re- searchers to execute and visually inspect annotation projection for specific sentence pairs. In this extended abstract, we first give …

Content truncated to 500 words