JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3204–3210 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3204 JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages ˇZeljko Agi´c Department of Computer Science IT University of Copenhagen, Denmark zeag@itu.dk Ivan Vuli´c PolyAI Ltd. London, United Kingdom ivan@poly-ai.com Abstract Viable cross-lingual transfer critically depends on the availability of parallel texts. Shortage of such resources imposes a development and evaluation bottleneck in multilingual process- ing. We introduce JW300, a parallel corpus of over 300 languages with around 100 thousand parallel sentences per language pair on aver- age. In this paper, we present the resource and showcase its utility in experiments with cross- lingual word embedding induction and multi- source part-of-speech projection. 1 Introduction In natural language processing (NLP) the rule of thumb is that if we possess some parallel data for a low-resource target language, then we can yield feasible basic tools such as part-of-speech taggers for that language. Without such distant supervi- sion, this task and many others remain unattain- able, leaving the majority of languages in the world without basic language technology. Parallel data features a prominent role in building multilingual word representations (Ruder et al., 2017), annota- tion projection for parts-of-speech and syntactic dependencies (Das and Petrov, 2011; Tiedemann, 2014) and naturally machine translation. The shortage of parallel data in turn creates a bottleneck in cross-lingual processing: without par- allel sentences, we cannot yield usable models, nor can we robustly evaluate them, if even just approx- imately (cf. Agi´c et al. 2017). This absence has over the recent years materialized the proxy fallacy, whereby intended low-resource methods are tested by proxy, exclusively on resource-rich languages, because of the absence of test data or the lack of effort to produce it for approximate evaluation. We seek to alleviate these issues by a significant new addition to the limited pool of parallel texts for low-resource languages. Figure 1: Our dataset JW300 in comparison to other massive parallel text collections with respect to multi- lingual breadth and volume of parallel sentences. The y-axis depicts the mean number of parallel sentences per language pair. Contributions. A massive collection of parallel texts for over 300 diverse languages is our main contribution to facilitate multilingual NLP. The dataset is freely available for all non-commercial use.1 We also show how simple techniques over our data yield competitive results in building cross- lingual word embeddings and annotation projection for part-of-speech tagger induction. 2 Dataset JW300 spans across 343 languages, and comprises a total of 1,335,376 articles, with a bit over 109 million sentences, and 1.48 billion tokens. Sources and structure. The data is a complete crawl of all the publications from the website jw.org. A vast majority of texts come from the magazines Awake! and Watchtower. While the texts do stem from a religious society, they cover an immense range of topics. The multilingual ar- ticles are mainly translations from a source in En- glish. The dataset is …

Content truncated to 500 words