Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3905–3912 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3905 Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus Eleni Metheniti* †, G¨unter Neumann ‡ *CLLE-CNRS, †IRIT-CNRS , ‡DFKI * †Toulouse, France ‡Saarbr¨ucken, Germany eleni.metheniti@{*univ-tlse2.fr, †irit.fr}, ‡neumann@dfki.de Abstract Multilingual, inflectional corpora are a scarce resource in the NLP community, especially corpora with annotated morpheme boundaries. We are evaluating a generated, multilingual inflectional corpus with morpheme boundaries, generated from the English Wiktionary (Metheniti and Neumann, 2018), against the largest, multilingual, high-quality inflectional corpus of the UniMorph project (Kirov et al., 2018). We confirm that the generated Wikinflection corpus is not of such quality as UniMorph, but we were able to extract a significant amount of words from the intersection of the two corpora. Our Wikinflection corpus benefits from the morpheme segmentations of Wiktionary/Wikinflection and from the manually-evaluated morphological feature tags of the UniMorph project, and has 216K lemmas and 5.4M word forms, in a total of 68 languages. Keywords: inflection, morphological inflection, inflectional corpus, inflection annotation 1. Introduction Inflection is the linguistic process in which a word acquires morphological features, which allow it to create syntactic dependencies with its context or express an additional nu- ance without changing the word’s core meaning, e.g. num- ber, time. To create inflection on a word-surface level, different languages make different choices on what trans- formations will occur to the stem of the word, what mor- phemes will be added (at the end, start, middle, and how many of them). Knowledge on the way a language cre- ates inflections is crucial, in order to be able to identify and create different forms of the base word, the lemma, (re-inflection) and be able to reverse the process (lemma- tization). In this paper, we are describing our method of creating a multilingual inflectional corpus, using two available in- flectional corpora; the corpus from the UniMorph project (Kirov et al., 2018), and the corpus we generated from the paper and code of the Wikinflection project (Methen- iti and Neumann, 2018). The UniMorph corpus is the largest to date inflectional corpus, including 108 languages and ∼10.7 word forms, and has been extensively evalu- ated and enriched with other resources. However, it only includes entire word forms, without information on a sub- word level; meanwhile, Wikinflection generates a corpus of 6.4M words in ∼140 languages, and offers the morpheme segmentations of word forms as they exist in the English Wiktionary templates. We test the robustness and quality of the generated Wikinflection corpus by generating an itera- tion of the Wikinflection corpus and running our old evalua- tion script. In addition, we also evaluate Wikinflection with the UniMorph corpus. Results show that Wikinflection is of lower quality than UniMorph, due to the lack of manual evaluation, with the most serious problem being the inabil- ity to capture all grammatical tags for some word forms. From our findings, we were also able to create a …

Content truncated to 500 words