A multilingual collection of CoNLL-U-compatible morphological lexicons

A multilingual collection of CoNLL-U-compatible morphological lexicons Benoˆıt Sagot Inria 2 rue Simone Iff, CS 42112, 75589 Paris Cedex 12, France benoit.sagot@inria.fr Abstract We introduce UDLexicons, a multilingual collection of morphological lexicons that follow the guidelines and format of the Universal Dependencies initiative. We describe the three approaches we use to create 53 morphological lexicons covering 38 languages, based on existing resources. These lexicons, which are freely available, have already proven useful for improving part-of-speech tagging accuracy in state-of-the-art architectures. Keywords: Morphological Lexicons, Universal Dependencies, Freely Available Language Resources 1. Introduction Morphological information belongs to the most fundamen- tal types of linguistic knowledge. It is often either encoded into morphological analysers or gathered in the form of morphological lexicons. Such lexicons, which constitute the focus of this paper, are collections of lexical entries that typically associate a wordform with a part-of-speech (or morphosyntactic category), morphological features (such as gender, tense, etc.) and a lemma. Beyond direct lexicon lookup, used in virtually all types of natural language pro- cessing applications and computational linguistic studies, morphological lexicons have been shown to significantly improve tasks such as part-of-speech tagging and parsing. There is currently no universally accepted way to en- code morphological lexical information. Past multilin- gual projects such as MULTEXT/MULTEXT-East (Ide and V´eronis, 1994; Erjavec, 2010) have resulted in the publica- tion of morphological lexicons for a number of languages based on the same set of categories and morphosyntactic features, but they are still limited in scope. Yet another type of language resource embeds morpholog- ical lexical information, namely treebanks. This type of resource has recently seen the emergence of a de facto trans-lingual standard and the publication of an increas- ing number of treebanks for numerous languages follow- ing a universal set of guidelines, encoded in the CoNLL-U format and gathered under the name Universal Dependen- cies (hereafter UD).1 This treebank collection (Nivre et al., 2016; Nivre et al., 2017) follows several previous initia- tives, such as the proposal of a universal part-of-speech tagset (Petrov et al., 2012) and the multilingual datasets re- leased in the context of several shared tasks and projects (Buchholz and Marsi, 2006; Nivre et al., 2007; Zeman et al., 2012; Seddah et al., 2013). The UD initiative has therefore allowed a simpler, unified access to treebank resources, giving a new impetus to re- search in topics such as multilingual and cross-lingual to- kenisation, part-of-speech tagging, dependency parsing and quantitative linguistics. It is therefore important for mor- phological lexicons, another major source of linguistic in- formation for such tasks, to also be available for many lan- 1http://www.universaldependencies.org guages following a universal set of guidelines. The obvious choice would be to make use of the UD guidelines them- selves. We have therefore developed a multilingual collection of morphological lexicons that follow the UD guidelines re- garding part-of-speech and morphological features. We used three main sources of lexical information: • In the context of the CoNLL 2017 UD morphologi- cal and syntactic analysis shared task (Zeman et al., 2017) …

Content truncated to 500 words