UniMorph 3.0: Universal Morphology

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3922–3931 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3922 UniMorph 3.0: Universal Morphology Arya D. McCarthy, Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia, Kyle Gorman, Ekaterina Vylomova, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, Timofey Arkhangelskij, Natalya Krizhanovsky, Andrew Krizhanovsky, Elena Klyachko, Alexey Sorokin, John Mansfield, Valts Ernštreits, Yuval Pinter, Cassandra L. Jacobs, Ryan Cotterell, Mans Hulden, and David Yarowsky Johns Hopkins University, Google, Turin (Italy), City University of New York, University of Melbourne, University of Helsinki, Higher School of Economics, Lomonosov Moscow State University, Karelian Research Centre of RAS, Georgia Institute of Technology, University of Wisconsin, ETH Zurich, University of Colorado Boulder Abstract The Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological paradigms for hundreds of diverse world languages. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. We have implemented several improvements to the extraction pipeline which creates most of our data, so that it is both more complete and more correct. We have added 66 new languages, as well as new parts of speech for 12 languages. We have also amended the schema in several ways. Finally, we present three new community tools: two to validate data for resource creators, and one to make morphological data available from the command line. UniMorph is based at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University in Baltimore, Maryland. This paper details advances made to the schema, tooling, and dissemination of project resources since the UniMorph 2.0 release described at LREC 2018. Keywords: morphology, lexical database, multilinguality 1. Introduction A notable majority of the world’s languages exhibit inflec- tional morphology. Inflected variants of a word express such distinctions as number, grammatical gender, tense, and as- pect. Explicit modeling of morphology has demonstrable benefits for language modeling (Blevins and Zettlemoyer, 2019), speech recognition (Creutz et al., 2007), parsing (Ho- hensee and Bender, 2012; Seeker and Çetino˘glu, 2015), word embedding (Cotterell et al., 2016b), and keyword search in audio (Narasimhan et al., 2014; Adams et al., 2019). A goal of research, then, is to leverage morpho- logical information to arrive at cross-linguistic or transfer- able representations for word forms. This has value both to linguists wishing to annotate and understand linguistic phenomena and also to practitioners who seek to use this information for cross-lingual transfer (Yarowsky and Ngai, 2001) or translation (Dyer et al., 2008). The Universal Morphology (UniMorph) project, centered at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University, is the largest multilingual effort to improve how NLP systems handle complex morphology across the world’s languages. The collaborative project releases data sets in dozens of languages, annotated in a common tag set. A lemma (citation form) and bundle of morphological features (represented by the universal tags) are related to a …

Content truncated to 500 words