Evaluation of Dictionary Creating Methods for Finno-Ugric Minority Languages

Evaluation of Dictionary Creating Methods for Finno-Ugric Minority Languages Zsanett Ferenczi, Iv´an Mittelholcz, Eszter Simon, Tam´as V´aradi Research Institute for Linguistics, Hungarian Academy of Sciences, H-1068 Budapest, Bencz´ur u. 33. {ferenczi.zsanett, mittelholcz.ivan, simon.eszter, varadi.tamas}@nytud.mta.hu Abstract In this paper, we present the evaluation of several bilingual dictionary building methods applied to {Komi-Permyak, Komi-Zyrian, Hill Mari, Meadow Mari, Northern Saami, Udmurt}–{English, Finnish, Hungarian, Russian} language pairs. Since these Finno-Ugric minority languages are under-resourced and standard dictionary building methods require a large amount of pre-processed data, we had to find alternative methods. In a thorough evaluation, we compare the results for each method, which proved our expectations that the precision of standard lexicon building methods is quite low for under-resourced languages. However, utilizing Wikipedia title pairs extracted via inter-language links and Wiktionary-based methods provided useful results. The newly created word pairs enriched with several linguistic information are to be deployed on the web in the framework of Wiktionary. With our dictionaries, the number of Wiktionary entries in the above mentioned Finno-Ugric minority languages can be multiplied. Keywords: bilingual dictionaries, evaluation, under-resourced languages, dictionary building methods 1. Introduction The research presented in this paper is part of a project whose general objective is to provide linguistically based support for several small Finno-Ugric (FU) digital commu- nities to generate online content and help revitalize the dig- ital functions of some FU minority languages. The prac- tical objective of the project is to create bilingual dictio- naries for six small FU languages (Komi-Permyak, Komi- Zyrian, Meadow Mari, Hill Mari, Northern Saami, and Ud- murt) paired with four major languages that are important for these small communities (English, Finnish, Hungarian, Russian) as well as to deploy the enriched lexical material on the web in the framework of the collaborative dictionary project Wiktionary. The status of any particular language of the world is usually described using the Expanded Graded Intergen- erational Disruption Scale (EGIDS) (Lewis and Simons, 2010), which gives an estimation of the overall develop- ment versus endangerment of the language. In this scale, the highest level is 0, where languages are world-wide used koin´es, while languages on level 10 are already extinct. Northern Saami is on the highest level among the above mentioned FU languages: its level is 2 (provincial), thus it is used in education, work, mass media, and government within some officially bilingual region of Norway, Sweden, and Finland. In the case of the Meadow Mari language, the EGIDS level is 4 (educational), which means that it is in vigorous use, with standardization and literature being sus- tained through a widespread system of institutionally sup- ported education. The EGIDS level of the other FU lan- guages (Komi-Permyak, Komi-Zyrian, Hill Mari, Udmurt) is 5, i.e. they are developing, which means that there is lit- erature which is available in a standardized form, though it is not yet widespread or sustainable. The above mentioned FU languages are not endangered but under-resourced, hence we could not collect enough data for building parallel and comparable corpora, on which …

Content truncated to 500 words