Collecting Language Resources for the Latvian e-Government Machine Translation Platform

Collecting Language Resources for the Latvian e-Government Machine Translation Platform Roberts Rozis, Andrejs Vasiļjevs, Raivis Skadiņš Tilde Vienibas gatve 75A, Riga, Latvia E-mail: roberts.rozis@tilde.lv, andrejs@tilde.com, raivis.skadins@tilde.lv Abstract This paper describes corpora collection activity for building large machine translation systems for Latvian e-Government platform. We describe requirements for corpora, selection and assessment of data sources, collection of the public corpora and creation of new corpora from miscellaneous sources. Methodology, tools and assessment methods are also presented along with the results achieved, challenges faced and conclusions made. Several approaches to address the data scarceness are discussed. We summarize the volume of obtained corpora and provide quality metrics of MT systems trained on this data. Resulting MT systems for English-Latvian, Latvian-English and Latvian-Russian are integrated in the Latvian e-service portal and are freely available on website HUGO.LV. This paper can serve as a guidance for similar activities initiated in other countries, particularly in the context of European Language Resource Coordination action. Keywords: corpus, parallel texts, machine translation, web crawling, e-Government, public sector information 1. Project Background and Goal This paper describes the work on language resource collection for the Latvian e-Government Machine Translation (MT) Platform providing multilingual access to the portal of public online services. To foster broader use of public online services, Latvian Government has created a centralized portal Latvija.lv1 . The aim of the portal is to ensure quick and convenient access to the services provided by the Latvian State institutions and municipalities. The portal provides guidance on requirements (forms, documents, payments, terms etc.) and administrative procedures in order to receive public and municipal services, as well as direct access to those services that are offered online. Currently this portal hosts 108 e-services. Considering the nationalities in Latvia, it is important to make this content accessible not only in Latvian (the official state language), but also in Russian (1/3 of population are Russians, Byelorussians, Ukrainians) and English (for most foreigners). Considering dynamically changing content of online services and high costs of human translation, making use of Machine Translation (MT) was adopted as the only viable solution to provide multilingual access. Latvian Culture Information Systems Centre2 launched a project to build Machine Translation Platform for e- Government with specific MT systems tuned for state administration domain. The resulting platform is now branded as HUGO.LV. The goal in the long run is to provide MT services not only to portal Latvija.lv but to all Latvian State institutions and keep developing and integrating machine translation in all related public e-services. Language technology company Tilde 3 was chosen as a technology partner in this project, and was commissioned to perform language resource collection and processing, MT systems building and technology delivery. 1 http://www.latvija.lv/ 2 http://www.kis.gov.lv/ The goal in the project was to create a large corpora of language resources and build the best possible MT systems for 3 language pairs – English-Latvian, Latvian-English and Latvian-Russian. Two systems had to be provided for every translation pair – generic MT system and system tuned to the specific …

Content truncated to 500 words