Providing a Catalogue of Language Resources for Commercial Users

Providing a Catalogue of Language Resources for Commercial Users Bente Maegaard1, Lina Henriksen2, Andrew Joscelyne3, Vesna Lusicky4, Margaretha Mazura5, Sussi Olsen6, Claus Povlsen7, Philippe Wacker8 1,2,6,7CLARIN-DK, University of Copenhagen, 3,8LT Innovate, Brussels, 4University of Vienna, 5EMF E-mail: bmaegaard@hum.ku.dk, linah@hum.ku.dk, aj@lt-innovate.eu, vesna.lusicky@univie.ac.at, mm@emfs.eu, saolsen@hum.ku.dk, cpovlsen@hum.ku.dk, phw@lt-innovate.eu Abstract Language resources (LR) are indispensable for the development of tools for machine translation (MT) or various kinds of computer-assisted translation (CAT). In particular language corpora, both parallel and monolingual are considered most important for instance for MT, not only SMT but also hybrid MT. The Language Technology Observatory will provide easy access to information about LRs deemed to be useful for MT and other translation tools through its LR Catalogue. In order to determine what aspects of an LR are useful for MT practitioners, a user study was made, providing a guide to the most relevant metadata and the most relevant quality criteria. We have seen that many resources exist which are useful for MT and similar work, but the majority are for (academic) research or educational use only, and as such not available for commercial use. Our work has revealed a list of gaps: coverage gap, awareness gap, quality gap, quantity gap. The paper ends with recommendations for a forward-looking strategy. Keywords: LR, catalogue, users 1. Introduction Language resources (LR) are indispensable for the development of tools for machine translation (MT) or various kinds of computer-assisted translation (CAT). In particular language corpora, both parallel and monolingual are considered most important for instance for MT, not only SMT but also hybrid MT. But corpora are expensive and labour-intensive to create or adapt e.g. for MT usability. Furthermore the extent of availability of LRs differs considerably from language to language. It is true that LRs have been created by EU and national projects and institutions, but they and information about them are scattered across Europe. In order to remedy this lack of overview for the professional user, it is important to apply a user-driven approach towards the identification and mapping of best practices in terms of collecting relevant LT resources. The Language Technology Observatory will provide easy access to information about LRs deemed to be useful for MT and other translation tools. In order to determine what aspects of an LR are useful for MT practitioners, a user study was made, providing a guide to the most relevant metadata and the most relevant quality criteria. In addition, knowledge and best practice has been extracted from previous studies (LetsMT!, META-SHARE etc.) on collecting relevant LT resources. 2. Related work We have taken the point of departure in existing catalogues and repositories. These comprise: CLARIN VLO (the Virtual Language Observatory, a search facility of metadata for language resources, META-SHARE, ELRA Catalogue of Language Resources, OPUS - the open parallel corpus, TAUS Data, a platform for sharing language data, LetsMT!, LIDER), FALCON ‘localization web’, PANACEA, a factory of Language Resources (LRs), TTC - Terminology extraction, translation tools, CESAR, EUROTERMBANK, JRC – Joint Research Centre. 3. User study Apart …

Content truncated to 500 words