SMT of German patents at WIPO: decompounding and verb structure pre-reordering

An efficient method to assist non-expert users in extending dictionaries by assigning stems and inflectional paradigms to unknown words Miquel Espl`a-Gomis mespla@dlsi.ua.es Rafael C. Carrasco carrasco@dlsi.ua.es V´ıctor M. S´anchez-Cartagena vmsanchez@dlsi.ua.es Mikel L. Forcada mlf@dlsi.ua.es Dept. de Llenguatges i Sistemes Inform`atics, Universitat d’Alacant, 03071 Alacant, Spain Felipe S´anchez-Mart´ınez fsanchez@dlsi.ua.es Juan Antonio P´erez-Ortiz japerez@dlsi.ua.es Abstract A method is presented to assist users with no background in linguistics in adding the unknown words in a text to monolingual dictionaries such as those used in rule- based machine translation systems. Adding a word to these dictionaries requires identi- fying its stem and the inflection paradigm to be used in order to generate all its word forms. Our method is based on a previous interactive approach in which non-expert users were asked to validate whether some tentative word forms were correct forms of the new word; these validations were then used to determine the most appropriate stem and paradigm. The previous approach was based on a set of intuitive heuristics designed both to obtain an estimate of the eligibility of each candidate stem/paradigm combination and to determine the word form to be validated at each step. Our new approach however uses formal models for both tasks (a hidden Markov model to esti- mate eligibility and a decision tree to select the word form) and achieves significantly better results. 1 Introduction Creation of the linguistic data (such as mono- lingual dictionaries, bilingual dictionaries, trans- fer rules, etc.) required by rule-based machine translation (RBMT) systems has usually involved teams of trained linguists. However, development costs could be significantly reduced by involving a broader group of non-expert users in the extension c⃝ 2014 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. of these resources. This may include, for instance, the very same users of the machine translation (MT) system or accidental collaborators recruited through crowdsourcing platforms (Wang et al., 2013). The scenario considered in this paper is that of non- expert users (in a general sense) who have to in- troduce into the two monolingual dictionaries1 of a RBMT system the unknown words found in an input text so that the system is subsequently able to correctly translate them.2 Note, however, that our method could be applied to the addition of entries into the morphological dictionaries used in many other natural language processing applications. The objective of our work is to obtain a system which can be used not only to add the particular unknown word form (for example, wants) to the dictionary, but also to assist in discovering an appropriate stem and a suitable inflection paradigm so that all the word forms of the unknown word and their associ- ated morphological inflection information (such as wants, verb, present, 3rd person or wanting, verb, gerund) can be inserted as well. Inflection paradigms are commonly introduced in RBMT systems in order to group regularities in the inflection of a set of words;3 a paradigm is usually defined as a …

Content truncated to 500 words