Creating Term and Lexicon Entries from Phrase Tables

Creating Term and Lexicon Entries from Phrase Tables Gregor Thurmair Linguatec g.thurmair@linguatec.de Vera Aleksić Linguatec v.aleksic@linguatec.de Abstract This document describes a tool which ex- tracts term and lexicon entries from SMT phrase tables, without further reference to monolingual data. It applies filters to such tables, and builds lexicon entries from the ‘good’ candidates. Error rates of the tool can be as low as 7.3%, accumu- lated from source, target, and transfer er- rors.1 1 Introduction It is a common understanding that machine trans- lation systems need to be adapted to the domain and text type they are supposed to translate. For knowledge-driven systems, such adaptation is done by means of lexicon update: The domain terminology is identified, and coded as a special additional lexicon repository, loaded at runtime. In the age of data-driven technology, terminolo- gy is extracted from corpus data, and so are translation equivalents for the found terms. 1.1 Task The task of the P2G (phrasetable2glossary) tool is to create proper bilingual lexicon entries from comparable corpus data; the technique should be usable for special domains, and should create output which can be imported into a backend (rule-based) MT system. The question what the target of a bilingual ex- traction component is, is difficult to define. Real term banks, even in the same domain, contain very different material, depending on the subdo- main and focus, and the skills of the translators involved. As a result, the term extraction process © 2012 European Association for Machine Translation. 1 This work was done in the context of the FP7-ICT projects ACCURAT (248347) (system core) and the PANACEA (248064) (language extensions). will always contain a step whereby humans in- vestigate a term list and decide which entry can- didates they want to keep for term bank import. The task of a term extraction tool is to prepare this candidate list. The quality of the extraction tool is determined by the effort it makes to go through this list. The approach of P2G consists of the following  steps: Step 1: Extract phrases with a good chance of being translations of each other. This means to apply word and phrase alignment to the input. Tools exist which do this.  Step 2: Not all phrases are well-formed terms. Therefore, the term candidates are fil- tered on several levels: Frequency filter: only phrases with a fre- quency and translation probability above a given threshold are considered as candidates. Fig. 1: Lexicon extraction workflow Proceedings of the 16th EAMT Conference, 28-30 May 2012, Trento, Italy 253 Linguistic filter: have an internal linguistic structure. Only candidates that match this structure are legal term candidates. Lexicon filter: This is an optional user- defined component which allows eliminat- ing non-terms.  Step 3: The resulting list will be given to human post-editing, to correct erroneous sys- tem decisions. The quality of the tool is a key factor for the efficiency of this step.  Step 4: The correction result will be import- ed into a rule-based MT …

Content truncated to 500 words