Manawi: Using Multi-Word Expressions and Named Entities to Improve Machine Translation

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 201–206, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Manawi: Using Multi-Word Expressions and Named Entities to Improve Machine Translation Liling Tan and Santanu Pal Applied Linguistics, Translation and Interpretation Department Universit¨at des Saarlandes liling.tan@uni-saarland.de santanu.pal@uni-saarland.de Abstract We describe the Manawi1 (�����) sys- tem submitted to the 2014 WMT transla- tion shared task. We participated in the English-Hindi (EN-HI) and Hindi-English (HI-EN) language pair and achieved 0.792 for the Translation Error Rate (TER) score2 for EN-HI, the lowest among the competing systems. Our main innova- tions are (i) the usage of outputs from NLP tools, viz. billingual multi-word ex- pression extractor and named-entity rec- ognizer to improve SMT quality and (ii) the introduction of a novel filter method based on sentence-alignment features. The Manawi system showed the potential of improving translation quality by incorpo- rating multiple NLP tools within the MT pipeline. 1 Introduction In this paper, we present Saarland University (USAAR) submission to Workshop for Machine Translation 2014 (WMT 2014) using the Manawi MT system. We participated in the generic trans- lation shared task for the English-Hindi (EN-HI) and Hindi-English (HI-EN) language pairs. Our Manawi system showcased the incorpora- tion of NLP tools output within the MT pipeline; a bilingual MWE extractor and a bilingual NE rec- ognizer for English and Hindi were implemented. The output from these NLP tools was appended to the training corpus prior to the SMT model train- ing with the MOSES toolkit (Koehn et al., 2007). The resulting system achieves the lowest Transla- tion Error Rate (TER) among competing systems for the English-Hindi language pair. 1Multi-word expression And Named-entity And Wikipedia titles (Manawi) 2Lower TER often results in better translation The rest of the paper is structured as follow: Section 2 describes the implementation of the NLP tools; Section 3 outlines the corpus pre-processing before the MT training process; Section 4 de- scribes the MT system setup; Section 5 describes a simple post-processing component to handle Out-Of-Vocabulary words; Section 6 presents the WMT shared task results for the Manawi system and Section 6 concludes the paper. 2 NLP Tools Implementation 2.1 Bilingual MWE in MT Multi-Word Expressions (MWE) are defined as “idiosyncratic interpretations that cross word boundaries” (Sag et al., 2002). MWE can be made up of collocations (e.g. seem ridiculous : behuda dikhai), frozen expressions (e.g. exception han- dling : apavada sancalaka) or name entities (e.g. Johnny Cash : Johni Kesh). Jackendoff (1997) claims that the frequency of MWE and the fre- quency of single words in a speaker’s lexicon are almost equivalent. Bilingual MWE has shown to be useful for a variety of NLP applications such as multilin- gual information retrieval (Vechtomova, 2005) and Crosslingual/Multilingual Word Sense Dis- ambiguation (Tan and Bond, 2013; Finlayson and Kulkarni, 2011). For machine translation, vari- ous studies had introduced bilingual MWE to im- prove MT system performance. Lambert (2005) introduced bilingual MWE by grouping them as a single token before training alignment models …

Content truncated to 500 words