Dudley North visits North London: Learning When to Transliterate to Arabic

Proceedings of NAACL-HLT 2013, pages 439–444, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Dudley North visits North London: Learning When to Transliterate to Arabic Mahmoud Azab Houda Bouamor Carnegie Mellon University P.O. Box 24866, Doha, Qatar {mazab, hbouamor, behrang, ko}@qatar.cmu.edu Behrang Mohit Kemal Oflazer Abstract We report the results of our work on automat- ing the transliteration decision of named en- tities for English to Arabic machine trans- lation. We construct a classification-based framework to automate this decision, evalu- ate our classifier both in the limited news and the diverse Wikipedia domains, and achieve promising accuracy. Moreover, we demon- strate a reduction of translation error and an improvement in the performance of an English-to-Arabic machine translation sys- tem. 1 Introduction Translation of named entities (NEs) is important for NLP applications such as Machine Translation (MT) and Cross-lingual Information Retrieval. For MT, NEs are major subset of the out-of-vocabulary terms (OOVs). Due to their diversity, they cannot always be found in parallel corpora, dictionaries or gazetteers. Thus, state-of-the-art of MT needs to handle NEs in specific ways. For instance, in the English-Arabic automatic translation example given in Figure 1, the noun ”North” has been erroneously translated to ”������� ���� /Al$mAlyp ” (indicating the north direction in English) instead of being translit- erated to ” ������ / nwrv”. As shown in Figure 1, direct translation of in- vocabulary terms could degrade translation quality. Also blind transliteration of OOVs does not neces- sarily contribute to translation adequacy and may ac- tually create noisy contexts for the language model and the decoder. English Input: Dudley North was an English merchant. SMT output: � ���� ������������ � ��� ��� ������� ���� ������ ���� kAn dwdly Al$mAlyp tAjr AlInjlyzyp. Correct Translation: ��� ������������ ��� ��� ������ ������ ���� kAn dwdly nwrv tAjr Injlyzy. Figure 1: Example of a NE translation error. An intelligent decision between translation and transliteration should use semantic and contextual information such as the type of the named-entity and the surrounding terms. In this paper, we con- struct and evaluate a classification-based framework to automate the translation vs. transliteration deci- sion. We evaluate our classifier both in the limited news and diverse Wikipedia domains, and achieve promising accuracy. Moreover, we conduct an ex- trinsic evaluation of the classifier within an English to Arabic MT system. In an in-domain (news) MT task, the classifier contributes to a modest (yet sig- nificant) improvement in MT quality. Moreover, for a Wikipedia translation task, we demonstrate that our classifier can reduce the erroneous translation of 60.5% of the named entities. In summary our contributions are: (a) We au- tomatically construct a bilingual lexicon of NEs paired with the transliteration/translation decisions in two domains.1 (b) We build a binary classi- fier for transliteration and translation decision with a promising accuracy (c) We demonstrate its utility 1The dataset can be found at http://www.qatar.cmu.edu/˜behrang/NETLexicon . 439 within an MT framework. 2 Learning when to transliterate We model the decision as a binary classification at the token level. …

Content truncated to 500 words