Confusion Network for Arabic Name Disambiguation and Transliteration in Statistical Machine Translation

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 433–443, Dublin, Ireland, August 23-29 2014. Confusion Network for Arabic Name Disambiguation and Translitera- tion in Statistical Machine Translation Young-Suk Lee IBM T. J. Watson Research Center 1101 Kitchawan Road Yorktown Heights, NY 10598, USA ysuklee@us.ibm.com Abstract Arabic words are often ambiguous between name and non-name interpretations, frequently leading to incorrect name translations. We present a technique to disambiguate and transliter- ate names even if name interpretations do not exist or have relatively low probability distribu- tions in the parallel training corpus. The key idea comprises named entity classing at the pre- processing step, decoding of a simple confusion network created from the name class label and the input word at the statistical machine translation step, and transliteration of names at the post-processing step. Human evaluations indicate that the proposed technique leads to a statis- tically significant translation quality improvement of highly ambiguous evaluation data sets without degrading the translation quality of a data set with very few names. 1 Introduction Arabic person and location names are often ambiguous between name and non-name interpretations, as noted in (Hermjakob et al., 2008; Zayed et al., 2013). (1) and (2) illustrate such ambiguities for Iraqi Arabic, where the ambiguous names and their translations are in bold-face and the Buckwalter transliteration of Arabic is provided in parentheses:1 (1) a.ب ﺔﺳرﺪﻤﻟا ﻢﯾ ﺔﻘﺸﺑ ﻦﻛﺎﺳ ﻲﻧاءاﺮﻀﺧ (Any sAkn b$qp ym Almdrsp bxDrA') I live in an apartment near the school in Khadraa b.ءاﺮﻀﺧﺔﻏﻮﺒﺼﻣ (mSbwgp xDrA') It is painted green (2) a. رﺪﻘﯿﺷحﺎﺒﺻﻚﻟإ لﻮﻘﯾ ($yqdr SbAH yqwl Alk) What can Sabah tell you? b.حﺎﺒﺻمﺎﺴﺣ ﺐﯿﻘﻧ ﺪﯿﻛأ ﺖﻧإ ﺮﯿﺨﻟا (SbAH Alxyr Ant Akyd nqyb HsAm) Good morning you must be captain Hosam In this paper, we propose a technique for disambiguating and transliterating Arabic names in an end-to-end statistical machine translation system. The key idea lies in name classing at the pre- processing step, decoding of a simple confusion network created from the class label $name, and the input word at the machine translation step, and transliteration of names by a character-based phrase transliteration model at the post-processing step. While Bertoldi et al. (2007) propose confusion network decoding to handle multiple speech recog- nition outputs for phrase translation and Dyer et al. (2008) generalize lattice decoding algorithm to This work is licenced under a Creative Commons Attribution 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http://creativecommons.org/licenses/by/4.0/ 1Arabic should be read from right to left, and the Buckwalter transliteration should be read from left to right. 433 tackle word segmentation ambiguities for hierarchical phrase-based translation, the current proposal is the first to deploy a confusion network for name disambiguation and translation. The character-based phrase transliteration model captures the asymmetry between Arabic and English vowel systems by treating English vowels as spontaneous words attachable to the neighboring target phrases for phrase (a sequence of characters) acquisition. Confusion network decoding enables the system to choose between name and other …

Content truncated to 500 words