Name Translation based on Fine-grained Named Entity Recognition in a Single Language

Name Translation based on Fine-grained Named Entity Recognition in a Single Language Kugatsu Sadamitsu, Itsumi Saito, Taichi Katayama, Hisako Asano and Yoshihiro Matsuo NTT Media Intelligence Laboratories, NTT Corporation 1-1 Hikari-no-oka, Yokosuka-shi, Kanagawa, 239-0847, Japan {sadamitsu.kugatsu, saito.itsumi, katayama.taichi, asano.hisako, matsuo.yoshihiro} @lab.ntt.co.jp Abstract We propose named entity abstraction methods with fine-grained named entity labels for improving statistical machine translation (SMT). The methods are based on a bilingual named entity recognizer that uses a monolingual named entity recognizer with transliteration. Through experiments, we demonstrate that incorporating fine-grained named entities into statistical machine translation improves the accuracy of SMT with more adequate granularity compared with the standard SMT, which is a non-named entity abstraction method. Keywords: Statistical Machine Translation, Extended Named Entity, Bilingual Named Entity Recognition 1. Introduction One of the issues in statistical machine translation (SMT) systems is named entity (NE) translation. There are two major problems with NEs. One, NEs cause data sparseness in training data due to the large range of NE tokens, and two, NEs are often treated as unknown words in test data because they have not been included in training data. For example, the two phrases “went to Tokyo by train” and “went to Osaka by train” are similar, but most translation phrase ta- bles would not treat these as a similar pair because of the difference between “Tokyo” and “Osaka”. Even worse, if only “Tokyo” appears in the training cor- pus, “Osaka” becomes a unknown word. One solu- tion for these problems is abstracting NE for SMT and transliterating the NEs. Several approaches to this ef- fect have been proposed (Li et al., 2013; Hermjakob et al., 2008; Hassan et al., 2007; Al-Onaizan and Knight, 2002; Knight and Graehl, 1997), but they used coarse NE classes such as “PERSON” and “GPE” and did not focus on the relevance between the granularity of NE classes and SMT. Dynamic clustering for chunks as EM algorithms (Och, 1999), in contrast, does not depend on the use of NEs. This method is effective in cases where the amount of the parallel corpus is enough, but it is reasonable to utilize another informa- tive resource when the amount of a parallel corpus is short. Our hypothesis is that focusing more on adequate fine- grained NE classes and incorporating them into SMT should improve the accuracy of SMT. In this paper, we propose an NE abstraction method with extended NE (ENE) labels (Sekine, 2008) as fine-grained NE labels and compare them with a standard SMT (with neither NER nor NE abstraction) using annotated bilingual NE test data. To this end, we also have to construct high accuracy bilingual NER to abstract NE chunks in training bilin- gual corpus to NE labels. It is relatively easy to extract NEs from both source and target languages when the NERs or NE resources of both languages exist. How- ever, occasionally, only fine-grained NE resources are available in a single language, as is the case with the Japanese NE annotated corpus belonging to Sekine’s ENE definition …

Content truncated to 500 words