A New Approach for English-Chinese Named Entity Alignment

A New Approach for English-Chinese Named Entity Alignment Donghui Feng∗ Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina Del Rey, CA, U.S.A, 90292 donghui@isi.edu Yajuan Lv† Ming Zhou† †Microsoft Research Asia 5F Sigma Center, No.49 Zhichun Road, Haidian Beijing, China, 100080 {t-yjlv, mingzhou}@microsoft.com Abstract∗ Traditional word alignment approaches cannot come up with satisfactory results for Named Entities. In this paper, we propose a novel approach using a maximum entropy model for named entity alignment. To ease the training of the maximum entropy model, bootstrapping is used to help supervised learning. Unlike previous work reported in the literature, our work conducts bilingual Named Entity alignment without word segmentation for Chinese and its performance is much better than that with word segmentation. When compared with IBM and HMM alignment models, experimental results show that our approach outperforms IBM Model 4 and HMM significantly. 1 Introduction This paper addresses the Named Entity (NE) alignment of a bilingual corpus, which means building an alignment between each source NE and its translation NE in the target language. Research has shown that Named Entities (NE) carry essential information in human language (Hobbs et al., 1996). Aligning bilingual Named Entities is an effective way to extract an NE translation list and translation templates. For example, in the following sentence pair, aligning the NEs, [Zhi Chun road] and [知春路] can produce a translation • template correctly. Can I get to [LN Zhi Chun road] by eight o’clock? • 八点我能到 [LN 知春路]吗? In addition, NE alignment can be very useful for Statistical Machine Translation (SMT) and Cross- Language Information Retrieval (CLIR). A Named Entity alignment, however, is not easy to obtain. It requires both Named Entity Recognition (NER) and alignment be handled correctly. NEs may not be well recognized, or only ∗ The work was done while the first author was visiting Microsoft Research Asia. parts of them may be recognized during NER. When aligning bilingual NEs in different languages, we need to handle many-to-many alignments. And the inconsistency of NE translation and NER in different languages is also a big problem. Specifically, in Chinese NE processing, since Chinese is not a tokenized language, previous work (Huang et al., 2003) normally conducts word segmentation and identifies Named Entities in turn. This involves several problems for Chinese NEs, such as word segmentation error, the identification of Chinese NE boundaries, and the mis-tagging of Chinese NEs. For example, “国防部长” in Chinese is really one unit and should not be segmented as [ON 国防 部]/长. The errors from word segmentation and NER will propagate into NE alignment. In this paper, we propose a novel approach using a maximum entropy model to carry out English- Chinese Named Entity1 alignment. NEs in English are first recognized by NER tools. We then investigate NE translation features to identify NEs in Chinese and determine the most probable alignment. To ease the training of the maximum entropy model, bootstrapping is used to help supervised learning. On the other hand, to avoid error …

Content truncated to 500 words