Neural Machine Translation Incorporating Named Entity

Proceedings of the 27th International Conference on Computational Linguistics, pages 3240–3250 Santa Fe, New Mexico, USA, August 20-26, 2018. 3240 Neural Machine Translation Incorporating Named Entity 1Arata Ugawa 2Akihiro Tamura 2Takashi Ninomiya 1,3Hiroya Takamura 1Manabu Okumura 1Department of Information and Communications Engineering, Tokyo Institute of Technology 2Graduate School of Science and Engineering, Ehime University 3National Institute of Advanced Industrial Science and Technology ugawa.a.aa@m.titech.ac.jp {tamura, ninomiya}@cs.ehime-u.ac.jp {takamura, oku}@pi.titech.ac.jp Abstract This study proposes a new neural machine translation (NMT) model based on the encoder- decoder model that incorporates named entity (NE) tags of source-language sentences. Con- ventional NMT models have two problems enumerated as follows: (i) they tend to have difficulty in translating words with multiple meanings because of the high ambiguity, and (ii) these mod- els’ ability to translate compound words seems challenging because the encoder receives a word, a part of the compound word, at each time step. To alleviate these problems, the encoder of the proposed model encodes the input word on the basis of its NE tag at each time step, which could reduce the ambiguity of the input word. Furthermore, the encoder introduces a chunk-level LSTM layer over a word-level LSTM layer and hierarchically encodes a source-language sen- tence to capture a compound NE as a chunk on the basis of the NE tags. We evaluate the proposed model on an English-to-Japanese translation task with the ASPEC, and English-to-Bulgarian and English-to-Romanian translation tasks with the Europarl corpus. The evaluation results show that the proposed model achieves up to 3.11 point improvement in BLEU. 1 Introduction Neural machine translation (NMT) models based on the encoder-decoder model, also known as the sequence-to-sequence model (Sutskever et al., 2014), have successfully shown their quality translation. Consequently, various NMT models are studied in the field of machine translation. To date, the most suc- cessful model is the bi-directional multi-layered encoder-decoder model with long short term memory (LSTM) (Hochreiter and Schmidhuber, 1997) and an attention mechanism (Luong et al., 2015; Bahdanau et al., 2015), also known as attention-based NMT. LSTM and the attention mechanism are introduced to mitigate the difficulty in handling long sentences in the encoder-decoder model. The conventional attention-based NMT model is known to achieve high translation accuracy in bilingual evaluation un- derstudy (BLEU). However, this model encounters two general problems: (i) it tends to have difficulty in translating words with multiple meanings because their translations have high ambiguity, and (ii) translation of compound words seems difficult because the encoder receives only a word, a part of the compound word, at each time step. This study proposes a new NMT model based on the encoder-decoder model, incorporating named entity (NE) information in source-language sentences. The proposed model alleviates the problems of the conventional attention-based NMT by incorporating information of NE tags to the encoder and modeling chunk information of NE tags as the encoder’s network structures. The encoder of the proposed model This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http://creativecommons.org/licenses/by/4.0/. 3241 encodes the input …

Content truncated to 500 words