Enhancing Phrase-Based Statistical Machine Translation by Learning Phrase Representations Using Long Short-Term Memory Network

Proceedings of Recent Advances in Natural Language Processing, pages 25–32, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_004 25 Enhancing Phrase-Based Statistical Machine Translation by Learning Phrase Representations Using Long Short-Term Memory Network Benyamin Ahmadnia1 and Bonnie J. Dorr2 1Department of Computer Science, Tulane University, New Orleans, LA, USA 2Institute for Human and Machine Cognition (IHMC), Ocala, FL, USA ahmadnia@tulane.edu , bdorr@ihmc.us Abstract Phrases play a key role in Machine Trans- lation (MT). In this paper, we apply a Long Short-Term Memory (LSTM) model over conventional Phrase-Based Statistical MT (PBSMT). The core idea is to use an LSTM encoder-decoder to score the phrase table generated by the PBSMT de- coder. Given a source sequence, the en- coder and decoder are jointly trained in order to maximize the conditional prob- ability of a target sequence. Analyti- cally, the performance of a PBSMT sys- tem is enhanced by using the conditional probabilities of phrase pairs computed by an LSTM encoder-decoder as an addi- tional feature in the existing log-linear model. We compare the performance of the phrase tables in the PBSMT to the performance of the proposed LSTM and observe its positive impact on translation quality. We construct a PBSMT model us- ing the Moses decoder and enrich the Lan- guage Model (LM) utilizing an external dataset. We then rank the phrase tables using an LSTM-based encoder-decoder. This method produces a gain of up to 3.14 BLEU score on the test set. 1 Introduction The three most essential components of a Sta- tistical Machine Translation (SMT) system are: (1) Translation Model (TM); (2) Language Model (LM); and (3) Reordering Model (RM). Among these models, the RM plays an important role in Phrase-Based SMT (PBSMT) (Marcu and Wong, 2002; Koehn et al., 2003), and it still remains a major focus of intense study (Kanouchi et al., 2016; Du and Way, 2017; Chen et al., 2019). The RM is required since different languages exercise different syntactic ordering. For instance, adjectives in English precede the noun, while they typically follow the noun in Spanish (the cloudy sky versus el cielo nublado); in Persian the verb precedes the subject, and in Chinese the verb comes last. As a result, source language phrases cannot be translated and placed in the same order in the generated translation in the target language, but phrase movements have to be considered. This is the role of the RM. Estimating the exact distance of movement for each phrase is too sparse; there- fore, instead, the lexicalized RM (Koehn, 2009) estimates phrase movements using only a few re- ordering types, such as a monotonous order, where the order is preserved, or a swap, when the order of two consecutive source phrases is inverted when their translations are placed in the target side. Neural Machine Translation (NMT) has been receiving significant attention due to its impressive translation performance (Bahdanau et al., 2015). NMT differs from SMT in its adoption of a large neural network to perform the entire translation process in one shot, for which an …

Content truncated to 500 words