Improved beam search with constrained softmax for NMT

Improved Beam Search with Constrained Softmax for NMT Xiaoguang Hu huxiaoguang@baidu.com Wei Li liwei08@baidu.com Xiang Lan lanxiang@baidu.com Hua Wu wu hua@baidu.com Haifeng Wang wanghaifeng@baidu.com Baidu Inc, Beijing, China Abstract We propose an improved beam search decoding algorithm with constrained softmax operations for neural machine translation (NMT). NMT is a newly emerging approach to predict the best translation by building a neural network instead of a log-linear model. It has achieved com- parable translation quality to the existing phrase-based statistical machine translation systems. However, how to perform efficient decoding for NMT is still challenging, especially for com- mercial systems which provide real-time translation service. Unlike the standard beam search algorithm, we use a priority queue to choose the best hypothesis for the next search, which drastically reduces search space. Another time consuming factor is the softmax operation in the output layer because of the large target vocabulary size. To solve this problem, we introduce a limited word set of translation candidates to greatly reduce the computation complexity. Our experiments show that, under the GPU environment, our method achieves a speed about 3.5 times faster than the well optimized baseline system without sacrificing the translation quality. Our method translates about 117 words per second, beating the real-time translation require- ments for practical MT systems. 1 Introduction Neural network models have become increasingly popular in recent machine translation tasks. Initially, the neural networks were designed for language models, such as neural network lan- guage model (Bengio et al., 2003) and recurrent neural network language model (Mikolov et al., 2010). Devlin et al. (2014) proposed a neural network joint model to improve transla- tion by considering the source contexts jointly. The above work integrated the neural network models into the traditional translation decoders as additional features in a log-linear framework (Och and Ney, 2002). Rather than applying a neural network as a part of the existing system, Cho et al. (2014) proposed a whole new RNN Encoder-Decoder approach which generates the target translation with a neural network directly. Bahdanau et al. (2014) extended the above ap- proach by allowing an RNN model to automatically (soft-)search for parts of a source sentence to predict a target word, and achieved a translation performance comparable to the existing state-of-the-art phrase-based systems (Koehn et al., 2003). Although NMT shows high potential, its decoding efficiency is still challenging. Cho et al. (2014) implemented a standard beam search decoding algorithm (Koehn, 2004) for an RNN Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 297 Encoder-Decoder system published as GroundHog 1. The beam search algorithm successfully reduces the search space from exponential size to polynomial size, and it is able to translate about ten words per second. Even though the speed is acceptable for most research tasks, it is not yet efficient enough to meet the requirement of commercial systems for providing real-time translation service. Huang and Chiang (2007) proposed the forest rescoring algorithm which succeed to accelerate the decoding by …

Content truncated to 500 words