Sequence-to-Sequence Learning as Beam-Search Optimization

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1296–1306, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush School of Engineering and Applied Sciences Harvard University Cambridge, MA, USA {swiseman,srush}@seas.harvard.edu Abstract Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general- purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work, we introduce a model and beam- search training scheme, based on the work of Daum´e III and Marcu (2005), that extends seq2seq to learn global sequence scores. This structured approach avoids classical biases as- sociated with local training and unifies the training loss with the test-time usage, while preserving the proven model architecture of seq2seq and its efficient training approach. We show that our system outperforms a highly- optimized attention-based seq2seq system and other baselines on three different sequence to sequence tasks: word ordering, parsing, and machine translation. 1 Introduction Sequence-to-Sequence learning with deep neural networks (herein, seq2seq) (Sutskever et al., 2011; Sutskever et al., 2014) has rapidly become a very useful and surprisingly general-purpose tool for nat- ural language processing. In addition to demon- strating impressive results for machine translation (Bahdanau et al., 2015), roughly the same model and training have also proven to be useful for sen- tence compression (Filippova et al., 2015), parsing (Vinyals et al., 2015), and dialogue systems (Ser- ban et al., 2016), and they additionally underlie other text generation applications, such as image or video captioning (Venugopalan et al., 2015; Xu et al., 2015). The dominant approach to training a seq2seq sys- tem is as a conditional language model, with training maximizing the likelihood of each successive tar- get word conditioned on the input sequence and the gold history of target words. Thus, training uses a strictly word-level loss, usually cross-entropy over the target vocabulary. This approach has proven to be very effective and efficient for training neural lan- guage models, and seq2seq models similarly obtain impressive perplexities for word-generation tasks. Notably, however, seq2seq models are not used as conditional language models at test-time; they must instead generate fully-formed word sequences. In practice, generation is accomplished by searching over output sequences greedily or with beam search. In this context, Ranzato et al. (2016) note that the combination of the training and generation scheme just described leads to at least two major issues: 1. Exposure Bias: the model is never exposed to its own errors during training, and so the in- ferred histories at test-time do not resemble the gold training histories. 2. Loss-Evaluation Mismatch: training uses a word-level loss, while at test-time we target improving sequence-level evaluation metrics, such as BLEU (Papineni et al., 2002). We might additionally add the concern of label bias (Lafferty et al., 2001) to the list, since word- probabilities at each time-step are locally normal- ized, guaranteeing that successors …

Content truncated to 500 words