Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1535–1546 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1141 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1535–1546 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1141 Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search Chris Hokamp ADAPT Centre Dublin City University chris.hokamp@computing.dcu.ie Qun Liu ADAPT Centre Dublin City University qun.liu@dcu.ie Abstract We present Grid Beam Search (GBS), an algorithm which extends beam search to allow the inclusion of pre-specified lex- ical constraints. The algorithm can be used with any model that generates a se- quence ˆy = {y0 . . . yT }, by maximizing p(y|x) = � t p(yt|x; {y0 . . . yt−1}). Lex- ical constraints take the form of phrases or words that must be present in the out- put sequence. This is a very general way to incorporate additional knowledge into a model’s output without requiring any modification of the model parameters or training data. We demonstrate the feasibil- ity and flexibility of Lexically Constrained Decoding by conducting experiments on Neural Interactive-Predictive Translation, as well as Domain Adaptation for Neural Machine Translation. Experiments show that GBS can provide large improvements in translation quality in interactive scenar- ios, and that, even without any user in- put, GBS can be used to achieve signifi- cant gains in performance in domain adap- tation scenarios. 1 Introduction The output of many natural language processing models is a sequence of text. Examples include automatic summarization (Rush et al., 2015), ma- chine translation (Koehn, 2010; Bahdanau et al., 2014), caption generation (Xu et al., 2015), and di- alog generation (Serban et al., 2016), among oth- ers. In some real-world scenarios, additional infor- mation that could inform the search for the opti- mal output sequence may be available at inference time. Humans can provide corrections after view- ing a system’s initial output, or separate classifi- cation models may be able to predict parts of the output with high confidence. When the domain of the input is known, a domain terminology may be employed to ensure specific phrases are present in a system’s predictions. Our goal in this work is to find a way to force the output of a model to contain such lexical constraints, while still taking advan- tage of the distribution learned from training data. For Machine Translation (MT) usecases in par- ticular, final translations are often produced by combining automatically translated output with user inputs. Examples include Post-Editing (PE) (Koehn, 2009; Specia, 2011) and Interactive- Predictive MT (Foster, 2002; Barrachina et al., 2009; Green, 2014). These interactive scenarios can be unified by considering user inputs to be lex- ical constraints which guide the search for the op- timal output sequence. In this paper, we formalize the notion of lexi- cal constraints, and propose a decoding algorithm which allows the specification of subsequences that are required to be present …

Content truncated to 500 words