Negative Lexically Constrained Decoding for Paraphrase Generation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6047–6052 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6047 Negative Lexically Constrained Decoding for Paraphrase Generation Tomoyuki Kajiwara Institute for Datability Science Osaka University, Osaka, Japan kajiwara@ids.osaka-u.ac.jp Abstract Paraphrase generation can be regarded as monolingual translation. Unlike bilingual machine translation, paraphrase generation rewrites only a limited portion of an input sentence. Hence, previous methods based on machine translation often perform conserva- tively to fail to make necessary rewrites. To solve this problem, we propose a neural model for paraphrase generation that first identifies words in the source sentence that should be paraphrased. Then, these words are para- phrased by the negative lexically constrained decoding that avoids outputting these words as they are. Experiments on text simplification and formality transfer show that our model im- proves the quality of paraphrasing by making necessary rewrites to an input sentence. 1 Introduction Paraphrase generation is a generic term for tasks that generate sentences semantically equivalent to input sentences. These techniques make it pos- sible to control information other than the mean- ing of the text. Typical paraphrase generation tasks include subtasks such as text simplification to control complexity, formality transfer to con- trol formality, grammatical error correction to con- trol fluency, and sentence compression to con- trol sentence length. These paraphrase gener- ation applications not only support communica- tion and language learning but also contribute to the performance improvement of other natu- ral language processing applications (Evans, 2011; ˇStajner and Popovi´c, 2016). aParaphrase generation can be considered as monolingual machine translation problem. Sentential paraphrases with different complexi- ties (Coster and Kauchak, 2011; Xu et al., 2015) and formalities (Rao and Tetreault, 2018) were created manually, and parallel corpora special- ized for each subtask were constructed. As in the field of machine translation, phrase- based (Coster and Kauchak, 2011; Xu et al., 2012) and syntax-based (Zhu et al., 2010; Xu et al., 2016) methods were proposed early. In re- cent years, the encode-decoder model based on the attention mechanism (Nisioi et al., 2017; Zhang and Lapata, 2017; Jhamtani et al., 2017; Niu et al., 2018) has been studied, inspired by the success of neural machine transla- tion (Bahdanau et al., 2015). In machine translation, all words appearing in an input sentence must be rewritten in the target language. However, paraphrase generation does not require rewriting of all words. When some cri- teria are provided, words not satisfying the criteria in the input sentence are identified and rewritten. For example, the criterion for text simplification is the textual complexity, and rewrites complex words to simpler synonymous words. Owing to the characteristics of the task where only a limited portion of an input sentence needs to be rewrit- ten, previous methods based on machine transla- tion often perform conservatively and fail to pro- duce necessary rewrites (Zhang and Lapata, 2017; Niu et al., 2018). To solve the problem of con- servative paraphrasing that copies many parts of …

Content truncated to 500 words