Learning Rhyming Constraints using Structured Adversaries

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6025–6031, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6025 Learning Rhyming Constraints using Structured Adversaries Harsh Jhamtani 1 Sanket Vaibhav Mehta 1 Jaime Carbonell 1 Taylor Berg-Kirkpatrick 2 1 School of Computer Science, Carnegie Mellon University 2 Department of Computer Science and Engineering, University of California San Diego {jharsh,svmehta,jgc}@cs.cmu.edu, tberg@ucsd.eng.edu Abstract Existing recurrent neural language models often fail to capture higher-level structure present in text: for example, rhyming patterns present in poetry. Much prior work on poetry generation uses manually defined constraints which are satisfied during decoding using ei- ther specialized decoding procedures or rejec- tion sampling. The rhyming constraints them- selves are typically not learned by the gener- ator. We propose an alternate approach that uses a structured discriminator to learn a po- etry generator that directly captures rhyming constraints in a generative adversarial setup. By causing the discriminator to compare po- ems based only on a learned similarity ma- trix of pairs of line ending words, the pro- posed approach is able to successfully learn rhyming patterns in two different English po- etry datasets (Sonnet and Limerick) without explicitly being provided with any phonetic in- formation. 1 Introduction Many existing approaches to text generation rely on recurrent neural networks trained using likeli- hood on sequences of words or characters. How- ever, such models often fail to capture overall structure and coherency in multi-sentence or long- form text (Bosselut et al., 2018; Holtzman et al., 2018). To rectify this, prior work has proposed losses which encourage overall coherency or other desired behavior (Li et al., 2016; Zhang and Lap- ata, 2017; Bosselut et al., 2018). However, most of these approaches rely on manually provided defi- nitions of what constitutes a good or suitable struc- ture, thereby limiting their applicability. In this pa- per we propose a method for English poetry gen- eration that directly learns higher-level rhyming constraints as part of a generator without requiring strong manual intervention. Prior works on poetry generation (Oliveira, 2017; Ghazvininejad et al., 2018) have focused mostly on ad-hoc decoding procedures to generate reasonable poetry, often re- lying on pruning from a set of candidate outputs to encourage desired behavior such as presence of explicitly-defined rhyming patterns. We propose an adversarial approach to poetry generation that, by adding structure and inductive bias into the discriminator, is able to learn rhyming constraints directly from data without prior knowl- edge. The role of the discriminator is to try to dis- tinguish between generated and real poems during training. We propose to add inductive bias via the choice of discriminator architecture: We require the discriminator to reason about poems through pairwise comparisons between line ending words. These learned word comparisons form a similar- ity matrix for the poem within the discriminator’s architecture. Finally, the discriminator evaluates the poem through a 2D convolutional classifier ap- plied directly to this matrix. This …

Content truncated to 500 words