GhostWriter: Using an LSTM for Automatic Rap Lyric Generation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1919–1924, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. GhostWriter: Using an LSTM for Automatic Rap Lyric Generation Peter Potash, Alexey Romanov, Anna Rumshisky Dept. of Computer Science University of Massachusetts Lowell Lowell, MA 01854 {ppotash,aromanov,arum}@cs.uml.edu Abstract This paper demonstrates the effectiveness of a Long Short-Term Memory language model in our initial efforts to generate un- constrained rap lyrics. The goal of this model is to generate lyrics that are simi- lar in style to that of a given rapper, but not identical to existing lyrics: this is the task of ghostwriting. Unlike previ- ous work, which defines explicit templates for lyric generation, our model defines its own rhyme scheme, line length, and verse length. Our experiments show that a Long Short-Term Memory language model pro- duces better “ghostwritten” lyrics than a baseline model. 1 Introduction Ghostwriting defines a distinction between the performer/presenter of text, lyrics, etc, and the cre- ator of text/lyrics. The goal is to present some- thing in a style that is believable enough to be credited to the performer. In the domain of rap specifically, rappers sometimes function as ghost- writers early on before embarking on their own public careers, and there are even businesses that provide written lyrics as a service 1. The goal of GhostWriter is to produce a system that can take a given artist’s lyrics and generate similar yet unique lyrics. To accomplish this, we must cre- ate a language model to produce text, while also understanding what ’style’ means in a quantitative sense. The contribution of this paper is three-fold: (1) we present the ghostwriting problem of producing similar yet different lyrics; (2) we present compu- tational, quantitative evaluation methods for these 1http://www.rap-rebirth.com/, http://www.precisionwrittens.com/ rap-ghostwriters-for-hire/ two aspects; (3) we evaluate the performance of a Long Short-Term Memory (LSTM) vs n-gram model for this problem. 2 Related Work Recent work (Sutskever et al., 2011; Graves, 2013) has shown the effectiveness of Recurrent Neural Networks (RNNs) for text generation. In their works, the authors use an RNN to create a language model at the character level. The results are inspiring, as the models learn various gram- matical and punctuation rules, such as opening and closing parentheses, plus learning a large vo- cabulary of English words at the character level. Graves (2013) uses a variation of an RNN called LSTM architecture which creates a better lan- guage model than a regular RNN. Text generation for artistic purposes, such as po- etry and lyrics, has also been explored, often using templates and constraints (Oliveira et al., 2014; Barbieri et al., 2012). In regards to rap lyrics, Wu et al. (2013) present a system for rap lyric gener- ation that produces a single line of lyrics that are meant to be a response to a single line of input. However, the work that is most similar to ours is that of Malmi et al. (2015). The authors create fixed 16-line verses, …

Content truncated to 500 words