A Melody-Conditioned Lyrics Language Model

Proceedings of NAACL-HLT 2018, pages 163–172 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics A Melody-conditioned Lyrics Language Model Kento Watanabe1, Yuichiroh Matsubayashi1, Satoru Fukayama2, Masataka Goto2, Kentaro Inui1,3, Tomoyasu Nakano2 1Graduate School of Information Sciences, Tohoku University, 2National Institute of Advanced Industrial Science and Technology (AIST), 3RIKEN Center for Advanced Intelligence Project {kento.w, y-matsu, inui}@ecei.tohoku.ac.jp, {s.fukayama, m.goto, t.nakano}@aist.go.jp Abstract This paper presents a novel, data-driven lan- guage model that produces entire lyrics for a given input melody. Previously proposed mod- els for lyrics generation suffer from the in- ability of capturing the relationship between lyrics and melody partly due to the unavail- ability of lyrics-melody aligned data. In this study, we first propose a new practi- cal method for creating a large collection of lyrics-melody aligned data and then create a collection of 1,000 lyrics-melody pairs aug- mented with precise syllable-note alignments and word/sentence/paragraph boundaries. We then provide a quantitative analysis of the correlation between word/sentence/paragraph boundaries in lyrics and melodies. We then propose an RNN-based lyrics language model conditioned on a featurized melody. Experi- mental results show that the proposed model generates fluent lyrics while maintaining the compatibility between boundaries of lyrics and melody structures. 1 Introduction Writing lyrics for a given melody is a challenging task. Unlike prose text, writing lyrics requires both knowledge and consideration of music-specific properties such as the structure of melody, rhythms, etc. (Austin et al., 2010; Ueda, 2010). A simple ex- ample is the correlation between word boundaries in lyrics and the rests in a melody. As shown in Figure 1, a single word spanning beyond a long melody rest can sound unnatural. When writing lyrics, a lyricist must consider such constraints in content and lexical selection, which can impose extra cognitive loads. This consideration when writing lyrics has mo- tivated a wide-range of studies for the task of computer-assisted lyrics writing (Barbieri et al., 2012; Abe and Ito, 2012; Potash et al., 2015; Watan- abe et al., 2017). Such studies aim to model the ⁄@@ Flute 1 , - 2) )))) ) ) ) ) ) )! - ) ) #) ) (! , ⁄ Fl 5 , - 2) ) ) ) ) ) ) ) ) ) ) , ) ) ) ) (! , ⁄ Fl 9 , - 2) ) ) ) ) ) ) ) ) ) )! - ) ) #) ) (! , ⁄ Fl 13 , - 2) ) ) ) ) ) ) ) ) ) ) , ) ) ) ) (! , ⁄ Fl 17 , - 7) ) ) ) ) 7)! . ) #) ) ) ) - 7) ) ) ) ) ( ⁄ Fl 21 , ) ) ) ) ) - 7) )!#) ) ) ) - 7) )! ) ) ) ' ⁄ Fl 25 )! ) ) ) ) #) ) )! ) ) ) )! - )!) ) ) 7)! . ) ) ⁄ Fl 29 )!) …

Content truncated to 500 words