Modeling Structural Topic Transitions for Automatic Lyrics Generation

PACLIC 28 !422 Modeling Structural Topic Transitions for Automatic Lyrics Generation Kento Watanabe1, Yuichiroh Matsubayashi1, Kentaro Inui1, and Masataka Goto2 1Graduate School of Information Sciences Tohoku University, JAPAN 2National Institute of Advanced Industrial Science and Technology (AIST), JAPAN 1{kento.w，y-matsu，inui}@ecei.tohoku.ac.jp 2m.goto@aist.go.jp Abstract By adopting recent advances in music creation technologies, such as digital audio worksta- tions and singing voice synthesizers, people can now create songs in their personal com- puters. Computers can also assist in creat- ing lyrics or generating them automatically, although this aspect has been less thoroughly researched and is limited to rhyme and meter. This study focuses on the structural relations in Japanese lyrics. We present novel genera- tion models that capture the topic transitions between units peculiar to the lyrics, such as verse/chorus and line. These transitions are modeled by a Hidden Markov Model (HMM) for representing topics and topic transitions. To verify that our models generate context- suitable lyrics, we evaluate the models us- ing a log probability of lyrics generation and fill-in-the-blanks-type test. The results show that the language model is far more effective than HMM-based models, but the HMM-based approach successfully captures the inter-verse/chorus and inter-line relations. In the result of experimental evaluation, our approach captures the inter-verse/chorus and inter-line relations. 1 Introduction Recent music creation technologies such as digi- tal audio workstations and singing voice synthe- sizers (Kenmochi and Oshita, 2007) have become immensely popular among enthusiasts of automat- ically created or vocally synthesized music. These technologies assist individuals with their musical creativity and thereby have promoted automatic song generation. To date, many individual and group musical amateurs have created songs and commercial activities. To satisfy the demand for composer-supportive automatic composition sys- tems and services, various systems, including Or- pheus (Fukayama et al., 2012), have been developed. Furthermore, as musical composition becomes eas- ier, there is a growing need for automatic lyrics gen- eration. However, lyrics generation has yet to be thor- oughly explored in the natural language processing field. While several works have tackled lyrics gener- ation based on lyric-specific characteristics, current methods are limited to local contexts, such as single sentences, which cannot capture the overall struc- ture of the generated lyrics (Barbieri et al., 2012; Ra- makrishnan A et al., 2009; Reddy and Knight, 2011; Wu et al., 2013; Greene et al., 2010). The contribution of our study is twofold: (1) To more comprehensively understand lyrics generation, we examine the characteristics or rules by which people identify Japanese lyrics writing and survey some previous methods. (2) Based on the survey, we construct three generation models as an initial step toward our aim. We focus on two types of informa- tion that are essential for lyrics creation: a language model for lyrics and topic transitions for passages. Experiments revealed that the language model is far more effective than models capturing topic tran- sitions. However, by capturing the topic transitions, we achieve consistency among the topics. Copyright 2014 by Kento Watanabe, Yuichiroh Matsubayashi, Kentaro Inui, and Masataka Goto 28th Pacific Asia …

Content truncated to 500 words