TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
30789Authors
nan
Abstract
AbstractLyric-to-melody generation is an important task in automatic songwriting. Previous lyric-to-melody generation systems usually adopt end-to-end models that directly generate melodies from lyrics, which suffer from several issues: 1) lack of paired lyric-melody training data; 2) lack of control on generated melodies. In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e.g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i.e., the system consists of a lyric-to-template module and a template-to-melody module). TeleMelody has two advantages. First, it is data efficient. The template-to-melody module is trained in a self-supervised way (i.e., the source template is extracted from the target melody) that does not need any lyric-melody paired data. The lyric-to-template module is made up of some rules and a lyric-to-rhythm model, which is trained with paired lyric-rhythm data that is easier to obtain than paired lyric-melody data. Second, it is controllable. The design of the template ensures that the generated melodies can be controlled by adjusting the musical elements in the template. Both subjective and objective experimental evaluations demonstrate that TeleMelody generates melodies with higher quality, better controllability, and less requirement on paired lyric-melody data than previous generation systems.
Keywords
Full Content
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5426 - 5437 December 7-11, 2022 ©2022 Association for Computational Linguistics 5426 5427 Twin-kle twin-kle lit -tle star, how I won-der what you are. I (C) IV (F) V (G) IV (F) V (G) I (C) C C C C F F G F F F F G G C 0 1 2 3 0 1 2 0 1 2 3 0 1 2 No No No No No No Half No No No No No No Chord Rhythm pattern Cadence Authentic C major (Tonality) 5428 Melody Lyric-to-Template Module Template-to-Melody Module Template Lyric Tonality Lyric Lyric-to-Rhythm Model Chord Progression Rhythm Pattern Cadence Punctuation-to-Cadence Mapping Template i�� Cadence Tonality Chord Rhythm Pattern ··· ··· Template ··· i�� �i+1�� Bar Position Duration Bar Melody ··· Position Pitch 5429 5430 5431 3-Cadence 3-Chord 2-Cadence 2-Chord 1-Rhythm 1-Cadence 1-Chord 0-Rhythm 0-Cadence 0-Chord Toanlity 3-Rhythm 2-Rhythm 0-Bar 0-Pitch 0-Position 0-Duration 1-Bar 1-Pitch 1-Position 1-Duration 2-Bar 2-Pitch 2-Position 2-Duration 3-Bar 3-Pitch 3-Position 3-Duration 1.0 0.8 0.6 0.4 0.2 0.0 1.0 3-Cadence 3-Chord 2-Cadence 2-Chord 1-Rhythm 1-Cadence 1-Chord 0-Rhythm 0-Cadence 0-Chord Toanlity 3-Rhythm 2-Rhythm 0-Bar 0-Pitch 0-Position 0-Duration 1-Bar 1-Pitch 1-Position 1-Duration 2-Bar 2-Pitch 2-Position 2-Duration 3-Bar 3-Pitch 3-Position 3-Duration 0.0 0.8 0.6 0.4 0.2 5432 � � days like this iwanttodrive a-way, days likethisiwanttodrive a-way. vi (Am) IV (F) V (G) I (C) . � � days like this want todrive a-way, days like this want i todrive -way a i VI (F) iv (Dm) v (Em) i (Am) days like this i want to drive a -way, days like this i want to drive a -way. V (G) I (C) V (G) I (C) 5433 5434 5435 5436 5437
Labeled Text
AbstractLyric-to-melody generation is an important task in automatic songwriting. Previous lyric-to-melody generation systems usually adopt end-to-end models that directly generate melodies from lyrics, which suffer from several issues: 1) lack of paired lyric-melody training data; 2) lack of control on generated melodies. In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e.g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i.e., the system consists of a lyric-to-template module and a template-to-melody module). TeleMelody has two advantages. First, it is data efficient. The template-to-melody module is trained in a self-supervised way (i.e., the source template is extracted from the target melody) that does not need any lyric-melody paired data. The lyric-to-template module is made up of some rules and a lyric-to-rhythm model, which is trained with paired lyric-rhythm data that is easier to obtain than paired lyric-melody data. Second, it is controllable. The design of the template ensures that the generated melodies can be controlled by adjusting the musical elements in the template. Both subjective and objective experimental evaluations demonstrate that TeleMelody generates melodies with higher quality, better controllability, and less requirement on paired lyric-melody data than previous generation systems.
Publication Details
- Paper ID:
- 30789
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan