Constrained Language Models for Interactive Poem Generation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3519–3529 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3519 Constrained Language Models for Interactive Poem Generation Andrei Popescu-Belis,1,2 `Alex R. Atrio,1,2 Valentin Minder,1 Aris Xanthos,3 Gabriel Luthier,1 Simon Mattei,1 Antonio Rodriguez3 1 HEIG-VD / HES-SO 2EPFL 3 Universit´e de Lausanne CH-1401 Yverdon-les-Bains CH-1015 Lausanne CH-1015 Lausanne Switzerland Switzerland Switzerland firstname.lastname@heig-vd.ch firstname.lastname@unil.ch Abstract This paper describes a system for interactive poem generation, which combines neural language models (LMs) for poem generation with explicit constraints that can be set by users on form, topic, emotion, and rhyming scheme. LMs cannot learn such constraints from the data, which is scarce with respect to their needs even for a well-resourced language such as French. We propose a method to generate verses and stanzas by combining LMs with rule-based algorithms, and compare several approaches for adjusting the words of a poem to a desired combination of topics or emotions. An approach to automatic rhyme setting using a phonetic dictionary is proposed as well. Our system has been demonstrated at public events, and log analysis shows that users found it engaging. Keywords: language models, text generation, poem generation, interactive systems, digital humanities. 1. Introduction Neural language models (LMs), which are probabil- ity distributions over sequences of words or characters, have recently enabled the generation of fluent sentences and texts. However, controlling such models in order to generate specific text structures remains difficult. We propose solutions for constrained language modeling with external features such as form, topics, emotions and rhymes, and integrate them into a system for inter- active poem generation, which enables the joint writ- ing of a poem by a human and a computer. Our CR- PO system1 leverages neural LMs to generate the ini- tial draft of a poem in a form selected by the user, and then enters a cycle of joint human-computer co-editing, in which the user can set various parameters, according to which the computer modifies the current creation. Manual editing is also possible at any stage. The CR- PO system has been demonstrated at a public exhibition and other events at our institutions. The goal of this paper is to present the poem gener- ation system and our approaches to constrain neural LMs according to different dimensions that character- ize poetry. We first present a functional view of CR-PO in Section 2, along with the setting and hardware. We introduce the third party software and the data used to build LMs in Section 3. Our solutions for constrained text generation are presented in Section 4, regarding the form of the poem (stanzas and lines), the adaptation of the poem to given topics or emotions, and the rhyming scheme. We present in Section 5 several actions we took towards the evaluation of CR-PO, and discuss re- lated work in Section 6. 1CR-PO stands for Cr´eation Po´etique Assist´ee par Ordi- nateur, i.e. computer assisted poetical creation in French. 2. Overview of the System …

Content truncated to 500 words