Composing Ci with Reinforced Non-autoregressive Text Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7219 - 7229 December 7-11, 2022 ©2022 Association for Computational Linguistics Composing Ci with Reinforced Non-autoregressive Text Generation Yan Song University of Science and Technology of China clksong@gmail.com Abstract Composing Ci (also widely known as Song Ci), a special type of classical Chinese po- etry, requires to follow particular format once their tune patterns are given. To automatically generate a well-formed Ci, text generation systems should strictly take into account pre- defined rigid formats (e.g., length and rhyme). Yet, most existing approaches regard Ci gener- ation as a conventional sequence-to-sequence task and use autoregressive models, while it is challenging for such models to properly han- dle the constraints (according to tune patterns) of Ci during the generation process. Moreover, consider that with the format prepared, Ci gen- eration can be operated by an efficient syn- chronous process, where autoregressive mod- els are limited in doing so since they fol- low the character-by-character generation pro- tocol. Therefore, in this paper, we propose to compose Ci through a non-autoregressive ap- proach, which not only ensure that the gener- ation process accommodates tune patterns by controlling the rhythm and essential meaning of each sentence, but also allow the model to perform synchronous generation. In addi- tion, we further improve our approach by ap- plying reinforcement learning to the genera- tion process with the rigid constraints of Ci as well as the diversity in content serving as re- wards, so as to further maintain the format and content requirement. Experiments on a col- lected Ci dataset confirm that our proposed ap- proach outperforms strong baselines and previ- ous studies in terms of both automatic evalua- tion metrics and human judgements.1 1 Introduction Ci is a special type of Chinese poetry and different from general poems, where their shape (including lengths and tunes) is determined by tune patterns (i.e., 词牌, also known as brand names) defining 1Related code and resources of the paper are available at https://github.com/synlp/CiGen. Figure 1: The first half of an example Ci following the tune pattern “念奴娇”3 (Charm of a Singer), where all essential words in this Ci are highlighted in blue. Trans- lations for each clause are provided for reference.4 the particular syllable and rhythm schemes that a Ci should stick to.2 Over the past one thousand years, composing Ci has long been an interesting game and continued to present days for Chinese people to demonstrate one’s literary and artistic accomplish- ments. Given that Ci composition is challenging, to perform this task with artificial intelligence is meaningful since it is a good test for controlled nat- ural language generation under specific constraints (i.e., the length and rhyme of Ci should follow the constraints given by the tune pattern). 2Originally, Ci emerged during the Tang Dynasty (618-907 A.D.), in response to the popularity of foreign musical tunes imported from the Inner Asia, and started its prevailing and was written all through the age in the Song Dynasty (960-1279 …

Content truncated to 500 words