Building the Directed Semantic Graph for Coherent Long Text Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2563–2572 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2563 Building the Directed Semantic Graph for Coherent Long Text Generation Ziao Wang , Xiaofeng Zhang∗, Hongwei Du Harbin Institute of Technology, Shenzhen, China 20b351002@stu.hit.edu.cn, {zhangxiaofeng, hwdu}@hit.edu.cn Abstract Generating long text conditionally depending on the short input text has recently attracted more and more research efforts. Most exist- ing approaches focus more on introducing ex- tra knowledge to supplement the short input text, but ignore the coherence issue of the gen- erated texts. To address aforementioned re- search issue, this paper proposes a novel two- stage approach to generate coherent long text. Particularly, we first build a document-level path for each output text with each sentence embedding as its node, and a revised self- organising map (SOM) is proposed to cluster similar nodes of a family of document-level paths to construct the directed semantic graph. Then, three subgraph alignment methods are proposed to extract the maximum matching paths or subgraphs. These directed subgraphs are considered to well preserve extra but rele- vant content to the short input text, and then they are decoded by the employed pre-trained model to generate coherent long text. Exten- sive experiments have been performed on three real-world datasets, and the promising results demonstrate that the proposed approach is su- perior to the state-of-the-art approaches w.r.t. a number of evaluation criteria. 1 Introduction Recently, in the domain of natural language gener- ation (NLG), the conditional long text generation (Shen et al., 2019) has been investigated with flour- ishing results (Wang et al., 2018; Tan et al., 2020; Hu et al., 2021; Ren et al., 2020). Essentially, this task aims to generate text in line with the main semantic meanings of the input text; and mean- while, it requires that the length of the generated text is significantly longer than that of the short input text. The key to this challenging task is how to generate extra but relevant content that is well positioned to express the semantic meanings of the *Corresponding author. input text. To generate extra content, the pointer- generator network (See et al., 2017) is proposed to copy terms from an external corpus. Alternatively, various deep generative models (Bowman et al., 2016) have been proposed to generate previously unseen content by sampling from the learnt distri- bution of the latent variables and there also exist some other research attempts (Hu et al., 2021). Although the aforementioned issues were par- tially alleviated by existing approaches, they obvi- ously ignored the logic coherency issue between the generated texts. To simplify the problem, here- inafter, the logic coherency refers to the generated paragraphs, with each generated to represent the same semantic meanings, and to have the correct se- quence order. Apparently, it is a non-trivial task to generate extra content having appropriate sequence order. First, it still remains an open challenge in the literature to generate extra content given the short input text, especially for …

Content truncated to 500 words