Coherent Story Generation with Structured Knowledge

Proceedings of Recent Advances in Natural Language Processing, pages 681–690 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_074 681 Coherent Story Generation with Structured Knowledge Congda Ma1 Kotaro Funakoshi1 Kiyoaki Shirai2 Manabu Okumura1 1Tokyo Institute of Technology 2Japan Advanced Institute of Science and Technology {ma, funakoshi, oku}@lr.pi.titech.ac.jp kshirai@jaist.ac.jp Abstract The emergence of pre-trained language models has taken story generation, which is the task of automatically generating a comprehensible story from limited information, to a new stage. Although generated stories from the language models are fluent and grammatically correct, the lack of coherence affects their quality. We propose a knowledge-based multi-stage model that incorporates the schema, a kind of struc- tured knowledge, to guide coherent story gener- ation. Our framework includes a schema acqui- sition module, a plot generation module, and a surface realization module. In the schema acquisition module, high-relevant structured knowledge pieces are selected as a schema. In the plot generation module, a coherent plot plan is navigated by the schema. In the surface real- ization module, conditioned by the generated plot, a story is generated. Evaluations show that our methods can generate more comprehensible stories than strong baselines, especially with higher global coherence and less repetition. 1 Introduction Story generation is a challenging text generation task that uses a limited text, such as a first sentence, as a clue to form a story. The story must not only be grammatically correct but also have logic and consistency, which is known as coherence. Illogical sentences, unrelated entities, and generic phrases are problems that affect the coherence of generated stories (Holtzman et al., 2020). Incorporating knowledge into models has shown promising results in the story generation task; exter- nal information is provided to the models to capture more features not given in the context. For example, Guan et al. (2020) applied commonsense knowl- edge to their language model by using pre-training. As shown in Table 1, while their method alleviates the information deficit and produces more fluent sentences, there is still room for improving coher- ence. In particular, what is needed is a more so- Context tyron is headed home from basketball practice. Human written he hears a noise coming from outside of his car. he decides to pull over and see what ’s going on. tyron sees a metal spike in his tire. he calls his dad for help and waits. GPT-2 her son has to take his test. her son has to take the test. his son, his brother, and their mother were both in school. her son, his brother, and their mother were both disappointed. Knowledge pre-trained GPT-2 he is ready to play. he is also dressed in his big black dress. when he leaves for the game he is surprised. he is headed home. Table 1: Stories generated by human and neural mod- els (Guan et al., 2020). Generated stories are illogi- cal, describe unrelated entities, and contain unnatural repetitions. Low global coherence makes automatically generated stories difficult to understand in spite of their sentences being grammatically correct. phisticated …

Content truncated to 500 words