Enhancing Topic-to-Essay Generation with External Commonsense Knowledge

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2002–2012 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2002 Enhancing Topic-to-Essay Generation with External Commonsense Knowledge Pengcheng Yang1,2∗, Lei Li3∗, Fuli Luo2, Tianyu Liu2, Xu Sun1,2 1Deep Learning Lab, Beijing Institute of Big Data Research, Peking University 2MOE Key Lab of Computational Linguistics, School of EECS, Peking University 3School of Computer Science and Technology, Xidian University {yang pc, luofuli, tianyu0421, xusun}@pku.edu.cn, tobiaslee@foxmail.com Abstract Automatic topic-to-essay generation is a chal- lenging task since it requires generating novel, diverse, and topic-consistent paragraph-level text with a set of topics as input. Previous work tends to perform essay generation based solely on the given topics while ignoring mas- sive commonsense knowledge. However, this commonsense knowledge provides additional background information, which can help to generate essays that are more novel and di- verse. Towards filling this gap, we propose to integrate commonsense from the external knowledge base into the generator through dy- namic memory mechanism. Besides, the ad- versarial training based on a multi-label dis- criminator is employed to further improve topic-consistency. We also develop a series of automatic evaluation metrics to comprehen- sively assess the quality of the generated es- say. Experiments show that with external com- monsense knowledge and adversarial training, the generated essays are more novel, diverse, and topic-consistent than existing methods in terms of both automatic and human evaluation. 1 Introduction Automatic topic-to-essay generation (TEG) aims at generating novel, diverse, and topic-consistent paragraph-level text given a set of topics. It not only has plenty of practical applications, e.g., benefiting intelligent education or assisting in keyword-based news writing (Lepp¨anen et al., 2017), but also serves as an ideal testbed for con- trollable text generation (Wang and Wan, 2018). Despite its wide applications described above, the progress in the TEG task lags behind other generation tasks such as machine translation (Bah- danau et al., 2014) or text summarization (Rush et al., 2015). Feng et al. (2018) are the first to pro- pose the TEG task and they utilize coverage vector ∗Equal Contribution. ������ ������ ���� ������������� ������ � ������ ����� ���������� ������ � ������ ������� ����������� ������ � ������ Figure 1: Toy illustration of the information volume on three different text generation tasks, which shows that the source information is extremely insufficient com- pared to the target output on the TEG task. to incorporate topic information for essay genera- tion. However, the model performance is not sat- isfactory. The generated essays not only lack nov- elty and diversity, but also suffer from poor topic- consistency. One main reason is that the source information is extremely insufficient compared to the target output on the TEG task. We summarize the comparison of information flow between the TEG task and other generation tasks in Figure 1. In machine translation and text summarization, the source input provides enough semantic informa- tion to generate the desired target text. However, the TEG task aims to generate paragraph-level text based solely …

Content truncated to 500 words