GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2398–2409 Barcelona, Spain (Online), December 8-13, 2020 2398 GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation Zhijing Jin∗ Amazon Shanghai AI Lab zhijing.jin@connect.hku.hk Qipeng Guo∗† Fudan University qpguo16@fudan.edu.cn Xipeng Qiu Fudan University xpqiu@fudan.edu.cn Zheng Zhang Amazon Shanghai AI Lab zhaz@amazon.com Abstract Data collection for the knowledge graph-to-text generation is expensive. As a result, research on unsupervised models has emerged as an active field recently. However, most unsupervised mod- els have to use non-parallel versions of existing small supervised datasets, which largely constrain their potential. In this paper, we propose a large-scale, general-domain dataset, GenWiki. Our unsupervised dataset has 1.3M text and graph examples, respectively. With a human-annotated test set, we provide this new benchmark dataset for future research on unsupervised text genera- tion from knowledge graphs.1 1 Introduction Text generation with deep learning models is data hungry. For example, in Figure 1, to make a model learn how to verbalize knowledge graphs, researchers need to collect a large number of human-annotated text and graph pairs. However, good annotation is both expensive and difficult to get – annotators need to have a thorough understanding of hundreds of edge types in the knowledge graphs, as well as proper verbalization of the text, so that the written text can conform to the distribution of the desired text style. Moreover, for dataset curators, checking the quality of annotation is also non-trivial. For example, the WebNLG dataset goes through five updates to fix errors in the annotation over the past 3 years.2 These difficulties in dataset collection makes parallel data-to-text datasets small-sized, and even non- existent for low-resource domains. To make problems worse, data-to-text models are, in many cases, infeasible to transfer from one domain to another. For example, a text generation model that can pro- duce Wikipedia biography-like descriptions cannot be used to generate introductions of plants. This can happen even between domains with similar content but different text styles. For example, given the knowledge graph triple “(Obama, birthYear, 1961),” one domain verbalizes it as “Obama was born in 1961,” whereas another domain prefers “Obama (1961 – ) ...”. A model trained in the first domain can only generate the entities correctly but fail on all other words in the second domain. To overcome the lack of labelled data and difficulty in domain adaptation, unsupervised data-to-text generation has emerged as an active research field recently (Freitag and Roy, 2018; Schmitt et al., 2020; Guo et al., 2020). However, the progress of this line of research is slowed down due to the lack of large- scale unsupervised datasets. Notably, the curation of graph-to-text unsupervised datasets are non-trivial, as it requires (1) same content distribution between graphs and text, (2) text with high-accuracy entity annotation, (3) a much larger scale than the supervised datasets, and (4) a human-annotated test set. Un- fortunately, lacking such an unsupervised dataset, most unsupervised works have to artificially remove the pairing information between text and …

Content truncated to 500 words