GameWikiSum: a Novel Large Multi-Document Summarization Dataset

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6645–6650 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6645 GameWikiSum: a Novel Large Multi-Document Summarization Dataset Diego Antognini, Boi Faltings Artificial Intelligence Laboratory ´Ecole Polytechnique F´ed´erale de Lausanne Lausanne, Switzerland firstname.lastname@epfl.ch Abstract Today’s research progress in the field of multi-document summarization is obstructed by the small number of available datasets. Since the acquisition of reference summaries is costly, existing datasets contain only hundreds of samples at most, resulting in heavy reliance on hand-crafted features or necessitating additional, manually annotated data. The lack of large corpora therefore hinders the development of sophisticated models. Additionally, most publicly available multi-document summarization corpora are in the news domain, and no analogous dataset exists in the video game domain. In this paper, we propose GameWikiSum, a new domain-specific dataset for multi-document summarization, which is one hundred times larger than commonly used datasets, and in another domain than news. Input documents consist of long professional video game reviews as well as references of their gameplay sections in Wikipedia pages. We analyze the proposed dataset and show that both abstractive and extractive models can be trained on it. We release GameWikiSum for further research: https://github.com/Diego999/GameWikiSum. Keywords: multi-document summarization, video games, professional reviews, wikipedia 1. Introduction With the growth of the internet in the last decades, users are faced with an increasing amount of information and have to find ways to summarize it. However, producing summaries in a multi-document setting is a challenging task; the lan- guage used to display the same information in a sentence can vary significantly, making it difficult for summariza- tion models to capture. Thus large corpora are needed to develop efficient models. There exist two types of sum- marization: extractive and abstractive. Extractive summa- rization outputs summaries in two steps, namely via sen- tence ranking, where an importance score is assigned to each sentence, and via the subsequent sentence selection, where the most appropriate sentence is chosen. In abstrac- tive summarization, summaries are generated word by word auto-regressively, using sequence-to-sequence or language models. Given the complexity of multi-document summa- rization and the lack of datasets, most researchers use ex- tractive summarization and rely on hand-crafted features or additional annotated data, both needing human expertise. To our knowledge, Liu et al. (2018) is the only work that has proposed a large dataset for multi-document summa- rization. By considering Wikipedia entries as a collection of summaries on various topics given by their title (e.g., Machine Learning, Stephen King), they create a dataset of significant size, where the lead section of an article is de- fined as the reference summary and input documents are a mixture of pages obtained from the article’s reference sec- tion and a search engine. While this approach benefits from the large number of Wikipedia articles, in many cases, ar- ticles contain only a few references that tend to be of the desired high quality, and most input documents end up be- ing …

Content truncated to 500 words