Two Huge Title and Keyword Generation Corpora of Research Articles

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6663–6671 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6663 Two Huge Title and Keyword Generation Corpora of Research Articles Erion C¸ ano, Ondˇrej Bojar Institute of Formal and Applied Linguistics, Charles University Prague, Czech Republic {cano, bojar}@ufal.mff.cuni.cz Abstract Recent developments in sequence-to-sequence learning with neural networks have considerably improved the quality of automatically generated text summaries and document keywords, stipulating the need for even bigger training corpora. Metadata of research articles are usually easy to find online and can be used to perform research on various tasks. In this paper, we introduce two huge datasets for text summarization (OAGSX) and keyword generation (OAGKX) research, containing 34 million and 23 mil- lion records, respectively. The data were retrieved from the Open Academic Graph which is a network of research profiles and publications. We carefully processed each record and also tried several extractive and abstractive methods of both tasks to create performance baselines for other researchers. We further illustrate the performance of those methods previewing their outputs. In the near future, we would like to apply topic modeling on the two sets to derive subsets of research articles from more specific disciplines. Keywords: text summarization, keyword generation, corpus construction, research articles, huge corpora 1. Introduction The ongoing tendency towards data-driven solutions for more and more tasks such as MT (Machine Translation), TS (Text Summarization), KG (Keyword Generation), and other tasks related to natural languages has created incen- tives for crawling the Web to produce large text corpora of various types. Furthermore, recent open data initiatives of governments1 and other institutions that encourage the pub- lication of more data on the Web have induced the same effect. From academia, there are initiatives such as Arnet- Miner (Tang et al., 2008) that try to integrate existing sci- entific data from various resources in common networks for easier retrieval and exploitation. Among the various types of texts published in the Web, the metadata of research arti- cles (e.g., titles, abstracts, keywords, etc.) are probably the easiest to find in large quantities, since they are usually not restricted. In fact, small corpora of research articles were used since the 90s to explore extractive KG (Witten et al., 1999; Turney, 1999) and TS (Mani and Bloedorn, 1997; Goldstein et al., 2000) techniques. Research on these tasks has switched from the extractive paradigm to the recent abstractive one that is based on sequence-to-sequence learning with neural networks. The respective models are usually data-hungry, emphasizing the need for larger corpora in both TS and KG tasks. In this paper, we first review the popular existing datasets used for TS and KG research. We later describe the processing steps we followed, starting from the retrieval of ArnetMiner OAG (Open Academic Graph) data collection to the creation of two novel and huge corpora: OAGSX2 and OAGKX.3 The first one contains more than 34 million records consisting of paper abstracts and titles. It …

Content truncated to 500 words