MAKED: Multi-lingual Automatic Keyword Extraction Dataset

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6170–6179 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6170 MAKED: Multi-lingual Automatic Keyword Extraction Dataset Yash Verma1, Anubhav Jangra2, Sriparna Saha2, Adam Jatowt3, Dwaipayan Roy1 1Indian Institute of Science Education and Research Kolkata, India 2Indian Institute of Technology Patna, India 3University of Innsbruck, Austria {yashv7523, anubhav0603, sriparna.saha, jatowt, dwaipayan.roy}@gmail.com Abstract Keyword extraction is an integral task for many downstream problems like clustering, recommendation, search and classification. Development and evaluation of keyword extraction techniques require an exhaustive dataset; however, currently, the community lacks large-scale multi-lingual datasets. In this paper, we present MAKED, a large-scale multi-lingual keyword extraction dataset comprising of 540K+ news articles from British Broadcasting Corporation News (BBC News) spanning 20 languages. It is the first keyword extraction dataset for 11 of these 20 languages. The quality of the dataset is examined by experimentation with several baselines. We believe that the proposed dataset will help advance the field of automatic keyword extraction given its size, diversity in terms of languages used, topics covered and time periods as well as its focus on under-studied languages. Keywords: keyword extraction, text processing, multi-lingual dataset. 1. Introduction The amount of information being uploaded on internet each day is increasing over time, making it harder to filter out relevant information tailored to an individual. The infor- mation over the web spans multiple languages, making the task of multi-lingual keyword extraction quite useful. At the same time the diversity of these languages makes the task also challenging. For example, Le (2015) explains that if we consider Japanese (ja) for the task of keyphrase ex- traction in the legal context, the candidates of interest are words, chunks, and clauses. However, for the same task in English (en) language, utilizing similar structural informa- tion will lead to a less optimal solution or may not improve the performance since chunks can lead to a noisy output for keyword extraction. Extracting representative words or phrases from a docu- ment is essential to quickly summarize and understand the topics covered within the text. Keyphrases are word(s) that convey the essence or the main topics of the document, and their extraction is essential for supporting or enhanc- ing many downstream tasks in the domain of information retrieval (Medelyan and Witten, 2008), text representation and summarization (Litvak and Last, 2008), document clus- tering (Han et al., 2007) and so on. Although many scien- tific articles and news articles are already associated with keywords, most documents are not; hence the development of dedicated models for the keyword extraction task is nec- essary. A large and diverse annotated corpus will then mo- tivate and foster the development of supervised techniques and evaluation of various keyword extraction methods. News articles are one of the most consumed and readily available types of documents and have been explored in many state-of-the-art transformer-based models (Zhang et al., 2020; Lewis et al., 2019; Raffel et al., 2019) for tasks like summarization, …

Content truncated to 500 words