WN-Salience: A Corpus of News Articles with Entity Salience Annotations

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2095–2102 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2095 WN-Salience: A Corpus of News Articles with Entity Salience Annotations Chuan Wu1,2, Evangelos Kanoulas2,3, Maarten de Rijke2, Wei Lu1 1School of Information Management, Wuhan University, Wuhan, China 2Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands 3Amsterdam Business School, University of Amsterdam, Amsterdam, The Netherlands wuchuan114@gmail.com, e.kanoulas@uva.nl, derijke@uva.nl, weilu@whu.edu.cn Abstract Entities can be found in various text genres, ranging from tweets and web pages to user queries submitted to web search engines. Existing research either considers all entities in the text equally important, or heuristics are used to measure their salience. We believe that a key reason for the relatively limited work on entity salience is the lack of appropriate datasets. To support research on entity salience, we present a new dataset, the WikiNews Salience dataset (WN-Salience), which can be used to benchmark tasks such as entity salience detection and salient entity linking. WN-Salience is built on top of Wikinews, a Wikimedia project whose mission is to present reliable news articles. Entities in Wikinews articles are identified by the authors of the articles and are linked to Wikinews categories when they are salient or to Wikipedia pages otherwise. The dataset is built automatically, and consists of approximately 7,000 news articles, and 90,000 in-text entity annotations. We compare the WN-Salience dataset against existing datasets on the task and analyze their differences. Furthermore, we conduct experiments on entity salience detection; the results demonstrate that WN-Salience is a challenging testbed that is complementary to existing ones. Keywords: entity salience, salience detection, Wikinews 1. Introduction Text modeling has traditionally made no distinction be- tween different terms in the text. Examples include bag of words representation, language models, and term weight- ing methods. Research on knowledge extraction and text semantics has shifted some of the attention towards utter- ances that represent real world entities, while recent work on entity linking (Shen et al., 2015) has made it possible to take entities into consideration in various downstream applications, such as information retrieval (Dalton et al., 2014; Raviv et al., 2016). Various corpora annotated with entities have been built for entity related research, such as FACC1 (Gabrilovich et al., 2013). However, these corpora make no distinction be- tween salient and non-salient entities, despite the fact that only few entities are central to a document. For instance, in the Web domain, fewer than 5% of the entities on a web page are salient to the page (Gamon et al., 2013). Many ex- isting publications have recognized the importance of un- derstanding entity salience (Fetahu et al., 2015; Tran et al., 2015; Xiong et al., 2018; Ponza et al., 2019; Wu et al., 2019). For example, automatically suggesting news pages for populating Wikipedia requires determining whether a news article should be referenced by an entity, consider- ing several aspects of the article, including entity salience, relative authority, and novelty of the article …

Content truncated to 500 words