ELMD: An Automatically Generated Entity Linking Gold Standard Dataset in the Music Domain

ELMD: An Automatically Generated Entity Linking Gold Standard Dataset in the Music Domain Sergio Oramas*, Luis Espinosa-Anke†, Mohamed Sordo**, Horacio Saggion†, Xavier Serra* * Music Technology Group, † Tractament Autom`atic del Llenguatge Natural, ** Center for Computational Science †* DTIC - Universitat Pompeu Fabra, Barcelona, Spain ** University of Miami, Coral Gables, FL, USA {sergio.oramas, luis.espinosa, horacio.saggion, xavier.serra}@upf.edu, mohamed.sordo@gmail.com Abstract In this paper we present a gold standard dataset for Entity Linking (EL) in the Music Domain. It contains thousands of musical named entities such as Artist, Song or Record Label, which have been automatically annotated on a set of artist biographies coming from the Music website and social network LAST.FM. The annotation process relies on the analysis of the hyperlinks present in the source texts and in a voting-based algorithm for EL, which considers, for each entity mention in text, the degree of agreement across three state-of-the-art EL systems. Manual evaluation shows that EL Precision is at least 94%, and due to its tunable nature, it is possible to derive annotations favouring higher Precision or Recall, at will. We make available the annotated dataset along with evaluation data and the code. Keywords: Entity Linking, Music Information Retrieval, Language Resources 1. Introduction When we refer to the Music Domain in a Natural Language Processing (NLP) context we refer to Music product re- views such as Albums or Songs, Music-related biographies or even song lyrics. While these are valuable resources in NLP for tasks like Sentiment Analysis, Music Information Retrieval (MIR), however, has barely exploited the infor- mation and knowledge that can be extracted from textual data. This opens up a vibrant area of research where MIR tasks may benefit dramatically from mining textual data (Sordo et al., 2015). For example, Music Recommendation is increasingly leveraging NLP in order to go beyond shal- low Music item features and shared listening habits (Ostuni et al., 2015). However, the intersection between NLP and MIR is vastly underdeveloped, with few available evalua- tion frameworks, Music-specific corpora and ad-hoc tools. An interesting common ground for exploiting the intersec- tion between NLP and MIR is the automatic generation of Music Knowledge Bases (KBs). These may be further exploited for computing Artist Similarity (Oramas et al., 2015a) or Music Recommendation (Sordo et al., 2015). While it is not the purpose of this paper to provide an overview of methods for KB learning, let us mention that generally the approach starts by processing large corpora in order to obtain an annotation for Music entity mentions in text, either simply as Music types (e.g. tagging ‘Yellow Submarine’ as Song) or performing Entity Linking (EL), e.g. tagging ‘Yellow Submarine’ as dbpedia.org/page/Yellow Submarine (song). However, this is not a trivial task as mentions to Music entities show language and register idiosyncrasies (Tata and Di Eugenio, 2010; Gruhl et al., 2009), and therefore a certain degree of tailoring is required in order to account for them. Let us consider multiword Music entities, which usually are those who pose greatest challenges for EL. …

Content truncated to 500 words