A Multilingual Test Collection for the Semantic Search of Entity Categories

A Multilingual Test Collection for the Semantic Search of Entity Categories Juliano Efson Sales†, Siamak Barzegar§,Wellington Franco‡, Bernhard Bermeitinger†, Tiago Cunha¶, Brian Davis§∓, Andr´e Freitas†δ and Siegfried Handschuh† †Department of Computer Science and Mathematics - University of Passau, Germany §Insight Centre for Data Analytics - National University of Ireland, Galway ‡Federal University of Cear´a, Campus Crate´us, Brazil ¶University of the International Integration of the Afro-Brazilian Lusophony, Brazil ∓Department of Computer Science - Maynooth University, Ireland δSchool of Computer Science - The University of Manchester, United Kingdom {juliano-sales, bernhard.bermeitinger, siegfried.handschuh}@uni-passau.de siamak.barzegar@insight-centre.org, wellington@crateus.ufc.br tiagotmc@unilab.edu.br, brian.davis@mu.ie, andre.freitas@manchester.ac.uk Abstract Humans naturally organise and classify the world into sets and categories. These categories expressed in natural language are present in all data artefacts from structured to unstructured data and play a fundamental role as tags, dataset predicates or ontology attributes. A better understanding of the category syntactic structure and how to match them semantically is a fundamental problem in the computational linguistics domain. Despite the high popularity of entity search, entity categories have not been receiving equivalent attention. This paper aims to present the task of semantic search of entity categories by defining, developing and making publicly available a multilingual test collection comprehending English, Portuguese and German. The test collections were designed to meet the demands of the entity search community in providing more representative and semantically complex query sets. In addition, we also provide comparative baselines and a brief analysis of the results. Keywords: semantic search, category search, paraphrasing, entity search, multilingual test collection Lang. Category EN American architectural styles EN French female artistic gymnasts EN Brazilian aerospace conglomerate PT Alfabetos derivados do latino PT Ordens e congregac¸˜oes religiosas cat´olicas PT Primeiros-ministros de Portugal DE Ortsteil von Anr¨ochte DE Soziale Bewegung als Thema DE Teilnehmer an Der Bachelor Table 1: Examples of categories in English (EN), Por- tuguese (PT) and German (DE). 1. Introduction Well-defined tasks and test collections are fundamental re- sources to allow reproducibility and comparability in in- formation retrieval in general, and entity search in particu- lar (Jones, 1981). As entities are the main target of a vast amount of search queries on the Web (Pound et al., 2010), over the years, a mature research community has emerged around the entity search domain. Despite the considerable number of challenges and cam- paigns released in the field (Elbedweihy et al., 2015), in re- cent years, the entity search community has encouraged the development of new tasks. According to Balog and Neu- mayer (2013) three action points were signed as priorities: 1. (i) getting more representative information needs and favouring long queries over short ones; 2. (ii) limiting search to a smaller, fixed set of entity types (as opposed to arbitrary types of entities); and 3. (iii) using test collections that integrate both structured and unstructured information about entities. In order to suit at least two of these priorities, this paper aims at presenting the task of semantic search of entity cat- egories by defining, developing and making publicly avail- able test collections, …

Content truncated to 500 words