Overview of the 2016 ALTA Shared Task: Cross-KB Coreference

Overview of the 2016 ALTA Shared Task: Cross-KB Coreference Andrew Chisholm Hugo.ai Sydney, Australia achisholm@hugo.ai Ben Hachey Hugo.ai Sydney, Australia bhachey@hugo.ai Diego Moll´a Macquarie University Sydney, Australia diego.molla-aliod@mq.edu.au Abstract This paper presents an overview of the 7th ALTA shared task that ran in 2016. The task was to disambiguate endpoints by de- termining whether two URLs were refer- ring to the same entity. We present the motivation for the task, the description of the data and the results of the participating teams. 1 Introduction Entity endpoints are URLs which reliably disam- biguate named entity mentions on the web. For example, the URL en.wikipedia.org/wiki/ Barack_Obama may be used in reference to US president Barack Obama. Inlinks to this page are unlikely to refer to some other entity, so we should consider this a disambiguating endpoint. While Wikipedia has been used extensively for automated entity recognition and disambigua- tion, many other endpoints may exist for an en- tity on the web. For example, nytimes.com/ topic/person/barack-obama and twitter. com/BarackObama may be used equivalently. This style of systematic entity indexing is character- istic of social sources (e.g. facebook.com/* ), news aggregation endpoints (e.g. nytimes.com/ topic/person/* ) and organisation directories (e.g. gtlaw.com/People/* ). These resources present a valuable and largely untapped source of entity information, both in the content they host and semantic resources that may be extracted from inbound links. The ALTA 2016 Shared task addresses the prob- lem of Cross-KB coreference resolution. Given two candidate endpoint URLs, systems must de- termine whether they refer to the same underly- ing entity. This pairwise version of the task serves as an important precursor to the general problem of clustering web endpoints into coreferent sets. These clusters act as aggregation points for infor- mation about entities, and may used for entity cen- tric information extraction which is not limited by the coverage constraints of any single structured KB. This paper is structured as follows. Section 2 describes the shared task. Section 3 gives a short survey of related research. Section 4 describes the data set that was used. Section 5 details the evalua- tion process. Section 6 briefly describes the partic- ipating systems. Section 7 presents and discusses the results. Finally, Section 8 concludes this paper. 2 The 2016 ALTA Shared Task The 2016 ALTA Shared Task is the 7th of the shared tasks organised by the Australasian Lan- guage Technology Association (ALTA). Like the previous ALTA shared tasks, it is targeted at uni- versity students with programming experience. The general objective of these shared tasks is to in- troduce university students to the sort of problems that are the subject of active research in a field of natural language processing. There are no limitations on the size of the teams or the means that they can use to solve the prob- lem, as long as the processing is fully automatic — there should be no human intervention. There are two categories: a student category and an open category. • All the members of teams from …

Content truncated to 500 words