Analysis and Enhancement of Wikification for Microblogs with Context Expansion

Proceedings of COLING 2012: Technical Papers, pages 441–456, COLING 2012, Mumbai, December 2012. Analysis and Enhancement of Wikification for Microblogs with Context Expansion Taylor CASSIDY 1 Heng JI1 Lev RAT INOV 2 Arkaitz ZUBIAGA1 Hongzhao HUANG1 (1) DEPARTMENT OF LINGUISTICS & DEPARTMENT OF COMPUTER SCIENCE THE GRADUATE CENTER & QUEENS COLLEGE CITY UNIVERSITY OF NEW YORK New York, NY, USA 10016 (2) GOOGLE INC. New York, NY, USA 10011 taylorcassidy64@gmail.com, hengjicuny@gmail.com ABSTRACT Disambiguation to Wikipedia (D2W) is the task of linking mentions of concepts in text to their corresponding Wikipedia entries. Most previous work has focused on linking terms in formal texts (e.g. newswire) to Wikipedia. Linking terms in short informal texts (e.g. tweets) is difficult for systems and humans alike as they lack a rich disambiguation context. We first evaluate an existing Twitter dataset as well as the D2W task in general. We then test the effects of two tweet context expansion methods, based on tweet authorship and topic-based clustering, on a state-of-the-art D2W system and evaluate the results. TITLE AND ABSTRACT IN BASQUE Testuinguruaren Hedapenaren Analisia eta Hobekuntza Mikroblogak Wikifikatzeko Esanahia Wikipediarekiko Argitzea (D2W) deritzo testuetan aurkitutako kontzeptuen aipamenak Wikipedian dagozkien sarrerei lotzeari. Aurreko lan gehienek testu formalak (newswire, esate baterako) lotu dituzte Wikipediarekin. Testu informalak (tweet-ak, esate baterako) lotzea, ordea, zaila da bai sistementzat eta baita gizakiontzat ere, argipena erraztuko luketen testuingururik ez dutelako. Lehenik eta behin, Twitter-en gainean sortutako datu-sorta bat, eta D2W ataza bera ebaluatzen ditugu. Ondoren, egungo D2W sistema baten gainean testuingurua hedatzeko bi teknika aztertu eta ebaluatzen ditugu. Bi teknika hauek tweet-aren egilean eta gaikako multzokatze metodo batean oinarritzen dira. KEYWORDS: Disambiguation to Wikipedia (D2W), Twitter, disambiguation context. KEYWORDS IN BASQUE: Wikipediarekiko Argitzea (D2W), Twitter, argipen testuingurua. 441 1 Introduction Determining the correct meaning of each word in a natural language text is a prerequisite for proper understanding. Disambiguation to Wikipedia (D2W) (Mihalcea and Csomai, 2007), the process of linking each concept mention in a text to a concept referent (i.e. a Wikipedia page), is a framework that supports the word sense disambiguation (WSD) task1. For example, consider the sentence, "BP said Halliburton destroyed Gulf Spill evidence". A D2W system should break the text into concept mentions and return a unique identifier (an article title, in the case of Wikipedia) for each concept. The intended meaning of each concept mention can be inferred in terms of its surface form and its context. Mention Wikipedia title BP BP said Press Release Halliburton Halliburton destroyed Spoliation of Evidence Gulf Spill Deepwater Horizon oil spill evidence Evidence Table 1: Desired D2W output D2W may benefit both human end-users and natural language processing (NLP) systems. When a document is Wikified a reader can more easily grasp its contents as information about related topics is readily accessible2. From a system-to-system perspective, a disambiguated corpus has the meanings of many of its terms grounded in a structurally rich ontology, and indeed there is evidence that D2W output (Ratinov and Roth, 2012; Vitale et al., 2012) can improve NLP systems. …

Content truncated to 500 words