Collaborative Ranking: A Case Study on Entity Linking

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 771–781, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Collaborative Ranking: A Case Study on Entity Linking Zheng Chen Computer Science Department Graduate Center City University of New York zchen1@gc.cuny.edu Heng Ji Computer Science Department Queens College and Graduate Center City University of New York hengji@cs.qc.cuny.edu Abstract In this paper, we present a new ranking scheme, collaborative ranking (CR). In con- trast to traditional non-collaborative ranking scheme which solely relies on the strengths of isolated queries and one stand-alone rank- ing algorithm, the new scheme integrates the strengths from multiple collaborators of a query and the strengths from multiple ranking algorithms. We elaborate three specific forms of collaborative ranking, namely, micro col- laborative ranking (MiCR), macro collabora- tive ranking (MaCR) and micro-macro collab- orative ranking (MiMaCR). Experiments on entity linking task show that our proposed scheme is indeed effective and promising. 1 Introduction Many natural language processing tasks can be for- malized as a ranking problem, namely to rank a collection of candidate “objects” with respect to a “query”. For example, intensive studies were de- voted to parsing in which multiple possible pars- ing trees or forests are ranked with respect to a sen- tence (Collins, 2000; Charniak and Johnson, 2005; Huang, 2008), machine translation in which multi- ple translation hypotheses are ranked with respect to a source sentence (Och, 2002; Shen et al., 2005), anaphora resolution in which multiple antecedents are ranked with respect to an anaphora (Yang et al., 2008), and question answering in which mul- tiple possible answers are ranked with respect to a question (Ravichandran et al., 2003). Previous stud- ies mainly focused on improving the ranking perfor- mance using one stand-alone learning algorithm on isolated queries. Although a wide range of learning algorithms (un- supervised, supervised or semi-supervised) is avail- able, each with its strengths and weaknesses, there is not a learning algorithm that can work best on all types of data. In such a situation, it would be desirable to build a “collaborative” model by integrating multiple models. Such an idea forms the basis of ensemble methodology and it is well- known that ensemble methods (e.g., bagging, boost- ing) can improve the performance of many prob- lems, in which classification is the most intensively studied (Rokach, 2009). The other situation is re- lated with isolated queries handled by learning al- gorithms. The single query may not be formulated with the best terms or the query itself may not con- tain comprehensive information required for a high- performance ranking algorithm. Therefore, tech- niques of query expansion or query reformulation can be introduced and previous research has shown the effectiveness of those techniques in such applica- tions as information retrieval and question answer- ing (Manning et al., 2008; Riezler et al., 2007). Nevertheless, previous research normally considers query reformulation as a new query for the ranking system, it would be more desirable to form a larger- scale “collaborative” group for …

Content truncated to 500 words