UVA: Language Modeling Techniques for Web People Search

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 468–471, Prague, June 2007. c⃝2007 Association for Computational Linguistics UVA: Language Modeling Techniques for Web People Search Krisztian Balog ISLA, University of Amsterdam kbalog@science.uva.nl Leif Azzopardi University of Glasgow leif@dcs.gla.ac.uk Maarten de Rijke ISLA, University of Amsterdam mdr@science.uva.nl Abstract In this paper we describe our participation in the SemEval 2007 Web People Search task. Our main aim in participating was to adapt language modeling tools for the task, and to experiment with various document represen- tations. Our main finding is that single pass clustering, using title, snippet and body to represent documents, is the most effective setting. 1 Introduction The goal of the Web People Search task at SemEval 2007 was to disambiguate person names in a web searching scenario (Artiles et al., 2007). Participants were presented with the following setting: given a list of documents retrieved from a web search engine using a person’s name as a query, group documents that refer to the same individual. Our aim with the participation was to adapt lan- guage modeling techniques to this task. To this end, we employed two methods: single pass cluster- ing (SPC) and probabilistic latent semantic analysis (PLSA). Our main finding is that the former leads to high purity, while the latter leads to high inverse pu- rity scores. Furthermore, we experimented with var- ious document representations, based on the snip- pets and body text. Highest overall performance was achieved with the combination of both. The remainder of the paper is organized as fol- lows. In Section 2 we present the two approaches we employed for clustering documents. Next, in Sec- tion 3 we discuss document representation and pre- processing. Section 4 reports on our experiments. We conclude in Section 5. 2 Modeling 2.1 Single Pass Clustering We employed single pass clustering (Hill., 1968) to automatically assign pages to clusters, where we as- sume that each cluster is a set of pages related to one particular sense of the person. The process for assignment was performed as fol- lows: The first document was taken and assigned to the first cluster. Then each subsequent document was compared against each cluster with a similarity measure based on the log odds ratio (initially, there was only the initial one created). A document was assigned to the most likely cluster, as long as the similarity score was higher than a threshold α; oth- erwise, the document was assigned to a new cluster, unless the maximum number of desired clusters η had been reached; in that case the document was as- signed to the last cluster (i.e., the left overs). The similarity measure we employed was the log odds ratio to decide whether the document was more likely to be generated from that cluster or not. This approach follows Kalt (1996)’s work on document classification using the document likelihood by rep- resenting the cluster as a multinomial term distribu- tion (i.e., a cluster language model) and predicting the probability of a document …

Content truncated to 500 words