Exploiting Latent Semantic Relations in Highly Linked Hypertext for Information Retrieval in Wikis

International Conference RANLP 2009 - Borovets, Bulgaria, pages 241–245 Exploiting Latent Semantic Relations in Highly Linked Hypertext for Information Retrieval in Wikis Tristan Miller∗ InQuira UK Ltd 50–52 Paul Street London EC2A 4LB, United Kingdom tmiller@inquira.com Bertin Klein∗ 7P Consulting GmbH Balcke-D¨urr-Allee 40882 Ratingen, Germany bertin.klein@7p-group.com Elisabeth Wolf∗ Ubiquitous Knowledge Processing Lab Technische Universit¨at Darmstadt, Germany wolf @tk.informatik.tu-darmstadt.de Abstract Good hypertext writing style mandates that link texts clearly indicate the nature of the link tar- get. While this guideline is routinely ignored in HTML, the lightweight markup languages used by wikis encourage or even force hypertext au- thors to use semantically appropriate link texts. This property of wiki hypertext makes it an ideal candidate for processing with latent se- mantic analysis, a factor analysis technique for finding latent transitive relations among natural- language documents. In this study, we design, implement, and test an LSA-based information retrieval system for wikis. Instead of a full-text index, our system indexes only link texts and document titles. Nevertheless, its precision ex- ceeds that of a popular full-text search engine, and is comparable to that of PageRank-based systems such as Google. Keywords latent semantic analysis, LSA, latent semantic indexing, LSI, in- formation retrieval, search engines, Wikipedia, wikis, hypertext, hyperlinks, large corpora 1 Introduction 1.1 Hypertext information retrieval Traditional information retrieval systems are based on a flat, vector-space document model designed for searching plain-text corpora. Such systems are defi- cient when it comes to hypertext, however, as they fail to account for the topological structure of the cor- pus. This led to the development of the first hypertext document scoring algorithms, PageRank [22, 7, 5] and HITS [18], which score documents on the basis of the source and number of incoming and outgoing links. ∗The research described in this paper was carried out while the authors were at the Knowledge Management Research Group of the German Research Center for Artificial Intelligence (DFKI GmbH) in Kaiserslautern, Germany. These scores are computed as a fixed point of a lin- ear equation; later scoring studies [11, 6] investigated statistical or hybrid approaches. In real-world IR applications, such as Google, the document scoring algorithm is combined with tradi- tional vector-space IR techniques, plus metadata such as the document titles and link texts. The use of this metadata is predicated on the assumption that it is semantically related to the document under consider- ation. That is, the document title should reflect the document content, and likewise the text of links should accurately describe the target document—practices which are mandated by authoritative hypertext style guides [24, 10, 9, 4]. However, these guidelines are routinely ignored in HTML Web pages, for two reasons. First, few HTML authoring tools remind, encourage, or compel the writer to adhere to them. For example, in a random sample of documents from a Web corpus [15], we found that 6% of document titles were either missing, empty, a copy of the filename, or some default string such as “Untitled Document”; a further 2 to 3% were so vague …

Content truncated to 500 words