Credibility Improves Topical Blog Post Retrieval

Proceedings of ACL-08: HLT, pages 923–931, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Credibility Improves Topical Blog Post Retrieval Wouter Weerkamp ISLA, University of Amsterdam weerkamp@science.uva.nl Maarten de Rijke ISLA, University of Amsterdam mdr@science.uva.nl Abstract Topical blog post retrieval is the task of rank- ing blog posts with respect to their relevance for a given topic. To improve topical blog post retrieval we incorporate textual credibility in- dicators in the retrieval process. We consider two groups of indicators: post level (deter- mined using information about individual blog posts only) and blog level (determined using information from the underlying blogs). We describe how to estimate these indicators and how to integrate them into a retrieval approach based on language models. Experiments on the TREC Blog track test set show that both groups of credibility indicators significantly improve retrieval effectiveness; the best per- formance is achieved when combining them. 1 Introduction The growing amount of user generated content avail- able online creates new challenges for the informa- tion retrieval (IR) community, in terms of search and analysis tasks for this type of content. The introduc- tion of a blog retrieval track at TREC (Ounis et al., 2007) has created a platform where we can begin to address these challenges. During the 2006 edition of the track, two types of blog post retrieval were considered: topical (retrieve posts about a topic) and opinionated (retrieve opinionated posts about a topic). Here, we consider the former task. Blogs and blog posts offer unique features that may be exploited for retrieval purposes. E.g., Mishne (2007b) incorporates time in a blog post retrieval model to account for the fact that many blog queries and posts are a response to a news event (Mishne and de Rijke, 2006). Data quality is an issue with blogs—the quality of posts ranges from low to edited news article-like. Some ap- proaches to post retrieval use indirect quality mea- sures (e.g., elaborate spam filtering (Java et al., 2007) or counting inlinks (Mishne, 2007a)). Few systems turn the credibility (Metzger, 2007) of blog posts into an aspect that can benefit the re- trieval process. Our hypothesis is that more credible blog posts are preferred by searchers. The idea of us- ing credibility in the blogosphere is not new: Rubin and Liddy (2006) define a framework for assessing blog credibility, consisting of four main categories: blogger’s expertise and offline identity disclosure; blogger’s trustworthiness and value system; infor- mation quality; and appeals and triggers of a per- sonal nature. Under these four categories the authors list a large number of indicators, some of which can be determined from textual sources (e.g., literary ap- peal), and some of which typically need non-textual evidence (e.g., curiosity trigger); see Section 2. We give concrete form to Rubin and Liddy (2006)’s indicators and test their impact on blog post retrieval effectiveness. We do not consider all indi- cators: we only consider indicators that are textual in nature, and to ensure reproducibility of our results, we only …

Content truncated to 500 words