Ranking Class Labels Using Query Sessions

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1607–1615, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Ranking Class Labels Using Query Sessions Marius Pas¸ca Google Inc. 1600 Amphitheatre Parkway Mountain View, California 94043 mars@google.com Abstract The role of search queries, as available within query sessions or in isolation from one an- other, in examined in the context of ranking the class labels (e.g., brazilian cities, busi- ness centers, hilly sites) extracted from Web documents for various instances (e.g., rio de janeiro). The co-occurrence of a class la- bel and an instance, in the same query or within the same query session, is used to re- inforce the estimated relevance of the class la- bel for the instance. Experiments over eval- uation sets of instances associated with Web search queries illustrate the higher quality of the query-based, re-ranked class labels, rel- ative to ranking baselines using document- based counts. 1 Introduction Motivation: The offline acquisition of instances (rio de janeiro, porsche cayman) and their correspond- ing class labels (brazilian cities, locations, vehicles, sports cars) from text has been an active area of re- search. In order to extract fine-grained classes of instances, existing methods often apply manually- created (Banko et al., 2007; Talukdar et al., 2008) or automatically-learned (Snow et al., 2006) extraction patterns to text within large document collections. In Web search, the relative ranking of documents returned in response to a query directly affects the outcome of the search. Similarly, the quality of the relative ranking among class labels extracted for a given instance influences any applications (e.g., query refinements or structured extraction) using the extracted data. But due to noise in Web data and limitations of extraction techniques, class labels ac- quired for a given instance (e.g., oil shale) may fail to properly capture the semantic classes to which the instance may belong (Kozareva et al., 2008). In- evitably, some of the extracted class labels will be less useful (e.g., sources, mutual concerns) or incor- rect (e.g., plants for the instance oil shale). In pre- vious work, the relative ranking of class labels for an instance is determined mostly based on features derived from the source Web documents from which the data has been extracted, such as variations of the frequency of co-occurrence or diversity of extraction patterns producing a given pair (Etzioni et al., 2005). Contributions: This paper explores the role of Web search queries, rather than Web documents, in inducing superior ranking among class labels ex- tracted automatically from documents for various in- stances. It compares two sources of indirect ranking evidence available within anonymized query logs: a) co-occurrence of an instance and its class label in the same query; and b) co-occurrence of an in- stance and its class label, as separate queries within the same query session. The former source is a noisy attempt to capture queries that narrow the search re- sults to a particular class of the instance (e.g., jaguar car maker). In comparison, …

Content truncated to 500 words