Latent Variable Models of Selectional Preference

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 435–444, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Latent variable models of selectional preference Diarmuid ´O S´eaghdha University of Cambridge Computer Laboratory United Kingdom do242@cl.cam.ac.uk Abstract This paper describes the application of so-called topic models to selectional pref- erence induction. Three models related to Latent Dirichlet Allocation, a proven method for modelling document-word co- occurrences, are presented and evaluated on datasets of human plausibility judge- ments. Compared to previously proposed techniques, these models perform very competitively, especially for infrequent predicate-argument combinations where they exceed the quality of Web-scale pre- dictions while using relatively little data. 1 Introduction Language researchers have long been aware that many words place semantic restrictions on the words with which they can co-occur in a syntactic relationship. Violations of these restrictions make the sense of a sentence odd or implausible: (1) Colourless green ideas sleep furiously. (2) The deer shot the hunter. Recognising whether or not a selectional restriction is satisfied can be an important trigger for metaphor- ical interpretations (Wilks, 1978) and also plays a role in the time course of human sentence process- ing (Rayner et al., 2004). A more relaxed notion of selectional preference captures the idea that certain classes of entities are more likely than others to fill a given argument slot of a predicate. In Natu- ral Language Processing, knowledge about proba- ble, less probable and wholly infelicitous predicate- argument pairs is of value for numerous applica- tions, for example semantic role labelling (Gildea and Jurafsky, 2002; Zapirain et al., 2009). The notion of selectional preference is not restricted to surface-level predicates such as verbs and mod- ifiers, but also extends to semantic frames (Erk, 2007) and inference rules (Pantel et al., 2007). The fundamental problem that selectional prefer- ence models must address is data sparsity: in many cases insufficient corpus data is available to reliably measure the plausibility of a predicate-argument pair by counting its observed frequency. A rarely seen pair may be fundamentally implausible (a carrot laughed) or plausible but rarely expressed (a manservant laughed).1 In general, it is benefi- cial to smooth plausibility estimates by integrating knowledge about the frequency of other, similar predicate-argument pairs. The task thus share some of the nature of language modelling; however, it is a task less amenable to approaches that require very large training corpora and one where the semantic quality of a model is of greater importance. This paper takes up tools (“topic models”) that have been proven successful in modelling document-word co-occurrences and adapts them to the task of selectional preference learning. Ad- vantages of these models include a well-defined generative model that handles sparse data well, the ability to jointly induce semantic classes and predicate-specific distributions over those classes, and the enhanced statistical strength achieved by sharing knowledge across predicates. Section 2 surveys prior work on selectional preference mod- elling and on semantic applications of topic models. Section 3 describes the models used …

Content truncated to 500 words