Probabilistic models of similarity in syntactic context

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1047–1057, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Probabilistic models of similarity in syntactic context Diarmuid ´O S´eaghdha Computer Laboratory University of Cambridge United Kingdom do242@cl.cam.ac.uk Anna Korhonen Computer Laboratory University of Cambridge United Kingdom Anna.Korhonen@cl.cam.ac.uk Abstract This paper investigates novel methods for in- corporating syntactic information in proba- bilistic latent variable models of lexical choice and contextual similarity. The resulting mod- els capture the effects of context on the inter- pretation of a word and in particular its effect on the appropriateness of replacing that word with a potentially related one. Evaluating our techniques on two datasets, we report perfor- mance above the prior state of the art for esti- mating sentence similarity and ranking lexical substitutes. 1 Introduction Distributional models of lexical semantics, which assume that aspects of a word’s meaning can be re- lated to the contexts in which that word is typically used, have a long history in Natural Language Pro- cessing (Sp¨arck Jones, 1964; Harper, 1965). Such models still constitute one of the most popular ap- proaches to lexical semantics, with many proven ap- plications. Much work in distributional semantics treats words as non-contextualised units; the models that are constructed can answer questions such as “how similar are the words body and corpse?” but do not capture the way the syntactic context in which a word appears can affect its interpretation. Re- cent developments (Mitchell and Lapata, 2008; Erk and Pad´o, 2008; Thater et al., 2010; Grefenstette et al., 2011) have aimed to address compositionality of meaning in terms of distributional semantics, lead- ing to new kinds of questions such as “how similar are the usages of the words body and corpse in the phrase the body/corpse deliberated the motion...?” and “how similar are the phrases the body deliber- ated the motion and the corpse rotted?”. In this pa- per we focus on answering questions of the former type and investigate models that describe the effect of syntactic context on the meaning of a single word. The work described in this paper uses probabilis- tic latent variable models to describe patterns of syn- tactic interaction, building on the selectional prefer- ence models of ´O S´eaghdha (2010) and Ritter et al. (2010) and the lexical substitution models of Dinu and Lapata (2010). We propose novel methods for incorporating information about syntactic context in models of lexical choice, yielding a probabilistic analogue to dependency-based models of contextual similarity. Our models attain state-of-the-art per- formance on two evaluation datasets: a set of sen- tence similarity judgements collected by Mitchell and Lapata (2008) and the dataset of the English Lexical Substitution Task (McCarthy and Navigli, 2009). In view of the well-established effectiveness of dependency-based distributional semantics and of probabilistic frameworks for semantic inference, we expect that our approach will prove to be of value in a wide range of application settings. 2 Related work The literature on distributional semantics is vast; in …

Content truncated to 500 words