Cross-Cutting Models of Lexical Semantics

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1405–1415, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Cross-Cutting Models of Lexical Semantics Joseph Reisinger Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 joeraii@cs.utexas.edu Raymond Mooney Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 mooney@cs.utexas.edu Abstract Context-dependent word similarity can be measured over multiple cross-cutting dimen- sions. For example, lung and breath are sim- ilar thematically, while authoritative and su- perficial occur in similar syntactic contexts, but share little semantic similarity. Both of these notions of similarity play a role in deter- mining word meaning, and hence lexical se- mantic models must take them both into ac- count. Towards this end, we develop a novel model, Multi-View Mixture (MVM), that rep- resents words as multiple overlapping clus- terings. MVM finds multiple data partitions based on different subsets of features, sub- ject to the marginal constraint that feature sub- sets are distributed according to Latent Dirich- let Allocation. Intuitively, this constraint fa- vors feature partitions that have coherent top- ical semantics. Furthermore, MVM uses soft feature assignment, hence the contribution of each data point to each clustering view is vari- able, isolating the impact of data only to views where they assign the most features. Through a series of experiments, we demonstrate the utility of MVM as an inductive bias for captur- ing relations between words that are intuitive to humans, outperforming related models such as Latent Dirichlet Allocation. 1 Introduction Humans categorize objects using multiple orthogo- nal taxonomic systems, where category generaliza- tion depends critically on what features are relevant to one particular system. For example, foods can be organized in terms of their nutritional value (high in fiber) or situationally (commonly eaten for Thanks- giving; Shafto et al. (2006)). Human knowledge- bases such as Wikipedia also exhibit such multiple clustering structure (e.g. people are organized by oc- cupation or by nationality). The effects of these overlapping categorization systems manifest them- selves at the lexical semantic level (Murphy, 2002), implying that lexicographical word senses and tra- ditional computational models of word-sense based on clustering or exemplar activation are too impov- erished to capture the rich dynamics of word usage. In this work, we introduce a novel probabilis- tic clustering method, Multi-View Mixture (MVM), based on cross-cutting categorization (Shafto et al., 2006) that generalizes traditional vector-space or distributional models of lexical semantics (Curran, 2004; Pad´o and Lapata, 2007; Sch¨utze, 1998; Tur- ney, 2006). Cross-cutting categorization finds multi- ple feature subsets (categorization systems) that pro- duce high quality clusterings of the data. For exam- ple words might be clustered based on their part of speech, or based on their thematic usage. Context- dependent variation in word usage can be accounted for by leveraging multiple latent categorization sys- tems. In particular, cross-cutting models can be used to capture both syntagmatic and paradigmatic no- tions of word relatedness, breaking up word features into multiple categorization systems and …

Content truncated to 500 words