CCG Categories for Distributional Semantic Models

Proceedings of Recent Advances in Natural Language Processing, pages 467–474, Hissar, Bulgaria, 7-13 September 2013. CCG Categories for Distributional Semantic Models Paramita Mirza University of Trento paramita.paramita@unitn.it Raffaella Bernardi University of Trento bernardi@disi.unitn.it Abstract For the last decade, distributional seman- tics has been an active area of research to address the problem of understanding the semantics of words in natural language. The core principal of the distributional se- mantic approach is that the linguistic con- text surrounding a given word, which is represented as a vector, provides important information about its meaning. In this pa- per we investigate the possibility to exploit Combinatory Categorial Grammar (CCG) categories as syntactic features to be rele- vant for characterizing the context vector and hence the meaning of words. We find that the CCG categories can enhance the representation of verb meaning. 1 Introduction The distributional semantic approach is based on the idea that the meaning of a word relies heavily on its context. Hence, the meaning of a word can be represented as a vector of its co-occurrence fre- quency with the neighbouring words. There have been several works that explore ways to improve the representation of word meaning by incorpo- rating syntactic information in the context vec- tor, dependency relations between words being the commonly used syntactic features. Dependency- based Distributional Semantic Models (DSMs) have been tested against several tasks and shown to be among the best performing word space mod- els (Erk and Pado, 2008; Cruys, 2008; Baroni and Lenci, 2009; Baroni and Lenci, 2010). In this paper we investigate an alternative view on the syntactic features that can be used to en- rich the context vector, namely Combinatory Cat- egorial Grammar (CCG) categories, which pro- vide a transparent relation between syntactic cate- gory and semantic type of a linguistic expression. Hence, we propose to build a CCG-based DSM using a corpus annotated by a CCG parser. We test the model on word categorization tasks, in particular concrete noun and verb categoriza- tion. We are interested in investigating how the role of context changes in capturing lexical mean- ing among the different word categories (nouns vs. verbs). Furthermore, we explore the perfor- mance of the model in capturing the different cat- egories of verbs, based on several verb classifica- tions studied in the literature. By comparing the model based on CCG cat- egories with an analogous one based on Part of Speech (PoS), we study the role of richer syntac- tic information in the task of word categorization. Finally, we include also function words (gram- matical words) in the context vector instead of assuming that only content words (i.e. nouns, verbs, adjectives, adverbs) are relevant in captur- ing the word meaning. We find that for some cases function words are useful to distinguish different classes of verbs. 2 “Supertags” for Distributional Semantic Models We propose to investigate the role of constituent structures and features encoding tense informa- tion, by building a distributional model with di- mensions tagged by “supertags”, namely by …

Content truncated to 500 words