Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 21–31, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering Clayton Greenberg, Asad Sayeed and Vera Demberg Computational Linguistics and Phonetics / M2CI Cluster of Excellence Saarland University 66123 Saarbr¨ucken, Germany {claytong,asayeed,vera}@coli.uni-saarland.de Abstract Most recent unsupervised methods in vector space semantics for assessing thematic fit (e.g. Erk, 2007; Baroni and Lenci, 2010; Sayeed and Demberg, 2014) create prototypical role- fillers without performing word sense disam- biguation. This leads to a kind of sparsity problem: candidate role-fillers for different senses of the verb end up being measured by the same “yardstick”, the single prototypical role-filler. In this work, we use three different feature spaces to construct robust unsupervised mod- els of distributional semantics. We show that correlation with human judgements on the- matic fit estimates can be improved consis- tently by clustering typical role-fillers and then calculating similarities of candidate role- fillers with these cluster centroids. The sug- gested methods can be used in any vector space model that constructs a prototype vec- tor from a non-trivial set of typical vectors. 1 Introduction Thematic fit estimations can be quite useful for many NLP applications and also for cognitive mod- els of human language processing difficulty, since human processing difficulty is highly sensitive to semantic plausibilities (Ehrlich and Rayner, 1981). For example, we expect that after the word mash, banana would be easier to process because it fits well as the patient, or direct object, of mash, but milk would be harder to process because it does not fit well. A common method for estimating the thematic fit between a verb and a proposed role filler involves computing a centroid, or vector average, over the most typical role fillers for that verb, and then cal- culating the cosine similarity between this centroid and the proposed role filler (Baroni and Lenci, 2010; Blacoe and Lapata, 2012; Erk, 2012). For instance, we use the cosine of the angle between the banana vector and a vector average of the 20 nouns that, according to training data, are most likely to be mashed as a score for how well banana fits as the patient of mash. Hopefully, the banana vector will be closer to the centroid than milk, so banana will have a higher cosine similarity to the centroid, and thus a higher thematic fit score, than milk. This conceptualization assumes that the most typ- ical fillers for a verb-role will all be variants of a single prototype, i.e. distributionally similar to each other. However, such an assumption may not be true for ambiguous verbs. A verb with many dif- ferent senses may have typical fillers for each sense, which fit relatively equally well, but are distribution- ally very different from one another. This means that the calculated prototypical filler will be a mixture of the arguments that are typical role fillers for the main senses …

Content truncated to 500 words