A Single Word is not Enough: Ranking Multiword Expressions Using Distributional Semantics

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2430–2440, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. A Single Word is not Enough: Ranking Multiword Expressions Using Distributional Semantics Martin Riedl and Chris Biemann Language Technology Computer Science Department, Technische Universit¨at Darmstadt Hochschulstrasse 10, D-64289 Darmstadt, Germany {riedl, biem}@cs.tu-darmstadt.de Abstract We present a new unsupervised mecha- nism, which ranks word n-grams accord- ing to their multiwordness. It heavily re- lies on a new uniqueness measure that computes, based on a distributional the- saurus, how often an n-gram could be re- placed in context by a single-worded term. In addition with a downweighting mech- anism for incomplete terms this forms a new measure called DRUID. Results show large improvements on two small test sets over competitive baselines. We demon- strate the scalability of the method to large corpora, and the independence of the mea- sure of shallow syntactic filtering. 1 Introduction While it seems intuitive to treat certain sequences of tokens as single terms, there is still consider- able controversy about the definition of what ex- actly such a multiword expression (MWE) con- stitutes. Sag et al. (2001) pinpoint the need of treating MWEs correctly and classify a range of syntactic formations that could form MWEs and define MWEs as being non-compositional with re- spect to the meaning of their parts. While the exact requirements on MWEs is bound to specific tasks (such as parsing, keyword extraction, etc.), we op- erationalize the notion of non-compositionality by using distributional semantics and introduce a new measure that works well for a range of task-based MWE definitions. Most previous MWE ranking approaches use the following mechanisms to determine multiwordness: part-of-speech (POS) tags, word/multiword frequency and significance of co-occurrence of the parts. In this paper we do not want to introduce ”yet another ranking function” but rather an additional mechanism, which performs ranking based on distributional semantics. Distributional semantics has already been used for MWE identification, but mainly to discriminate between compositional and non- compositional MWEs (Schone and Jurafsky, 2001; Salehi et al., 2014; Hermann and Blunsom, 2014). Here we introduce a new concept to de- scribe the multiwordness of a term by its unique- ness. Using the uniqueness score we measure how likely a term in context can be replaced by a single word. This measure is motivated by the semiotic consideration that due to parsimony con- cepts are often expressed as single words. Further- more, we implement a context-aware punishment, called incompleteness, which degrades the score of terms that seem incomplete regarding their con- texts. Both concepts are combined into a single score we call DRUID, which is calculated based on a distributional thesaurus. In the following, we show the impact of that new method for French and English and also examine the effect of cor- pus size on MWE extraction. Additionally, we report on results without using any linguistic pre- processing except tokenization. 2 Related Work The generation of MWE dictionaries has drawn much attention …

Content truncated to 500 words