Learning Semantic Composition to Detect Non-compositionality of Multiword Expressions

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1733–1742, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Learning Semantic Composition to Detect Non-compositionality of Multiword Expressions Majid Yazdani Computer Science Department University of Geneva majid.yazdani@unige.ch Meghdad Farahmand Computer Science Department University of Geneva meghdad.farahmand@ unige.ch James Henderson Xerox Research Center Europe james.henderson@ xrce.xerox.com Abstract Non-compositionality of multiword ex- pressions is an intriguing problem that can be the source of error in a variety of NLP tasks such as language genera- tion, machine translation and word sense disambiguation. We present methods of non-compositionality detection for En- glish noun compounds using the unsu- pervised learning of a semantic compo- sition function. Compounds which are not well modeled by the learned semantic composition function are considered non- compositional. We explore a range of dis- tributional vector-space models for seman- tic composition, empirically evaluate these models, and propose additional methods which improve results further. We show that a complex function such as polyno- mial projection can learn semantic compo- sition and identify non-compositionality in an unsupervised way, beating all other baselines ranging from simple to complex. We show that enforcing sparsity is a useful regularizer in learning complex composi- tion functions. We show further improve- ments by training a decomposition func- tion in addition to the composition func- tion. Finally, we propose an EM algo- rithm over latent compositionality annota- tions that also improves the performance. 1 Introduction Multiword Expressions (MWEs) are sequences of words that exhibit some kind of idiosyncrasy. This idiosyncrasy can be semantic, statistical, or syn- tactic1. Ivory tower, speed limit, and at large 1MWEs can have other less significant kinds of idiosyn- crasies. For instance lexical idiosyncrasy as in ad hoc, and pragmatic idiosyncrasy as in good morning (Baldwin and Kim, 2010) are examples of semantically, statistically and syn- tactically idiosyncratic MWEs respectively. Note that an MWE can be idiosyncratic at several lev- els. In general, semantically idiosyncratic MWEs are commonly referred to as non-compositional (Baldwin and Kim, 2010) and statistically idiosyn- cratic MWEs are commonly referred to as col- locations (Sag et al., 2002). Non-compositional MWEs are those whose meaning can not be read- ily inferred from the meaning of their constituents and collocations are those MWEs whose con- stituents co-occur more than expected by chance. Collocations constitute the largest subset of all kinds of MWEs, however, non-compositional ones cause more problems in various NLP tasks, for ex- ample word sense disambiguation (McCarthy et al., 2003) and machine translation (Lin, 1999). It may also be more challenging to model non- compositionality than collocational weight as the former has to do with modelling the semantics and the latter can to some extent be modeled by con- ventional statistical measures such as mutual in- formation. Detecting non-compositionality in an automatic fashion has been the aim of much pre- vious research. In this paper, we capture non-compositionality of English Noun Compounds (NCs)2 based on the assumption that the majority of the compounds are compositional, for …

Content truncated to 500 words