Identifying Multi-word Expressions by Leveraging Morphological and Syntactic Idiosyncrasy

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 10–18, Beijing, August 2010 Identifying Multi-word Expressions by Leveraging Morphological and Syntactic Idiosyncrasy Hassan Al-Haj Language Technologies Institute Carnegie Mellon University hhaj@cs.cmu.edu Shuly Wintner Department of Computer Science University of Haifa shuly@cs.haifa.ac.il Abstract Multi-word expressions constitute a sig- nificant portion of the lexicon of every natural language, and handling them cor- rectly is mandatory for various NLP appli- cations. Yet such entities are notoriously hard to define, and are consequently miss- ing from standard lexicons and dictionar- ies. Multi-word expressions exhibit id- iosyncratic behavior on various levels: or- thographic, morphological, syntactic and semantic. In this work we take advan- tage of the morphological and syntactic idiosyncrasy of Hebrew noun compounds and employ it to extract such expressions from text corpora. We show that relying on linguistic information dramatically im- proves the accuracy of compound extrac- tion, reducing over one third of the errors compared with the best baseline. 1 Introduction Multi-word expressions (MWEs) are notoriously hard to define. They span a range of constructions, from completely frozen, semantically opaque id- iomatic expressions, to frequent but morpholog- ically productive and semantically compositional collocations. Various linguistic processes (ortho- graphic, morphological, syntactic, semantic, and cognitive) apply to MWEs in idiosyncratic ways. Notably, MWEs blur the distinction between the lexicon and the grammar, since they often have some properties of words and some of phrases. In this work we define MWEs as expressions whose linguistic properties (morphological, syn- tactic or semantic) are not directly derived from the properties of their word constituents. This is a functional definition, driven by a practical mo- tivation: any natural language processing (NLP) application that cares about morphology, syntax or semantics must consequently store MWEs in the lexicon. MWEs are numerous and constitute a signif- icant portion of the lexicon of any natural lan- guage. They are a heterogeneous class of con- structions with diverse sets of characteristics. Morphologically, some MWEs allow some of their constituents to freely inflect while restricting (or even preventing) the inflection of other con- stituents. MWEs may allow constituents to un- dergo non-standard morphological inflections that they would not undergo in isolation. Some MWEs contain words that never occur outside the context of the MWE. Syntactically, some MWEs appear in one rigid pattern (and a fixed order), while oth- ers permit various syntactic transformations. Se- mantically, the compositionality of MWEs (i.e., the degree to which the meaning of the whole ex- pression results from combining the meanings of its individual words when they occur in isolation) is gradual. These morphological, syntactic and semantic idiosyncrasies make MWEs a challenge for NLP applications (Sag et al., 2002). They are even more challenging in languages with complex mor- phology, because of the unique interaction of mor- phological and orthographic processes with the lexical specification of MWEs (Oflazer et al., 2004; Alegria et al., 2004). Because the idiosyncratic features of MWEs cannot be predicted on the basis of their com- ponent words, they must be stored …

Content truncated to 500 words