Deep Learning Models For Multiword Expression Identification

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 54–64, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Deep Learning Models For Multiword Expression Identification Waseem Gharbieh and Virendra C. Bhavsar and Paul Cook Faculty of Computer Science, University of New Brunswick Fredericton, NB E3B 5A3 Canada {waseem.gharbieh,bhavsar,paul.cook}@unb.ca Abstract Multiword expressions (MWEs) are lex- ical items that can be decomposed into multiple component words, but have prop- erties that are unpredictable with respect to their component words. In this paper we propose the first deep learning mod- els for token-level identification of MWEs. Specifically, we consider a layered feed- forward network, a recurrent neural net- work, and convolutional neural networks. In experimental results we show that con- volutional neural networks are able to out- perform the previous state-of-the-art for MWE identification, with a convolutional neural network with three hidden layers giving the best performance. 1 Introduction Multiword expressions (MWEs) are lexical items that can be decomposed into multiple component words, but have properties that are idiomatic, i.e., marked or unpredictable, with respect to proper- ties of their component words (Baldwin and Kim, 2010). MWEs include a wide range of phenom- ena such as noun compounds (e.g., speed limit and monkey business), verb–particle constructions (e.g., clean up and throw out), and verb–noun id- iomatic combinations (e.g., hit the roof and blow the whistle), as well as named entities (e.g., Prime Minister Justin Trudeau) and proverbs (e.g., Two wrongs don’t make a right). One particular chal- lenge for natural language processing (NLP) is MWE identification — i.e., to identify which to- kens in running text correspond to MWEs so that they can be analyzed accordingly. The challenges posed by MWEs have led to them to be referred to as a “pain in the neck” for NLP (Sag et al., 2002); nevertheless, incorporating knowledge of MWEs into NLP applications can lead to improvements in tasks including machine translation (Carpuat and Diab, 2010), information retrieval (Newman et al., 2012), and opinion mining (Berend, 2011). Recent work on token-level MWE identification has focused on methods that are applicable to the full spectrum of kinds of MWEs (Schneider et al., 2014a), in contrast to earlier work that tended to focus on specific kinds of MWEs (Uchiyama et al., 2005; Fazly et al., 2009; Fothergill and Baldwin, 2012). Deep learning is an emerging class of ma- chine learning models that have recently achieved promising results on a range of NLP tasks such as machine translation (Bahdanau et al., 2015; Sutskever et al., 2014), named entity recognition (Lample et al., 2016), natural language generation (Li et al., 2015), and sentence classification (Kim, 2014). Such models have, however, not yet been applied to broad-coverage MWE identification. In this paper we propose the first deep learn- ing models for broad-coverage MWE identifica- tion. Specifically, we propose and evaluate a layered feedforward network, a recurrent neural network, and two convolutional neural networks. We compare these models against the previous state-of-the-art (Kirilin et al., 2016) …

Content truncated to 500 words