Bridging the Gap: Attending to Discontinuity in Identification of Multiword Expressions

Proceedings of NAACL-HLT 2019, pages 2692–2698 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2692 Bridging the Gap: Attending to Discontinuity in Identification of Multiword Expressions Omid Rohanian†∗, Shiva Taslimipoor†∗, Samaneh Kouchaki‡, Le An Ha†, Ruslan Mitkov† †Research Group in Computational Linguistics, University of Wolverhampton, UK ‡Institute of Biomedical Engineering, University of Oxford, UK {omid.rohanian, shiva.taslimi, l.a.ha, r.mitkov}@wlv.ac.uk samaneh.kouchaki@eng.ox.ac.uk Abstract We introduce a new method to tag Multiword Expressions (MWEs) using a linguistically in- terpretable language-independent deep learn- ing architecture. We specifically target dis- continuity, an under-explored aspect that poses a significant challenge to computational treat- ment of MWEs. Two neural architectures are explored: Graph Convolutional Network (GCN) and multi-head self-attention. GCN leverages dependency parse information, and self-attention attends to long-range relations. We finally propose a combined model that integrates complementary information from both, through a gating mechanism. The ex- periments on a standard multilingual dataset for verbal MWEs show that our model outper- forms the baselines not only in the case of dis- continuous MWEs but also in overall F-score.1 1 Introduction Multiword expressions (MWEs) are linguistic units composed of more than one word whose meanings cannot be fully determined by the se- mantics of their components (Sag et al., 2002; Baldwin and Kim, 2010). As they are fraught with syntactic and semantic idiosyncrasies, their automatic identification remains a major challenge (Constant et al., 2017). Occurrences of discontin- uous MWEs are particularly elusive as they in- volve relationships between non-adjacent tokens (e.g. put one of the blue masks on). While some previous studies disregard discon- tinuous MWEs (Legrand and Collobert, 2016), others stress the importance of factoring them in (Schneider et al., 2014). Using a CRF-based and a transition-based approach respectively, Moreau et al. (2018) and Al Saied et al. (2017) try to ∗*The first two authors contributed equally. 1The code is available on https://github.com/ omidrohanian/gappy-mwes. capture discontinuous occurrences with help from dependency parse information. Previously ex- plored neural MWE identification models (Ghar- bieh et al., 2017) suffer from limitations in dealing with discontinuity, which can be attributed to their inherently sequential nature. More sophisticated architectures are yet to be investigated (Constant et al., 2017). Graph convolutional neural networks (GCNs) (Kipf and Welling, 2017) and attention-based neural sequence labeling (Tan et al., 2018) are methodologies suited for modeling non-adjacent relations and are hence adapted to MWE identifi- cation in this study. Conventional GCN (Kipf and Welling, 2017) uses a global graph structure for the entire input. We modify it such that GCN fil- ters convolve nodes of dependency parse tree on a per-sentence basis. Self-attention, on the other hand, learns representations by relating different parts of the same sequence. Each position in a se- quence is linked to any other position with O(1) operations, minimising maximum path (compared to RNN’s O(n)) which facilitates gradient flow and makes it theoretically well-suited for learning long-range dependencies (Vaswani et al., 2017). The difference in the two approaches moti- vates our attempt to incorporate them into a hy- …

Content truncated to 500 words