Idiom-Aware Compositional Distributed Semantics

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1204–1213 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Idiom-Aware Compositional Distributed Semantics Pengfei Liu Kaiyu Qian Xipeng Qiu∗ Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {pfliu14,kyqian, xpqiu,xjhuang}@fudan.edu.cn Abstract Idioms are peculiar linguistic construc- tions that impose great challenges for rep- resenting the semantics of language, es- pecially in current prevailing end-to-end neural models, which assume that the se- mantics of a phrase or sentence can be literally composed from its constitutive words. In this paper, we propose an idiom- aware distributed semantic model to build representation of sentences on the basis of understanding their contained idioms. Our models are grounded in the literal- first psycholinguistic hypothesis, which can adaptively learn semantic composi- tionality of a phrase literally or idiomat- ically. To better evaluate our models, we also construct an idiom-enriched senti- ment classification dataset with consider- able scale and abundant peculiarities of id- ioms. The qualitative and quantitative ex- perimental analyses demonstrate the effi- cacy of our models. The newly-introduced datasets are publicly available at http: //nlp.fudan.edu.cn/data/ 1 Introduction Currently, neural network models have achieved great success for many natural language process- ing (NLP) tasks , such as text classification (Zhao et al., 2015; Liu et al., 2017), semantic matching (Liu et al., 2016a,b), and machine translation (Cho et al., 2014). The key factor of these neural mod- els is how to compose a phrase or sentence rep- resentation from its constitutive words. Typically, a shared compositional function is used to com- pose word vectors recursively until obtaining the representation of the phrase or sentence. The form ∗Corresponding author. of compositional function involves many kinds of neural networks, such as recurrent neural networks (Hochreiter and Schmidhuber, 1997; Chung et al., 2014), convolutional neural networks (Collobert et al., 2011; Kalchbrenner et al., 2014), and re- cursive neural networks (Socher et al., 2013; Tai et al., 2015; Zhu et al., 2015). However, these methods show an obvious de- fect in representing idiomatic phrases, whose se- mantics are not literal compositions of the individ- ual words. For example, “pulling my leg” is idiomatic, and its meaning cannot be directly derived from a literal combination of its con- tained words. Due to its importance, some pre- vious work focuses on automatic identification of idioms (Katz and Giesbrecht, 2006; Li and Sporleder, 2009; Fazly et al., 2009; Peng et al., 2014; Salton et al., 2016). However, challenge re- mains to take idioms into account to improve neu- ral based semantic representations of phrases or sentences. Motivated by the literal-first psycholinguistic hypothesis proposed by Bobrow and Bell (1973), in this paper, we propose an end-to-end neural model for idiom-aware distributed semantic rep- resentation, in which we adopt a neural architec- ture of recursive network (Socher et al., 2013; Tai et al., 2015; Zhu et al., 2015) to learn the composi- tional semantics over a constituent …

Content truncated to 500 words