Can Pre-trained Language Models Interpret Similes as Smart as Human?

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7875 - 7887 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Can Pre-trained Language Models Interpret Similes as Smart as Human? Qianyu He1∗, Sijie Cheng1∗, Zhixu Li1†, Rui Xie3, Yanghua Xiao1,2† 1Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University 2Fudan-Aishu Cognitive Intelligence Joint Research Center, Shanghai, China 3Meituan, Shanghai, China qyhe21@m.fudan.edu.cn, rui.xie@meituan.com, {sjcheng20, zhixuli, shawyh}@fudan.edu.cn Abstract Simile interpretation is a crucial task in natural language processing. Nowadays, pre-trained language models (PLMs) have achieved state- of-the-art performance on many tasks. How- ever, it remains under-explored whether PLMs can interpret similes or not. In this paper, we investigate the ability of PLMs in simile in- terpretation by designing a novel task named Simile Property Probing, i.e., to let the PLMs infer the shared properties of similes. We construct our simile property probing datasets from both general textual corpora and human- designed questions, containing 1,633 exam- ples covering seven main categories. Our em- pirical study based on the constructed datasets shows that PLMs can infer similes’ shared properties while still underperforming humans. To bridge the gap with human performance, we additionally design a knowledge-enhanced training objective by incorporating the simile knowledge into PLMs via knowledge embed- ding methods. Our method results in a gain of 8.58% in the probing task and 1.37% in the downstream task of sentiment classifica- tion. The datasets and code are publicly avail- able at https://github.com/Abbey4799/PLMs- Interpret-Simile. 1 Introduction A simile is a figure of speech comparing two fun- damentally different entities via shared properties (Paul, 1970). There are two types of similes as illustrated in Figure 1, closed similes explicitly re- veal the shared properties between the topic entity and the vehicle entity, such as the property “slow” shared by “lady” and “snail” in the sentence “The old lady walks as slow as a snail”; while open similes do not state the shared property such as the sentence “The old lady walks like a snail”. Similes play a vital role in human expression to make literal ∗Equal contribution †Corresponding author Figure 1: Examples of two types of similes. Whether the component property is stated determines the type of simile. utterances more vivid and graspable and are widely used in the corpus of various domains (Liu et al., 2018; Chakrabarty et al., 2020a; Zhang et al., 2020). It is estimated that over 30% of the comparisons can be regarded as similes in product reviews (Nic- ulae and Danescu-Niculescu-Mizil, 2014). Simile interpretation is a crucial task in natural language processing (Veale and Hao, 2007; Qadir et al., 2016; Chakrabarty et al., 2021a), which can assist several downstream tasks such as understand- ing more sophisticated figurative language (Veale and Hao, 2007) and sentiment analysis (Niculae and Danescu-Niculescu-Mizil, 2014; Qadir et al., 2015). Take the simile “the lawyer is like a shark” for an example. Although all words in this simile are neutral, this simile expresses a negative affect since “lawyer” …

Content truncated to 500 words