Probing Simile Knowledge from Pre-trained Language Models

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5875 - 5887 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Probing Simile Knowledge from Pre-trained Language Models Weijie Chen1∗, Yongzhu Chang2*, Rongsheng Zhang2*, Jiashu Pu2 Guandan Chen2, Le Zhang2, Yadong Xi2, Yijiang Chen1 and Chang Su1† 1 School of Informatics, Xiamen University, Xiamen, China 2 Fuxi AI Lab, NetEase Inc., Hangzhou, China chenwj@stu.xmu.edu.cn, {changyongzhu,zhangrongsheng}@corp.netease.com Abstract Simile interpretation (SI) and simile genera- tion (SG) are challenging tasks for NLP be- cause models require adequate world knowl- edge to produce predictions. Previous works have employed many hand-crafted resources to bring knowledge-related into models, which is time-consuming and labor-intensive. In recent years, pre-trained language models (PLMs) based approaches have become the de- facto standard in NLP since they learn generic knowledge from a large corpus. The knowl- edge embedded in PLMs may be useful for SI and SG tasks. Nevertheless, there are few works to explore it. In this paper, we probe simile knowledge from PLMs to solve the SI and SG tasks in the unified framework of sim- ile triple completion for the first time. The backbone of our framework is to construct masked sentences with manual patterns and then predict the candidate words in the masked position. In this framework, we adopt a sec- ondary training process (Adjective-Noun mask Training) with the masked language model (MLM) loss to enhance the prediction diver- sity of candidate words in the masked position. Moreover, pattern ensemble (PE) and pattern search (PS) are applied to improve the quality of predicted words. Finally, automatic and hu- man evaluations demonstrate the effectiveness of our framework in both SI and SG tasks. 1 Introduction The simile, which is a special type of metaphor, is defined as a figurative expression in which two fundamentally different things are explicitly com- pared, usually using “like” or “as” (Israel et al., 2004; Zeng et al., 2020). It is widely used in litera- ture because it can inspire the reader’s imagination (Paul, 1970) by giving a vivid and unexpected anal- ogy between two objects with similar attributes. ∗Equal contribution. Work is done during Weijie’s intern- ship at NetEase Inc.. †Corresponding author. E-mail: suchang@xmu.edu.cn. (a) Simile Interpretation ( love, ? , rose ) beautiful, thorny, wizen, … Simile Triple Completion (STC) (b) Simile Generation ( love, beautiful, ? ) rose, swan, spring, … Figure 1: In the form of triple, the tasks of Simile In- terpretation and Simile Generation can be unified into Simile Triple Completion. A simile sentence usually contains three key ele- ments: the tenor, the attribute and the vehicle,1 which can be defined in the form of a triple (tenor, attribute, vehicle) (Song et al., 2021). For example, the simile sentence “Love is as thorny as rose” can be extracted as the triple (love, thorny, rose), where the tenor is “love”, the vehicle is “rose”, and the attribute is “thorny”. Note that a simile triple can produce different simile sentences with different templates. …

Content truncated to 500 words