Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4353–4362 August 1–6, 2021. ©2021 Association for Computational Linguistics 4353 Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction Kosuke Yamada1 Ryohei Sasano1,2 Koichi Takeda1 1Graduate School of Informatics, Nagoya University, Japan 2RIKEN Center for Advanced Intelligence Project, Japan yamada.kosuke@c.mbox.nagoya-u.ac.jp, {sasano,takedasu}@i.nagoya-u.ac.jp Abstract Contextualized word representations have proven useful for various natural language pro- cessing tasks. However, it remains unclear to what extent these representations can cover hand-coded semantic information such as se- mantic frames, which specify the semantic role of the arguments associated with a pred- icate. In this paper, we focus on verbs that evoke different frames depending on the con- text, and we investigate how well contextual- ized word representations can recognize the difference of frames that the same verb evokes. We also explore which types of representation are suitable for semantic frame induction. In our experiments, we compare seven different contextualized word representations for two English frame-semantic resources, FrameNet and PropBank. We demonstrate that sev- eral contextualized word representations, espe- cially BERT and its variants, are considerably informative for semantic frame induction. Fur- thermore, we examine the extent to which the contextualized representation of a verb can es- timate the number of frames that the verb can evoke. 1 Introduction Contextualized word representations such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019) are known to be effective in many natural language processing tasks such as question answering, nat- ural language inference, and semantic textual sim- ilarity. Contextualized word representations can generate different representations of the same word in different contexts and distinguish the polysemy of a word. It has been reported that this property is effective in word sense disambiguation (WSD) (Hadiwinoto et al., 2019) and word sense induction (WSI) (Amrami and Goldberg, 2018). Therefore, it appears that contextualized word representations can also be leveraged to induce semantic frames from a large corpus automatically. A semantic frame is defined on the basis of the semantic roles that a predicate can take as its argu- ments. FrameNet1 (Baker et al., 1998) and Prop- Bank2 (Palmer et al., 2005) are the two most well- known resources of semantic frames, both of which are manually compiled. These resources are used not only for semantic parsing (Yang and Mitchell, 2017) but also for information extraction (Gangemi et al., 2016), question answering (Shen and Lapata, 2007), and document summarization (Cheung and Penn, 2013). These frame-semantic resources define the frame and semantic roles, and they provide example sen- tences in which they are annotated. For example, the verb “support” in FrameNet is defined to evoke two frames: the SUPPORTING frame and the EV- IDENCE frame. Sentences (1) and (2) below are examples where these frames are annotated. In Sen- tence (1), “support” means ‘supporting a person or a thing’ and evokes the SUPPORTING frame. Its arguments are annotated with the semantic roles of Supporter and Supported. In Sentence (2), “support” means ‘corroborating’ and evokes the EVIDENCE frame. …

Content truncated to 500 words