Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3036 - 3049 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering Jun Gao1 Wei Wang3 Changlong Yu4 Huan Zhao5 Wilfred Ng4 Ruifeng Xu1,2∗ 1Harbin Institute of Technology (Shenzhen) 2Peng Cheng Laboratory 3Tsinghua University imgaojun@gmail.com xuruifeng@hit.edu.cn weiwangorg@163.com 4HKUST, Hong Kong, China 54Paradigm. Inc. {cyuaq,wilfred}@cse.ust.hk zhaohuan@4paradigm.com Abstract Representations of events described in text are important for various tasks. In this work, we present SWCC: a Simultaneous Weakly supervised Contrastive learning and Clustering framework for event representation learning. SWCC learns event representations by making better use of co-occurrence in- formation of events. Specifically, we intro- duce a weakly supervised contrastive learn- ing method that allows us to consider mul- tiple positives and multiple negatives, and a prototype-based clustering method that avoids semantically related events being pulled apart. For model training, SWCC learns represen- tations by simultaneously performing weakly supervised contrastive learning and prototype- based clustering. Experimental results show that SWCC outperforms other baselines on Hard Similarity and Transitive Sentence Similarity tasks. In addi- tion, a thorough analysis of the prototype- based clustering method demonstrates that the learned prototype vectors are able to implicitly capture various relations between events. Our code will be available at https://github. com/gaojun4ever/SWCC4Event. 1 Introduction Distributed representations of events, are a com- mon way to represent events in a machine-readable form and have shown to provide meaningful fea- tures for various tasks (Lee and Goldwasser, 2018; Rezaee and Ferraro, 2021; Deng et al., 2021; Mar- tin et al., 2018; Chen et al., 2021). Obtaining ef- fective event representations is challenging, as it requires representations to capture various relations between events. Figure 1 presents four pairs of events with different relations. Two events may share the same event attributes (e.g. event types and sentiments), and there may also be a causal or temporal relation between two events. ∗Corresponding author Jun Gao is currently a research intern at 4Paradigm. PersonX wanted a new career PersonX applies for a job PersonX graduates from college PersonX murders PersonY PersonX is shot and killed PersonX achieved goal PersonX broke record PersonX gets a diploma Sentiment Event type Temporal Relation Casual Relation Figure 1: Four pairs of events with different relations. Stars represent prototypes and circles represent events. Early works (Weber et al., 2018) exploit eas- ily accessible co-occurrence relation of events to learn event representations. Although the use of co- occurrence relation works well, it is too coarse for deep understanding of events, which requires fine- grained knowledge (Lee and Goldwasser, 2019). Recent works focus on fine-grained knowledge, such as discourse relations (Lee and Goldwasser, 2019; Zheng et al., 2020) and commonsense knowl- edge (e.g. sentiments and intents) (Sap et al., 2019; Ding et al., 2019). Concretely, Lee and Goldwasser (2019) and Zheng et al. (2020) leverage 11 dis- course relation types to model event script knowl- edge. Ding et al. (2019) incorporate manually la- beled commonsense …

Content truncated to 500 words