Improving Event Temporal Relation Classification via Auxiliary Label-Aware Contrastive Learning

Improving Event Temporal Relation Classification via Auxiliary Label-Aware Contrastive Learning Tiesen Sun School of Computer Science and Technology, Dalian University of Technology, Dalian, China Lishuang Li ∗ School of Computer Science and Technology, Dalian University of Technology, Dalian, China lils@dlut.edu.cn Abstract Event Temporal Relation Classification (ETRC) is crucial to natural language understanding. In recent years, the mainstream ETRC methods may not take advantage of lots of semantic informa- tion contained in golden temporal relation labels, which is lost by the discrete one-hot labels. To alleviate the loss of semantic information, we propose learning Temporal semantic information of the golden labels by Auxiliary Contrastive Learning (TempACL). Different from traditional con- trastive learning methods, which further train the PreTrained Language Model (PTLM) with un- supervised settings before fine-tuning on target tasks, we design a supervised contrastive learning framework and make three improvements. Firstly, we design a new data augmentation method that generates augmentation data via matching templates established by us with golden labels. Secondly, we propose patient contrastive learning and design three patient strategies. Thirdly we design a label-aware contrastive learning loss function. Extensive experimental results show that our TempACL effectively adapts contrastive learning to supervised learning tasks which remain a challenge in practice. TempACL achieves new state-of-the-art results on TB-Dense and MA- TRES and outperforms the baseline model with up to 5.37%F1 on TB-Dense and 1.81%F1 on MATRES. 1 Introduction The temporal relations of events are used to describe the occurring sequence of events in an article. Therefore understanding the temporal relations of events in articles is useful for many downstream tasks such as timeline creation (Leeuwenberg and Moens, 2018), generating stories (Goldfarb-Tarrant et al., 2020), forecasting social events (Jin et al., 2021), and reading comprehension (Ning et al., 2020). Hence, the ETRC task is an important and popular natural language understanding research topic among NLP community. The ETRC task is to determine the occurrence sequence of a given event pair. The context of the event pair is usually given to aid judgment. Ning et al. (2019) first encoded the event pairs into embedded representations and then used fully connected layers as a classifier to generate confidence scores for each category of temporal relations. All related works of the NLP community since then have followed the classification view: classifying the embedded representations. Naturally, we can encode the context and events into a better embedding space in which the different relations are distinguished well, to get better classification results. Traditionally, all recent works use one-hot vectors to represent golden temporal relation labels in the training stage. However, the one-hot vector reduces the label with practical semantics to the zero-one vector. It makes the embedded representations extracted by the ETRC models waiting for classifying be the similarities of the instances with the same label. But, the similarities are not equal to the label seman- ∗*Corresponding authors ©2022 China National Conference on Computational Linguistics Published under Creative Commons Attribution 4.0 International License Foundation: The National Natural Science Foundation of China (No. 62076048), The Science …

Content truncated to 500 words