Team INF-UFRGS at SemEval-2023 Task 7: Supervised Contrastive Learning for Pair-level Sentence Classification and Evidence Retrieval

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 700–706 July 13-14, 2023 ©2023 Association for Computational Linguistics Team INF-UFRGS at SemEval-2023 Task 7: Supervised Contrastive Learning for Pair-level Sentence Classification and Evidence Retrieval Abel C. Dias Filipe F. Dias Higor Moreira Viviane P. Moreira João L. D. Comba Instituto de Informática - UFRGS - Brazil {abel.correa,ffdias,hmoreira,viviane,comba}@inf.ufrgs.br Abstract This paper describes the EvidenceSCL sys- tem submitted by our team (INF-UFRGS) to SemEval-2023 Task 7: Multi-Evidence Natu- ral Language Inference for Clinical Trial Data (NLI4CT). NLI4CT is divided into two tasks, one for determining the inference relation be- tween a pair of statements in clinical trials and a second for retrieving a set of support- ing facts from the premises necessary to justify the label predicted in the first task. Our ap- proach uses pair-level supervised contrastive learning to classify pairs of sentences. We trained EvidenceSCL on two datasets created from NLI4CT and additional data from other NLI datasets. We show that our approach can address both goals of NLI4CT, and although it reached an intermediate position in the ranking of participating system, there is room for im- provement in the technique. 1 Introduction There has been a significant increase in medical publications in recent years, including clinical trial data. The SemEval-2023 Task 7, called Multi- Evidence Natural Language Inference for Clinical Trial Data (NLI4CT), addresses the problem of large-scale interpretability and evidence retrieval from breast cancer clinical trial reports (Jullien et al., 2023). Currently, there are about 10K reports of breast cancer, making it difficult for clinical practitioners who want to provide care based on re- liable clinical evidence to analyze all these reports carefully. Thus, NLI4CT is divided into Task 1 - Textual Entailment and Task 2 - Evidence Retrieval (ER). The goal of Task 1 is to determine the infer- ence relation (contradiction/entailment) between statement pairs (premise and hypothesis), and Task 2 is to output the supporting facts from the premise to justify the label in Task 1. Recent work leveraged Natural Language Infer- ence (NLI) data to enhance sentence representa- tions, improving results in downstream tasks. Pair- SupCon (Zhang et al., 2021) adds a discrimina- tion head to separate entailment and contradiction instances while learning high-level semantic in- formation from the sentence pairs and negative examples. A linear classification head enables sup- porting pairwise entailment and contradiction rea- soning. Similarly, PairSCL (Li et al., 2022) adds a cross-attention module to enhance pair-level repre- sentation by calculating the token-level co-attention matrix to indicate the relevance of the tokens in the premises and hypotheses. Our approach explores the cross-attention mo- dule in PairSCL for evidence retrieval by separa- ting evidence from non-evidence in the embedding space. EvidenceSCL is built on top of PairSCL (Li et al., 2022), employing a supervised contrastive loss in the evidence retrieval objective and a clas- sification head in the textual entailment objective. We also created two datasets combining MedNLI (Romanov and Shivade, 2018), a small part of MultiNLI (Williams et al., …

Content truncated to 500 words