CPIC at SemEval-2023 Task 7: GPT2-Based Model for Multi-evidence Natural Language Inference for Clinical Trial Data

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 397–401 July 13-14, 2023 ©2023 Association for Computational Linguistics CPIC at SemEval-2023 Task 7: GPT2-based Model for Multi-evidence Natural Language Inference for Clinical Trial Data Mingtong Huang, Junxiang Ren, Lang Liu, Ruilin Song, Wenbo Yin China Pacific Insurance (Group) Co. Ltd., China {huangmingtong, renjunxaing, liulang-009, songruilin, yinwenbo-001}@cpic.com.cn Abstract This paper describes our system submitted for SemEval Task 7, Multi-Evidence Natural Lan- guage Inference for Clinical Trial Data. The task consists of 2 subtasks. Subtask 1 is to de- termine the relationships between clinical trial data (CTR) and statements. Subtask 2 is to out- put a set of supporting facts extracted from the premises with the input of CTR premises and statements. Through experiments, we found that our GPT2-based pre-trained models can obtain good results in Subtask 2. Therefore, we use the GPT2-based pre-trained model to fine-tune Subtask 2. We transform the evidence retrieval task into a binary class task by com- bining premises and statements as input, and the output is whether the premises and state- ments match. We obtain a top-5 score in the evaluation phase of Subtask 2. 1 Introduction In recent years, there has been a significant increase in the number of Clinical Trial Reports (CTR) pub- lications. Currently, the number of CTRs related to breast cancer alone exceeds 10,000. With this increasing trend, it is impossible for clinical prac- titioners to keep all the existing literature up to date in the future in order to provide personalized, evidence-based care (DeYoung et al., 2020). In this scenario, Natural Language Inference (NLI) (Mac- cartney, 2009) provides a great opportunity to sup- port the large-scale interpretation and retrieval of medical evidence. A successful natural language inference system enables clinical practitioners to obtain the latest medical evidence for better person- alized evidence-based care (Sutton et al.). There- fore, the study of SemEval Task 7 (Jullien et al., 2023) is particularly meaningful to provide tech- nical ideas for the large-scale interpretation and retrieval of medical evidence. Task 7 is based on CTRs of breast cancer and contains two subtasks. Subtask 1 is Textual En- tailment, which aims to determine the relationship (implication and contradiction) between CTRs and statements. Subtask 2 is Evidence Retrieval, which outputs a set of supporting evidence extracted from the CTR premise with given CTR and statements to prove the prediction labels in Subtask 1. Our system focuses on Subtask 2 which reasons about the statements supported by the facts in the premises by means of the generative capability of GPT-2 (Radford et al., 2019). Firstly, our system concatenates the premises (i.e., facts in CTRs) and the inference(i.e., statements) through the prmopt template to obtain the inference sequence. Then the inference sequence is input into GPT-2, and the output label is 0 or 1. The label 1 means that the premise fact and the statement are successfully matched, so the fact can be added to the corre- sponding group of supporting facts. Finally, after the matching …

Content truncated to 500 words