NCUEE-NLP at SemEval-2023 Task 7: Ensemble Biomedical LinkBERT Transformers in Multi-evidence Natural Language Inference for Clinical Trial Data

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 776–781 July 13-14, 2023 ©2023 Association for Computational Linguistics Abstract This study describes the model design of the NCUEE-NLP system for the SemEval- 2023 NLI4CT task that focuses on multi- evidence natural language inference for clinical trial data. We use the LinkBERT transformer in the biomedical domain (denoted as BioLinkBERT) as our main system architecture. First, a set of sentences in clinical trial reports is extracted as evidence for premise-statement inference. This identified evidence is then used to determine the inference relation (i.e., entailment or contradiction). Finally, a soft voting ensemble mechanism is applied to enhance the system performance. For Subtask 1 on textual entailment, our best submission had an F1-score of 0.7091, ranking sixth among all 30 participating teams. For Subtask 2 on evidence retrieval, our best result obtained an F1-score of 0.7940, ranking ninth of 19 submissions. 1 Introduction Natural Language Inference (NLI) seeks to determine whether a given hypothesis is true (i.e., entailment), false (contradiction), or undetermined (neutral) according to a known premise. Methods used for NLI task range from earlier symbolic and statistical approaches to recent deep-learning- based models (Storks et al., 2019). The BERT- BiLSTM-Attention model was proposed for medical text inference (Lee et al., 2019). A knowledge adaptive approach was derived to encode the premise/hypothesis text for improving medical NLI (Chowdhury et al., 2020). Structured domain knowledge from the Unified Medical Language System (UMLS) was incorporated into the ESIM model (Chen et al., 2017) to improve NLI performance in the medical domain (Sharma et al., 2019). Clinical domain knowledge of BERT models (Devlin et al., 2019) was explored using lexical retrieval, syntactic retrieval, and classification models for external knowledge integration (Sushil et al., 2021). The SemEval-2023 Task 7 focuses on multi- evidence natural language inference for clinical trial data (denoted as NLI4CT) (Jullien et al., 2023). The NLI4CT task is mainly based on a collection of Clinical Trial Reports (CTR) for breast cancer, with statements, explanations, and labels annotated by domain expert annotators. It includes two subtasks: 1) Textual Entailment: determining the inference relation (i.e., entailment or contradiction) between CTR-statement pairs; 2) Evidence Retrieval: extracting a set of supporting facts in a given CTR premise to justify the label predicted in Subtask 1. The MEDIQA-2019 shared task covers an NLI subtask in the medical domain (Ben Abacha et al., 2019), in which most systems built on the BERT model are pre-trained on a large-scale open domain corpus and its variants focus on domain-specific knowledge such as SciBERT (Beltagy et al., 2019), BioBERT (Lee et al., 2020), and ClinicalBERT (Alsentzer et al., 2019). Given the promising results obtained by most such approaches, we apply BERT-like neural networks to the NLI task in the clinical domain. This paper describes the NCUEE-NLP (National Central University, Dept. of Electrical Engineering, Natural Language Processing Lab) system for SemEval-2023 NLI4CT task. We use biomedical LinkBERT (Yasunaga et al., 2022) as our main system architecture to first extract a …

Content truncated to 500 words