Enhancing the DeBERTa Transformers Model for Classifying Sentences from Biomedical Abstracts

Enhancing the DeBERTa Transformers Model for Classifying Sentences from Biomedical Abstracts Abdul Aziz, Md. Akram Hossain, and Abu Nowshed Chy Department of Computer Science and Engineering University of Chittagong, Chattogram-4331, Bangladesh {aziz.abdul.cu, akram.hossain.cse.cu}@gmail.com, and nowshed@cu.ac.bd Abstract Evidence-based medicine (EBM) is defined as making clinical decisions about individual pa- tients based on the best available evidence. It is beneficial for making better clinical decisions, caring for patients, and providing information about the therapy, prognosis, diagnosis, and other health care issues. However, it is a chal- lenging task to build an automatic sentence classifier for EBM owing to a lack of clinical context, uncertainty in medical knowledge, dif- ficulty in finding the best evidence, and domain- specific words in medical articles. To address these challenges, ALTA 2022 introduced a task to build automatic sentence classifiers for EBM that can map the content of biomedical ab- stracts into a set of pre-defined categories. This paper presents our participation in this task where we propose a transformers-based classi- fication approach to identify the category of the content from biomedical abstracts. We perform fine-tuning on DeBERTa pre-trained transformers model to extract the contextual- ized features representation. Later, we em- ploy a multi-sample dropout strategy and 5-fold cross-fold training to predict the more accurate class labels. Experimental results show that our proposed method achieved competitive perfor- mance among the participants. 1 Introduction Personalized medicine based on the context of pri- mary clinical evidence has become one of the most engaging and promising tasks in biomedical re- search. To suggest personalized medicine, prac- titioners require to study a lot of publications of medical science related to patient diagnosis. This kind of study is known as evidence-based medicine (EBM) (Masic et al., 2008) where the decision is taken based on some control traits and evidence **The first two authors have equal contributions. including Population (P), Intervention (I), Compar- ison (C), and Outcome (O), in short PICO. To automate the EBM process, (Kim et al., 2011) explored a classification task where the sentences are collected from the medical abstracts. As an expansion of this work, ALTA 2022 1 organized a shared task where they address the control traits as PIBOSO by the inclusion of three new classes including Background (B), Study Design (S), and Other (O) to improve the search performance. Here, Other (O) refers to the sentence with irrelevant content. To demonstrate a clear view of the task definition, we articulate a few examples in Table 1. Sentence Label The aim of this non-randomized study is to evaluate a group of patients treated by VP and KP procedures and to discuss related risks. [0 0 1 0 1 0] We evaluated drug effect through physical examinations and symp- tom scales. [0 0 0 0 0 1] Table 1: Example of ALTA 2022 task . Here, labels are population, intervention, background, outcome, study design, and other. The 0 and 1 in the label field denotes its existence in the corresponding sentence. However, the ambiguous clinical context, the randomness of …

Content truncated to 500 words