Overview of the 2022 ALTA Shared task: PIBOSO sentence classification, 10 years later

Overview of the 2022 ALTA Shared task: PIBOSO sentence classification, 10 years later Diego Moll´a School of Computing Macquarie University Sydney, Australia diego.molla-aliod@mq.edu.au Abstract The 2022 ALTA shared task has been running annually since 2010. This year, the shared task is a re-visit of the 2012 ALTA shared task. The purpose of this task is to classify sentences of medical publications using the PIBOSO taxon- omy. This is a multi-label classification task which can help medical researchers and prac- titioners conduct Evidence Based Medicine (EBM). In this paper we present the task, the evaluation criteria, and the results of the sys- tems participating in the shared task. 1 Introduction Within the practice of Evidence Based Medicine (EBM), the medical practitioner integrates individ- ual clinical expertise with the best external evi- dence at point of care (Sackett et al., 1996). Find- ing the best available evidence, however, is increas- ingly difficult given the large amount of medical publications. For example, at the time of writ- ing, PubMed contains more than 34 million ci- tations for biomedical literature1. From 2020 to present, CORD-19, a resource of medical publi- cations about COVID-19, SARS-COV-2, and re- lated coronaviruses, has increased from an initial set of 28,000 papers (Wang et al., 2020) to over 1,000,0002. To assist with the task of finding the best avail- able evidence, best EBM practice suggests users to formulate queries that focus on specific aspects of the clinical information sought. PIBOSO (Kim et al., 2011) is a pre-defined set of such aspects of clinical information, and systems participating in the 2012 ALTA shared task classified sentences 1https://pubmed.ncbi.nlm.nih.gov/, ac- cessed on 15 November 2022. 2https://www.kaggle.com/ datasets/allen-institute-for-ai/ CORD-19-research-challenge, accessed on 15 November 2022. from medical publications into PIBOSO labels (Amini et al., 2012). In 2022, 10 years later, ALTA has re-visited the task, to find out whether recent advances in machine learning would allow to im- prove the quality of such classifiers. This paper presents the results of systems partic- ipating in the 2022 ALTA shared task. Section 2 describes the PIBOSO taxonomy. Section 3 briefly mentions related work between the 2012 and the 2022 ALTA shared tasks. Section 4 describes the evaluation framework. Section 5 presents two sim- ple baselines that were made available to the par- ticipating teams. Section 6 presents the results and briefly describes the methods of participating sys- tems. Finally, Section 7 concludes this paper. 2 PIBOSO EBM guidelines recommend the use of structured queries that focus on specific aspects of clinical information (Richardson et al., 1995). One of the most widely used systems is PICO, which defines 4 types of information: Population, for example the number and type of participants in a study; Intervention, such as the treatment applied to the population; Comparison (if appropriate), for ex- ample alternative interventions or placebo; and Outcome of an intervention. Different variants and extensions of PICO have been proposed. The ALTA 2012 and 2022 shared tasks use PIBOSO (Kim et al., 2011). This schema removes the Comparison tag and …

Content truncated to 500 words