Feature Stacking for Sentence Classification in Evidence-Based Medicine

Feature Stacking for Sentence Classification in Evidence-Based Medicine Marco Lui NICTA VRL Department of Computing and Information Systems University of Melbourne mhlui@unimelb.edu.au Abstract We describe the feature sets and methodol- ogy that produced the winning entry to the ALTA 2012 Shared Task (sentence classi- fication in evidence-based medicine). Our approach is based on a variety of feature sets, drawn from lexical and structural in- formation at the sentence level, as well as sequential information at the abstract level. We introduce feature stacking, a met- alearner to combine multiple feature sets, based on an approach similar to the well- known stacking metalearner. Our system attains a ROC area-under-curve of 0.972 and 0.963 on two subsets of test data. 1 Introduction The ALTA Shared Task 20121 was a sentence- level classification problem in the domain of biomedical abstracts. Given a collection of ab- stracts pre-segmented into discrete sentences, the task is to label each sentence according to one of 6 pre-defined classes. The dataset used was in- troduced by Kim et al. (2011), which also give a description of the classes and an analysis of their distribution. In this work, we will describe the winning entry, focusing on the feature sets and machine learning techniques used. The main contributions of this work are: (1) additional features to describe sentences for au- tomatic classification of sentences to support evi- dence based medicine beyond those of Kim et al. (2011), (2) a method for performing the task that does not use a sequential learning algorithm, and (3) a method to combine multiple feature sets that outperforms a standard concatenation approach. 1http://www.alta.asn.au/events/ sharedtask2012 2 Task Description The dataset of Kim et al. (2011) (hereafter re- ferred to as NICTA-PIBOSO) consists of 11616 sentences (10379 after headings are removed), manually annotated over the 6 PIBOSO classes (Kim et al., 2011). For the shared task, NICTA- PIBOSO was divided by the competition orga- nizers into train and test partitions. Participants were given labels for the training sentences, and asked to produce an automatic system to predict the labels of the test instances. We do not give further details of the task as it will be covered in much greater depth by the shared task organizers in a paper that will appear alongside this paper. The shared task was hosted on Kaggle,2 and as part of Kaggle’s standard competition structure, the test dataset was further subdivided into “pub- lic” and “private” subsets. Participants did not know which test sentence belonged to which sub- set. Each submission by a participant consisted of predictions over the entire test set, and Kag- gle then automatically computed the competition metric broken down over the public and private subsets. Participants were allowed to submit up to 2 entries per day, and upon submission were im- mediately given a score on the public subset. The score on the private subset was withheld until af- ter the conclusion of the submission period. Final ranking of competitors is based on the private sub- set of the …

Content truncated to 500 words