Ensemble Classification of Grants using LDA-based Features

Ensemble Classification of Grants using LDA-based Features Ioannis Korkontzelos∗, Beverley Thomas†, Makoto Miwa‡, Sophia Ananiadou∗ ∗National Centre for Text Mining, School of Computer Science, University of Manchester, UK {ioannis.korkontzelos, sophia.ananiadou}@manchester.ac.uk †Biotechnology and Biological Sciences Research Council (BBSRC), Polaris House, Swindon, UK beverley.thomas@bbsrc.ac.uk ‡Toyota Technological Institute, Nagoya, Japan makoto-miwa@toyota-ti.ac.jp Abstract Classifying research grants into useful categories is a vital task for a funding body to give structure to the portfolio for analysis, informing strategic planning and decision-making. Automating this classification process would save time and effort, providing the accuracy of the classifications is maintained. We employ five classification models to classify a set of BBSRC-funded research grants in 21 research topics based on unigrams, technical terms and Latent Dirichlet Allocation models. To boost precision, we investigate methods for combining their predictions into five aggregate classifiers. Evaluation confirmed that ensemble classification models lead to higher precision. It was observed that there is not a single best-performing aggregate method for all research topics. Instead, the best-performing method for a research topic depends on the number of positive training instances available for this topic. Subject matter experts considered the predictions of aggregate models to correct erroneous or incomplete manual assignments. Keywords: research grant classification, document classification, topic models 1. Introduction On a worldwide scale, funding bodies fund thousands of research grants per year. In the United Kingdom, national funding bodies are mostly organised by discipline in re- search councils. For example, the Biotechnology and Bi- ological Sciences Research Council (BBSRC) funds re- search in biotechnology and biology, the Medical Research Council (MRC) funds research in medicine, the Engi- neering and Physical Sciences Research Council (EPSRC0 funds research in engineering and physical sciences etc. BBSRC processes, on average, over 1600 research grant applications per year. Classifications give structure to research portfolios, en- abling timely, accurate portfolio analysis, to inform strate- gic planning and decision making, whilst also assisting in the process of peer review. Currently classification is a manual and subjective process taking considerable time and effort. Being able to assign classifications automatically will enable greater accuracy, consistency and timelines and has the potential to be used across a broad range of funding mechanisms. In addition, automatic classification offers the potential to amend the set of topics flexibly. The efficiency savings can free up capacity to perform increasingly sophis- ticated analyses. However, automatic assignment is only meaningful if the associated error rate is very limited, even if no assignment can be produced for some grants. Increas- ing classification accuracy requires that enough manually annotated data is available, to avoid error propagation and ensure that the resulting classification model will be able to generalise. The current study showed that automatic classi- fications were useful in correcting manual assignments. In this study, we address the task of classifying grants into research topics. We combine a variety of classifiers whose features encode the words and technical terms in the de- scription of funded grants as well as Latent Dirichlet Allo- cation (LDA) topics. In particular, for each research …

Content truncated to 500 words