Does Multi-Task Learning Always Help?: An Evaluation on Health Informatics

Does Multi-Task Learning Always Help? An Evaluation on Health Informatics Tasks Aditya Joshi, Sarvnaz Karimi, Ross Sparks, C´ecile Paris, C Raina MacIntyre CSIRO Data61, Sydney, Australia Kirby Institute, University of New South Wales, Sydney, Australia {firstname.lastname}@csiro.au , r.macintyre@unsw.edu.au Abstract Multi-Task Learning (MTL) has been an at- tractive approach to deal with limited labeled datasets or leverage related tasks, for a vari- ety of NLP problems. We examine the benefit of MTL for three specific pairs of health in- formatics tasks that deal with: (a) overlapping symptoms for the same classification problem (personal health mention classification for in- fluenza and for a set of symptoms); (b) over- lapping medical concepts for related classifi- cation problems (vaccine usage and drug us- age detection); and, (c) related classification problems (vaccination intent and vaccination relevance detection). We experiment with a simple neural architecture: a shared layer fol- lowed by task-specific dense layers. The nov- elty of this work is that it compares alterna- tives for shared layers for these pairs of tasks. While our observations agree with the promise of MTL as compared to single-task learning, for health informatics, we show that the benefit also comes with caveats in terms of the choice of shared layers and the relatedness between the participating tasks. 1 Introduction Health informatics is the discipline concerned with the systematic processing of data, infor- mation and knowledge in medicine and health- care (Hasman, 1998). Health informatics tasks tend to be specific in terms of parameters such as symptoms, regions of interest or the phenomenon to be detected. As a result, datasets for different health informatics tasks have been reported. How- ever, it remains to be seen if these datasets or clas- sification tasks help each other in terms of how similar the participating datasets or tasks are. In this paper, we examine the utility of Multi-Task Learning (MTL) for several pairs of health infor- matics tasks that are related in different ways. MTL pertains to the class of learning algorithms that jointly train predictors for more than one task. In Natural Language Processing (NLP) research, MTL using deep learning has been used either to learn shared representations for related tasks, or to deal with limited labeled datasets (Xue et al., 2007; Zhang and Yeung, 2012; Søgaard and Gold- berg, 2016; Ruder, 2017; Liu et al., 2017) for a variety of NLP problems such as sentiment analy- sis (Huang et al., 2013; Mishra et al., 2017). Most of this work that uses MTL presents architectures utilising multiple shared and task-specific layers. In contrast, we wish to see if the benefit comes from the simplistic notion of ‘learning these clas- sifiers together’. Therefore, we use a basic archi- tecture for our MTL experiments consisting of a single shared layer and single task-specific lay- ers, and experiment with different alternatives for the shared layer. This simplicity allows us to un- derstand the benefit of MTL in comparison with Single-Task Learning (STL) for different configu- rations of shared layers, for task pairs that are …

Content truncated to 500 words