NER for Medical Entities in Twitter using Sequence to Sequence Neural Networks

NER for Medical Entities in Twitter using Sequence to Sequence Neural Networks Antonio Jimeno Yepes♦ and Andrew MacKinlay♦♠ ♦ IBM Research – Australia, Melbourne, VIC, Australia ♠ Dept of Computing and Information Systems, University of Melbourne, Australia {antonio.jimeno,admackin}@au1.ibm.com Abstract Social media sites such as Twitter are attractive sources of information due to their combination of accessibility, timeli- ness and large data volumes. Identifica- tion of medical entities in Twitter can sup- port tasks such public health surveillance. We propose an approach to perform anno- tation of medical entities using a sequence to sequence neural network. Results show that our approach improves over previous work based on CRF in the annotation of two medical entities types in Twitter. 1 Introduction Public health surveillance (Nsubuga et al., 2006) is the systematic collection, analysis and monitor- ing of population health for the public good using a variety of tools. Governments rely mostly on aggregated health data from health care centres, which cannot be used in real-time scenarios such as syndromic surveillance. Twitter is a promising source of information due to its availability and the large quantity of informa- tion published, with more than 500 million posts published each day. It can be potentially used to mine information about the status of the popula- tion (Jimeno Yepes et al., 2015a), and can then be used in combination with other sources to em- power government decision makers. Natural language processing can leverage the short messages in Twitter by extracting pieces of structured information from them which can be aggregated for data analysis. Identifying entities of interest in tweets is relevant for several tasks, including public health surveillance. Several ap- proaches have been used to perform named en- tity recognition in Twitter, which range from dic- tionary matching approaches to machine learn- ing methods. Named entity recognition (NER) in Twitter has unique challenges compared to many other sources of text. Tweets are short (with at most 140 characters), and often contain highly in- formal language, idioms, humour, typos and gram- matical errors (Baldwin et al., 2013; Jimeno Yepes et al., 2015b). Effective biomedical NER methods for Twitter rely on machine learning methods such as condi- tional random fields (CRF) (Lafferty et al., 2001). Learning algorithms like CRF typically consider a limited span of one token before and/or after the token being annotated, which may omit valuable contextual information in order to work within the limitations of the learning algorithms. We use a recurrent neural network under the as- sumption that it can manage dependencies in lan- guage better than traditional methods such as CRF. The proposed approach uses a sequence to se- quence network (Sutskever et al., 2014) to analyse text in a tweet producing an encoding vector which is then used to perform the annotation by a second LSTM (Long Short Term Memory) node. Word embeddings generated from several million tweets are used to represent the tokens in text, which is the only feature engineering required in this neu- ral network approach. This approach improves …

Content truncated to 500 words