HLP@UPenn at SemEval-2017 Task 4A: A simple, self-optimizing text classification system combining dense and sparse vectors

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 640–643, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics HLP@UPenn at SemEval-2017 Task 4A: A simple, self-optimizing text classification system combining dense and sparse vectors Abeed Sarker Graciela Gonzalez Institute of Biomedical Informatics Department of Biostatistics, Epidemiology and Informatics The Perelman School of Medicine University of Pennsylvania {abeed,gragon}@upenn.edu Abstract We present a simple supervised text clas- sification system that combines sparse and dense vector representations of words, and the generalized representations of words via clusters. The sparse vectors are gen- erated from word n-gram sequences (1- 3). The dense vector representations of words (embeddings) are learned by train- ing a neural network to predict neighbor- ing words in a large unlabeled dataset. To classify a text segment, the different vector representations of it are concatenated, and the classification is performed using Sup- port Vector Machines (SVMs). Our sys- tem is particularly intended for use by non- experts of natural language processing and machine learning, and, therefore, the sys- tem does not require any manual tuning of parameters or weights. Given a train- ing set, the system automatically gener- ates the training vectors, optimizes the rel- evant hyper-parameters for the SVM clas- sifier, and trains the classification model. We evaluated this system on the SemEval- 2017 English sentiment analysis task. In terms of average F1-Score, our system ob- tained 8th position out of 39 submissions (F1-Score: 0.632, average recall: 0.637, accuracy: 0.646). 1 Introduction Text classification is one of the most fundamen- tal natural language processing tasks, and involves the categorization of texts based on their lexical contents. In its simplest form, text classification is binary in nature, such as the categorization of spam vs. non-spam email (Youn and McLeod, 2007). Researchers from distinct fields are ex- posed to a wide range of text classification prob- lems. Even within a specific domain, such as the medical domain, there is a multitude of text clas- sification tasks and problems, such as assessing the qualities of published papers (Kilicoglu et al., 2009; Sarker et al., 2015), outcome polarity clas- sification (Sarker et al., 2011), biomarker classifi- cation (Davis et al., 2015), and adverse drug re- action mention detection (Sarker and Gonzalez, 2015) to name a few. Early automated text clas- sification systems were rule-based in nature (e.g., Sarker and Moll´a-Aliod (2010)) mostly because of the absence of sufficient annotated data. However, such rule-based systems are generally limited in terms of performance and/or overfit to the target problem. They particularly suffer in terms of per- formance when exposed to unseen datasets. With the rapid increase in text-based data in all domains (e.g., the largest medical database, Medline,1 now indexes over 23 million articles), most efficient text classification systems now use machine learn- ing. Such systems utilize annotated data, and au- tomatically extract features from large volumes of annotated text to perform classification. The rules that are used in classification are not hard-coded or predetermined, but are learned from the …

Content truncated to 500 words