SWATCS65: Sentiment Classification Using an Ensemble of Class Projects

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 631–635, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics SWATCS65: Sentiment Classification Using an Ensemble of Class Projects Richard Wicentowski Swarthmore College 500 College Avenue Swarthmore, PA 19081 richardw@cs.swarthmore.edu Abstract This paper presents the SWATCS65 ensem- ble classifier used to identify the sentiment of tweets. The classifier was trained and tested using data provided by Semeval-2015, Task 10, subtask B with the goal to label the sen- timent of an entire tweet. The ensemble was constructed from 26 classifiers, each written by a group of one to three undergraduate stu- dents in the Fall 2014 offering of a natural lan- guage processing course at Swarthmore Col- lege. Each of the classifiers was designed in- dependently, though much of the early struc- ture was provided by in-class lab assignments. There was high variability in the final perfor- mance of each of these classifiers, which were combined using a weighted voting scheme with weights correlated with performance us- ing 5-fold cross-validation on the provided training data. The system performed very well, achieving an F1 score of 61.89. 1 Introduction Workshops designed around competitions such as Semeval-2015 provide an excellent entry-point for undergraduate students to work on real-world prob- lems in the field by providing both the training and test data as well as a framework for comparing their work to the state-of-the-art. These competitions have a low barrier to entry while also providing students with an external motivation to continually improve their systems. As part of the Fall 2015 offering of CPSC 065 at Swarthmore College1, undergraduate students en- rolled in the class were required to build a classifier for Semeval-2015 Task 10, subtask B (Rosenthal et al., 2015). The goal of this task was to provide a labeling of the sentiment expressed in a tweet: either negative, neutral or positive. Fifty-one students were enrolled in the class and each student worked in a small group. Of the 26 groups, 23 were comprised of two students, one group had three students, and two had only one student. Approximately 35% of the students in the class (18 of 51) took this class as their first upper- level course in the discipline, having completed only the equivalents of CS1 and CS2 prior to this class. The classifiers were developed over a seven week period beginning in the eighth week of the course. 2 Required Components Each group was provided with boilerplate code to read in the tweets and were tasked with writing a Naive Bayes classifier to label each of the tweets. In the first two weeks, groups were required to first evaluate their system using five-fold cross- validation without any preprocessing of the tweets using only unigrams. Then they compared those results to those obtained after performing a few basic preprocessing steps (removal of stopwords, case-folding, and simple handling of negation) and 1http://goo.gl/ydgE5r 631 tokenization using Twokenizer (Owoputi et al., 2013). In the third week, students read three papers …

Content truncated to 500 words