Columbia NLP: Sentiment Detection of Sentences and Subjective Phrases in Social Media

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 198–202, Dublin, Ireland, August 23-24, 2014. Columbia NLP: Sentiment Detection of Sentences and Subjective Phrases in Social Media Sara Rosenthal Dept. of Computer Science Columbia University New York, NY 10027, USA sara@cs.columbia.edu Apoorv Agarwal Dept. of Computer Science Columbia University New York, NY 10027, USA apoorv@cs.columbia.edu Kathleen McKeown Dept. of Computer Science Columbia University New York, NY 10027, USA kathy@cs.columbia.edu Abstract We present two supervised sentiment de- tection systems which were used to com- pete in SemEval-2014 Task 9: Senti- ment Analysis in Twitter. The first sys- tem (Rosenthal and McKeown, 2013) clas- sifies the polarity of subjective phrases as positive, negative, or neutral. It is tai- lored towards online genres, specifically Twitter, through the inclusion of dictionar- ies developed to capture vocabulary used in online conversations (e.g., slang and emoticons) as well as stylistic features common to social media. The second sys- tem (Agarwal et al., 2011) classifies entire tweets as positive, negative, or neutral. It too includes dictionaries and stylistic fea- tures developed for social media, several of which are distinctive from those in the first system. We use both systems to par- ticipate in Subtasks A and B of SemEval- 2014 Task 9: Sentiment Analysis in Twit- ter. We participated for the first time in Subtask B: Message-Level Sentiment De- tection by combining the two systems to achieve improved results compared to ei- ther system alone. 1 Introduction In this paper we describe two prior sentiment de- tection algorithms for social media. Both systems (Rosenthal and McKeown, 2013; Agarwal et al., 2011) classify the polarity of sentence phrases and This work is licensed under a Creative Commons At- tribution 4.0 International License. Page numbers and pro- ceedings footer are added by the organizers. License details: http://creativecommons.org/licenses/by/4.0/ tweets as positive, negative, or neutral. These al- gorithms were used to participate in the the expres- sion level task (Subtask A) and message level task (Subtask B) of the SemEval-2014 Task 9: Senti- ment Analysis in Twitter (Rosenthal et al., 2014) which one of the authors helped organize. We first show improved results compared to our participation in the prior year in the expression- level task (Subtask A) by incorporating a new dic- tionary and new features into the system. Our fo- cus this year was on Subtask B which we partici- pated in for the first time. We integrated two sys- tems to achieve improved results compared to ei- ther system alone. Our analysis shows that the first system performs better on recall while the second system performs better on precision. We used con- fidence metrics outputted by the systems to deter- mine which answer should be used. This resulted in a slight improvement in the Twitter dataset com- pared to either system alone. In this rest of this paper, we discuss related work, the methods for each system, and experiments and results for each subtask using the data provided by Semeval-2014 Task 9: Sentiment Analysis in …

Content truncated to 500 words