University_of_Warwick: SENTIADAPTRON - A Domain Adaptable Sentiment Analyser for Tweets - Meets SemEval

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 768–772, Dublin, Ireland, August 23-24, 2014. University of Warwick: SENTIADAPTRON - A Domain Adaptable Sentiment Analyser for Tweets - Meets SemEval Richard Townsend University of Warwick Richard.Townsend@warwick.ac.uk Aaron Kalair University of Warwick Aaron.Kalair@warwick.ac.uk Ojas Kulkarni University of Warwick Ojas.Kulkarni@warwick.ac.uk Rob Procter University of Warwick Rob.Procter@warwick.ac.uk Maria Liakata University of Warwick M.Liakata@warwick.ac.uk Abstract We give a brief overview of our system, SentiAdaptron, a domain-sensitive and do- main adaptable system for twitter analy- sis in tweets, and discuss performance on SemEval (in both the constrained and un- constrained scenarios), as well as implica- tions arising from comparing the intra- and inter- domain performance on our twitter corpus. 1 Introduction A domain is broadly defined as a set of documents demonstrating a similar distribution of words and linguistic patterns. Task 9 of SemEval treats Twit- ter as a single domain with respect to sentiment analysis. However previous research has argued for the topic-specific treatment of sentiment given domain-specific nuances and the over-generality of current sentiment analysis systems with respect to applications in the social sciences (Thelwall and Buckley, 2013). Thelwal’s method - manually ex- tending a sentiment lexicon for a particular topic or domain - highlights that expression of senti- ment varies from one domain to another. Rather than relying on the manual extension of lexica, we developed an approach to Twitter sentiment clas- sification that is domain sensitive. To this effect we gathered tweets from three primary domains - financial news, political opinion, technology com- panies and their products - and trained our system on one domain while adapting to the other. Us- ing this methodology we obtained both intrinsic as well as extrinsic evaluation of the system on real world applications with promising results. As our approach to sentiment analysis has been influ- enced by the task description of SemEval 2013 we This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ Positive Negative Neutral Objective SemEval 11610 6332 905 189887 Our corpus 10725 17837 3514 36904 (a) Contextual polarity. Positive Negative Neutral Objective SemEval 4215 1798 4082 1243 Our corpus 1090 1711 1191 - (b) Message polarity. Table 1: The distribution of sentiment classes be- tween SemEval and our corpus at word and tweet level. decided to also evaluate the system on SemEval’s data, since it provides a well established bench- mark. In the following we briefly describe our system and corpus and discuss our approach for the SemEval submission. 2 A Corpus of Three Domains: Source of Unconstrained Data Our goal in developing SentiAdaptron was domain adaptive tweet-level classification. We decided to follow SemEval 2013 and collect both word-level as well as message level annotations. We prepared a corpus of 4000 tweets with a balanced cover- age of financial, political and technology related tweets. Tweets were collected using keywords from domain-specific websites: the final list was chosen after evaluating each …

Content truncated to 500 words