INESC-ID: Sentiment Analysis without Hand-Coded Features or Linguistic Resources using Embedding Subspaces

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 652–656, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics INESC-ID: Sentiment Analysis without hand-coded Features or Liguistic Resources using Embedding Subspaces Ramon F. Astudillo, Silvio Amir, Wang Ling, Bruno Martins†, M´ario Silva, Isabel Trancoso Instituto de Engenharia de Sistemas e Computadores Investigac¸˜ao e Desenvolvimento Rua Alves Redol 9 Lisbon, Portugal {ramon.astudillo, samir, wlin, mjs, isabel.trancoso}@inesc-id.pt †bruno.g.martins@tecnico.ulisboa.pt Abstract We present the INESC-ID system for the mes- sage polarity classification task of SemEval 2015. The proposed system does not make use of any hand-coded features or linguistic resources. It relies on projecting pre-trained structured skip-gram word embeddings into a small subspace. The word embeddings can be obtained from large amounts of Twitter data in unsupervised form. The sentiment analy- sis supervised training is thus reduced to find- ing the optimal projection which can be car- ried out efficiently despite the little data avail- able. We analyze in detail the proposed ap- proach and show that a competitive system can be attained with only a few configuration pa- rameters. 1 Introduction Web-based social networks are a rich data source for both businesses and academia. However, the sheer volume, diversity and rate of creation of social me- dia, imposes the need for automated analysis tools. The growing interest in this problem motivated the creation of a shared task for Twitter Sentiment Anal- ysis (Nakov et al., 2013). The Message Polarity Classification task consists in classifying a message as positive, negative, or neutral in sentiment. A great deal of research has been done on meth- ods for sentiment analysis on user generated con- tent. However, state-of-the-art systems still largely depend on linguistic resources, extensive feature en- gineering and tuning. Indeed, if we look at the best performing systems from SemEval 2014 (Zhu et al., 2014), (Malandrakis et al., 2014), both make exten- sive use of these resources, including hundreds of thousands of features, special treatment for nega- tion, multi-word expressions or special strings like emoticons. In this paper we present the INESC-ID system for the 2015 SemEval message polarity classifica- tion task (Rosenthal et al., 2015). The system is able to learn good message representations for message polarity classification directly from raw text with a simple tokenization scheme. Our approach is based on using large amounts of unlabeled data to induce word embeddings, that is, continuous word represen- tations containing contextual information. Instead of using these word embeddings directly with, for instance, a logistic regression classifier, we estimate a sentiment subspace of the embeddings. The idea is to find a projection of the embedding space that is meaningful for the supervised task. In the proposed model, we jointly learn the sentiment subspace pro- jection and the classifier using the SemEval train- ing data. The resulting system attains state-of-the- art performance without hand-coded features or lin- guistic resources and only a few configuration pa- rameters. 2 Unsupervised Learning of Word Embeddings Unsupervised word embeddings trained from large amounts of unlabeled data have …

Content truncated to 500 words