Aicyber at SemEval-2016 Task 4: i-vector based sentence representation

Proceedings of SemEval-2016, pages 120–125, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics Aicyber at SemEval-2016 Task 4: i-vector based sentence representation Steven Du, Zhang Xi www.aicyber.com No. 39 Nanjing Road, Capitaland International Trade Center Block B 1604 Tianjin China {steven,zhangxi}@aicyber.com Abstract This paper introduces aicyber’s systems for SemEval 2016 , Task 4A. The first system is build on vector space model (VSM), the second system is build on a new framework to estimate sentence vector, it is inspired by the i-vector in speaker verification domain. Both systems are evaluated on SemEval 2016 (Task4A) as well as IMDB dataset. Evaluation results show that the i-vector based sentence vector is an alternative approach to present sentence. 1 Introduction The SemEval 2016 Task 4 is sentiment analysis in tweets. The subtask, task A focused on classifying tweets into three classes: positive, negative or neu- tral sentiment (Nakov et al., 2016). This paper will first presents the submitted sys- tem used by team aicyber. Then a new framework of estimating sentence vector will be introduced and evaluated. 2 The aicyber system This section introduces the submitted system for team aicyber. The text data is first being processed by tweet tokenizer, emoticons are preserved as to- kens. Bag-of-ngram feature is extracted and filtered by a TF-IDF (Salton, 1991) selection. Resulting fea- ture dimension is around 3800, it is then reduced to 400 by truncated singular value decomposition (SVD) (Klema and Laub, 1980; Halko et al., 2009). This process is also known as Latent Semantic Anal- ysis (LSA) or Vector Space Model (Turney and Pan- tel, 2010). Finally a Linear Discriminant Analysis (LDA) classifier (Hastie et al., 2009) is trained to classify the test data. The SemEval 2016 training dataset which con- tains 3887 tweets are selected to train the TF-IDF, SVD and LDA. Development dataset is used for tun- ing parameters and develop-test dataset are used for local testing. Task A adopted Macro-F1 measure as evaluation metric (Nakov et al., 2016): F PN 1 = F Pos 1 + F Neg 1 2 (1) The Macro-F1 for Aicyber system measured on development, develop-test and 2016 Tweet set, are respectively 0.4514, 0.4787 and 0.4024. It is obvious that this classification problem has not been satisfactorily answered. One possible rea- son is the unbalanced training data causes system bias towards positive classes. There are 2101 posi- tive, 1292 neutral but only 494 negative tweets. Another reason is the size of labeled training set is too small,with only 3887 tweets, which could hardly cover a reasonable amount of words. As a result, the bag-of-ngarm features learned from this training set, could not generalized well. This motivate us to seeking alternative feature representation of tweet, that is sentence vector. 3 Sentence vector (Le and Mikolov, 2014) proposed sentence vec- tor or paragraph vector (PV) which could learn the continuous distributed vector representation for 120 text of variable-length and achieved promising re- sult on movie review texts. It is inspired by word2vec (Mikolov et al., 2013) …

Content truncated to 500 words