Joint Emotion Analysis via Multi-task Gaussian Processes

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1798–1803, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Joint Emotion Analysis via Multi-task Gaussian Processes Daniel Beck† Trevor Cohn‡ Lucia Specia† †Department of Computer Science, University of Sheffield, United Kingdom {debeck1,l.specia}@sheffield.ac.uk ‡Computing and Information Systems, University of Melbourne, Australia t.cohn@unimelb.edu.au Abstract We propose a model for jointly predicting multiple emotions in natural language sen- tences. Our model is based on a low-rank coregionalisation approach, which com- bines a vector-valued Gaussian Process with a rich parameterisation scheme. We show that our approach is able to learn correlations and anti-correlations between emotions on a news headlines dataset. The proposed model outperforms both single- task baselines and other multi-task ap- proaches. 1 Introduction Multi-task learning (Caruana, 1997) has been widely used in Natural Language Processing. Most of these learning methods are aimed for Do- main Adaptation (Daum´e III, 2007; Finkel and Manning, 2009), where we hypothesize that we can learn from multiple domains by assuming sim- ilarities between them. A more recent use of multi-task learning is to model annotator bias and noise for datasets labelled by multiple annotators (Cohn and Specia, 2013). The settings mentioned above have one aspect in common: they assume some degree of posi- tive correlation between tasks. In Domain Adap- tation, we assume that some “general”, domain- independent knowledge exists in the data. For an- notator noise modelling, we assume that a “ground truth” exists and that annotations are some noisy deviations from this truth. However, for some set- tings these assumptions do not necessarily hold and often tasks can be anti-correlated. For these cases, we need to employ multi-task methods that are able to learn these relations from data and correctly employ them when making predictions, avoiding negative knowledge transfer. An example of a problem that shows this be- haviour is Emotion Analysis, where the goal is to automatically detect emotions in a text (Strappa- rava and Mihalcea, 2008; Mihalcea and Strappa- rava, 2012). This problem is closely related to Opinion Mining (Pang and Lee, 2008), with sim- ilar applications, but it is usually done at a more fine-grained level and involves the prediction of a set of labels (one for each emotion) instead of a single label. While we expect some emotions to have some degree of correlation, this is usually not the case for all possible emotions. For instance, we expect sadness and joy to be anti-correlated. We propose a multi-task setting for Emotion Analysis based on a vector-valued Gaussian Pro- cess (GP) approach known as coregionalisation ( ´Alvarez et al., 2012). The idea is to combine a GP with a low-rank matrix which encodes task corre- lations. Our motivation to employ this model is three-fold: • Datasets for this task are scarce and small so we hypothesize that a multi-task approach will results in better models by allowing a task to borrow statistical strength from other tasks; • The annotation scheme is subjective and very fine-grained, and …

Content truncated to 500 words