Hierarchical versus Flat Classification of Emotions in Text

Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 140–146, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Hierarchical versus Flat Classification of Emotions in Text Diman Ghazi (a), Diana Inkpen (a), Stan Szpakowicz (a, b) (a) School of Information Technology and Engineering, University of Ottawa (b) Institute of Computer Science, Polish Academy of Sciences {dghaz038,diana,szpak}@site.uottawa.ca Abstract We explore the task of automatic classifica- tion of texts by the emotions expressed. Our novel method arranges neutrality, polarity and emotions hierarchically. We test the method on two datasets and show that it outperforms the corresponding “flat” approach, which does not take into account the hierarchical informa- tion. The highly imbalanced structure of most of the datasets in this area, particularly the two datasets with which we worked, has a dramat- ic effect on the performance of classification. The hierarchical approach helps alleviate the effect. 1 Introduction Computational approaches to emotion analysis have focused on various emotion modalities, but there was only limited effort in the direction of automatic recognition of emotion in text (Aman, 2007). Oleveres et al.(1998), as one of the first works in emotion detection in text, uses a simple Natural Language Parser for keyword spotting, phrase length measurement and emoticon identification. They apply a rule-based expert system to construct emotion scores based on the parsed text and con- textual information. However their simple word- level analysis system is not sufficient when the emotion is expressed by more complicated phrases and sentences. More advanced systems for textual emotion recog- nition performed sentence-level analysis. Liu et al. (2003), proposed an approach aimed at understand- ing the underlying semantics of language using large-scale real-world commonsense knowledge to classify sentences into “basic” emotion categories. They developed a commonsense affect model enabling the analysis of the affective qualities of text in a robust way. In SemEval 2007, one of the tasks was carried out in an unsupervised setting and the emphasis was on the study of emotion in lexical semantics (Strappa- rava and Mihalcea, 2008; Chaumartin, 2007; Koza- reva et al., 2007; Katz et al., 2007). Neviarouskaya et al.(2009) applied a rule-based approach to affect recognition from a blog text. However, statistical and machine learning approaches have became a method of choice for constructing a wide variety of NLP applications (Wiebe et al., 2005). There has been previous work using statistical methods and supervised machine learning, includ- ing (Aman, 2007; Katz et al., 2007; Alm, 2008; Wilson et al., 2009). Most of that research concen- trated on feature selections and applying lexical semantics rather than on different learning schemes. In particular, only flat classification has been considered. According to Kiritchenko et al. (2006), “Hierar- chical categorization deals with categorization problems where categories are organized in hierar- chies”. Hierarchical text categorization places new items into a collection with a predefined hierar- chical structure. The categories are partially or- dered, usually from more generic to more specific. Koller and Sahami (1997) carried out …

Content truncated to 500 words