Targeted Sentiment to Understand Student Comments

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2471–2481, Osaka, Japan, December 11-17 2016. Targeted Sentiment to Understand Student Comments Charles Welch and Rada Mihalcea University of Michigan 2260 Hayward Street Ann Arbor, MI 48109, USA {cfwelch,mihalcea}@umich.edu Abstract We address the task of targeted sentiment as a means of understanding the sentiment that students hold toward courses and instructors, as expressed by students in their comments. We introduce a new dataset consisting of student comments annotated for targeted sentiment and describe a system that can both identify the courses and instructors mentioned in student comments, as well as label the students’ sentiment toward those entities. Through several comparative evaluations, we show that our system outperforms previous work on a similar task. 1 Introduction Sentiment analysis is the computational study of people’s opinions or emotions; it is a challenging prob- lem that is increasingly being used for decision making by individuals and organizations (Pang and Lee, 2008). There is a significant body of research on sentiment analysis, addressing entire documents (Agar- wal and Bhattacharyya, 2005), including blogs (Godbole et al., 2007; Annett and Kondrak, 2008) and reviews (Yi et al., 2003; Cabral and Hortacsu, 2010); sentences (Yu and Hatzivassiloglou, 2003; Nigam and Hurst, 2004) or otherwise short spans of texts such as tweets (Pak and Paroubek, 2010; Kouloumpis et al., 2011); and phrases (Wilson et al., 2005; Turney, 2002). More recent work has also addressed the task of aspect sentiment (Pontiki et al., 2015; Thet et al., 2010; Lakkaraju et al., 2014), which aims to address the sentiment toward attributes of the target entity, such as the service in a restaurant (Sauper and Barzilay, 2013), or the camera of a mobile phone (Chamlertwat et al., 2012). In this paper we address the task of targeted sentiment, defined as the task of identifying the sentiment (positive, negative) or lack thereof (neutral) that a writer holds toward entities mentioned in a statement. Targeted sentiment has been only recently introduced as a task, to our knowledge with contributions from only two research groups that focused primarily on settings with scarce resources (Mitchell et al., 2013; Zhang et al., 2015). While previous work on data sets such as product reviews can give an accurate measure of sentiment toward products (as explicit targets of the opinions being expressed in the reviews), some corpora include additional challenges. Targeted sentiment addresses the challenge of identifying entities in running text (e.g., Twitter, student comments), and attributing separate sentiment to each mentioned entity. In our work, we focus on an application-driven task, namely that of understanding students’ sentiment towards courses and instructors as expressed in their comments. As an example, consider the statement: (1) I thought that natural language processing with professor Doe was a great class. We want to recognize the targets “natural language processing” (a course) and “Doe” (an instructor), as well as a positive sentiment toward the course, and a neutral sentiment toward the instructor. We approach targeted sentiment as a …

Content truncated to 500 words