Annotating Opinions and Opinion Targets in Student Course Feedback

Annotating Opinions and Opinion Targets in Student Course Feedback Janaka Chathuranga, Shanika Ediriweera, Ravindu Hasantha, Pranidhith Munasinghe and Surangika Ranathunga Department of Computer Science and Engineering University of Moratuwa, Katubedda 10400, Sri Lanka {janaka.13, shanika.13, ravindu.13, pranidhith.13, surangika}@cse.mrt.ac.lk Abstract In this paper, we present a student course feedback corpus, a novel resource for opinion target extraction and sentiment analysis. The corpus is developed with the main aim of summarizing general feedback given by students on undergraduate-level courses. In this corpus, opinion targets, opinion expressions, and polarities of the opinion expressions towards the opinion targets are annotated. Opinion targets are basically the important key points in feedback that the students have shown their sentiment towards, such as “Lecture Slides”, and “Teaching Style”. The uniqueness of the corpus, annotation methodology, difficulties faced during annotating, and possible usages of the corpus are discussed in this paper. Keywords: Corpus Annotation, General Student Feedback, Opinion Targets, Summarization, Sentiment 1. Introduction and Motivation Student feedback is widely used in present in order to enhance the quality of teaching and learning process. Feedback is collected from students as online forms, as well as handwritten documents. Since it takes a considerable effort to read and understand all the feedback given by the students, the best way is to read all the feedback and create a summary that covers all the aspects of the given feedback. Our intention was to design a system to automatically summarize student feedback for different aspects of a course, such as teaching style of the lecturer, learning environment, and presentation slides. The summarization process contains 3 phases. 1. Identifying and extracting all the opinion targets in the given feedback. 2. Clustering the opinion targets into unique categories. 3. Determining the sentimental polarity of the targets and getting a statistic of polarity for each target cluster. In order to extract opinions and targets, the most promising technique is supervised learning (Luo & Litman, 2015; Luo, Liu, & Litman, 2016; Luo, Liu, W., Liu, F., & Litman, 2016). In order to apply the supervised machine learning techniques, we need a student course feedback corpus annotated with a suitable annotation scheme. Our data corpus consists of student responses collected from an undergraduate Computer Science and Engineering course. General responses were collected from 20 lectures and workshops. They contain 973 student responses in total with 2,395 sentences. In the annotation scheme, we annotate the opinion target and the opinion expression that shows the polarity of the target. For example, in the sentence “The lecture is really good”, we annotate “lecture” as the opinion target and “really good” as the opinion expression. Polarity of the target “lecture” becomes positive by the annotated opinion expression. We used the BIO (Beginning-Inside-Outside) (Sang & Veenstra, 1999) sequence labelling scheme to tag the annotated word phrases. BIO tagging includes “B” tags for the first word in a phrase, “I” tags for the other annotated words inside the phrase, and “O” tags for the words that are not annotated. We used BIO tagging because …

Content truncated to 500 words