Automatic Summarization of Student Course Feedback

Proceedings of NAACL-HLT 2016, pages 80–85, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Automatic Summarization of Student Course Feedback Wencan Luo† Fei Liu‡ Zitao Liu† Diane Litman† †University of Pittsburgh, Pittsburgh, PA 15260 ‡University of Central Florida, Orlando, FL 32716 {wencan, ztliu, litman}@cs.pitt.edu feiliu@cs.ucf.edu Abstract Student course feedback is generated daily in both classrooms and online course discussion forums. Traditionally, instructors manually analyze these responses in a costly manner. In this work, we propose a new approach to sum- marizing student course feedback based on the integer linear programming (ILP) frame- work. Our approach allows different student responses to share co-occurrence statistics and alleviates sparsity issues. Experimental results on a student feedback corpus show that our approach outperforms a range of baselines in terms of both ROUGE scores and human eval- uation. 1 Introduction Instructors love to solicit feedback from students. Rich information from student responses can reveal complex teaching problems, help teachers adjust their teaching strategies, and create more effective teaching and learning experiences. Text-based stu- dent feedback is often manually analyzed by teach- ing evaluation centers in a costly manner. Albeit useful, the approach does not scale well. It is there- fore desirable to automatically summarize the stu- dent feedback produced in online and offline envi- ronments. In this work, student responses are col- lected from an introductory materials science and engineering course, taught in a classroom setting. Students are presented with prompts after each lec- ture and asked to provide feedback. These prompts solicit “reflective feedback” (Boud et al., 2013) from the students. An example is presented in Table 1. Prompt Describe what you found most interesting in today’s class Student Responses S1: The main topics of this course seem interesting and correspond with my major (Chemical engineering) S2: I found the group activity most interesting S3: Process that make materials S4: I found the properties of bike elements to be most interesting S5: How materials are manufactured S6: Finding out what we will learn in this class was interesting to me S7: The activity with the bicycle parts S8: “part of a bike” activity ... (rest omitted, 53 responses in total.) Reference Summary - group activity of analyzing bicycle’s parts - materials processing - the main topic of this course Table 1: Example student responses and a reference summary created by the teaching assistant. ‘S1’–‘S8’ are student IDs. In this work, we aim to summarize the student re- sponses. This is formulated as an extractive sum- marization task, where a set of representative sen- tences are extracted from student responses to form a textual summary. One of the challenges of sum- marizing student feedback is its lexical variety. For example, in Table 1, “bike elements” (S4) and “bi- cycle parts” (S7), “the main topics of this course” (S1) and “what we will learn in this class” (S6) are different expressions that communicate the same or similar meanings. In fact, we observe 97% of the bi- grams appear only once or twice in …

Content truncated to 500 words