Summarizing Student Responses to Reflection Prompts

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1955–1960, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Summarizing Student Responses to Reflection Prompts Wencan Luo Computer Science Department University of Pittsburgh Pittsburgh, PA 15260, USA wencan@cs.pitt.edu Diane Litman Computer Science Department and LRDC University of Pittsburgh Pittsburgh, PA 15260, USA litman@cs.pitt.edu Abstract We propose to automatically summarize student responses to reflection prompts and introduce a novel summarization algo- rithm that differs from traditional methods in several ways. First, since the linguis- tic units of student inputs range from sin- gle words to multiple sentences, our sum- maries are created from extracted phrases rather than from sentences. Second, the phrase summarization algorithm ranks the phrases by the number of students who semantically mention a phrase in a sum- mary. Experimental results show that the proposed phrase summarization ap- proach achieves significantly better sum- marization performance on an engineering course corpus in terms of ROUGE scores when compared to other summarization methods, including MEAD, LexRank and MMR. 1 Introduction Educational research has demonstrated the effec- tiveness of reflection prompts (Boud et al., 2013) to enhance interaction between instructors and stu- dents (Van den Boom et al., 2004; Menekse et al., 2011). However, summarizing student responses to these prompts for large courses (e.g., introduc- tory STEM, MOOCs) is an onerous task for hu- mans and poses challenges for existing summa- rization methods. First, the linguistic units of stu- dent inputs range from single words to multiple sentences. Second, we assume that the concepts (represented as phrases) mentioned by more stu- dents should get more attention from the instruc- tor. Based on this assumption, we introduce the notion of student coverage, defined as the number of students who semantically mention a particular phrase. The more student coverage a phrase has, Reflection prompt: Describe what was confusing or needed more detail. Student Responses: S1: Graphs of attraction/repulsive & interatomic separation S2: Property related to bond strength S3: The activity was difficult to comprehend as the text fuzzing and difficult to read. S4: Equations with bond strength and Hooke’s law S5: I didn’t fully understand the concept of thermal expansion S6: The activity ( Part III) S7: Energy vs. distance between atoms graph and what it tells us S8: The graphs of attraction and repulsion were confusing to me...(rest omitted, 53 student responses in total.) Human Reference Summary: 1) Graphs of attraction/ repulsive & atomic separation [10] 2) Properties and equations with bond strength [7] 3) Coefficient of thermal expansion [6] 4) Activity part III [4] Table 1: An example reflection prompt, 53 stu- dent responses and a gold-standard summary. The numbers in the square brackets indicate the num- ber of students who semantically mention each phrase (i.e., student coverage). the more important it is. To illustrate the new task, an example is shown in Table. 1. In this work, we propose a phrase summariza- tion method that addresses the above challenges. First, our summaries are created from extracted …

Content truncated to 500 words