LIMSIILES: Basic English Substitution for Student Answer Assessment at SemEval 2013

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 598–602, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics LIMSIILES: Basic English Substitution for Student Answer Assessment at SemEval 2013 Martin Gleize LIMSI-CNRS & ENS B.P. 133 91403 ORSAY CEDEX, France gleize@limsi.fr Brigitte Grau LIMSI-CNRS & ENSIIE B.P. 133 91403 ORSAY CEDEX, France bg@limsi.fr Abstract In this paper, we describe a method for as- sessing student answers, modeled as a para- phrase identification problem, based on sub- stitution by Basic English variants. Basic En- glish paraphrases are acquired from the Sim- ple English Wiktionary. Substitutions are ap- plied both on reference answers and student answers in order to reduce the diversity of their vocabulary and map them to a common vocabulary. The evaluation of our approach on the SemEval 2013 Joint Student Response Analysis and 8th Recognizing Textual Entail- ment Challenge data shows promising results, and this work is a first step toward an open- domain system able to exhibit deep text un- derstanding capabilities. 1 Introduction Automatically assessing student answers is a chal- lenging natural language processing task (NLP). It is a way to make test grading easier and improve adaptive tutoring (Dzikovska et al., 2010), and is the goal of the SemEval 2013’s task 7, titled Joint Stu- dent Response Analysis. More specifically, given a question, a known correct “reference answer” and a 1- or 2-sentence student answer, the goal is to deter- mine the student’s answer accuracy (Dzikovska et al., 2013). This can be seen as a paraphrase identi- fication problem between student answers and refer- ence answers. Paraphrase identification searches whether two sentences have essentially the same meaning (Culi- cover, 1968). Automatically generating or extract- ing semantic equivalences for the various units of language – words, phrases, and sentences – is an im- portant problem in NLP and is being increasingly employed to improve the performance of several NLP applications (Madnani and Dorr, 2010), like question-answering and machine translation. Paraphrase identification would benefit from a precise and broad-coverage semantic language model. This is unfortunately difficult to obtain to its full extent for any natural language, due to the size of a typical lexicon and the complexity of grammat- ical constructions. Our hypothesis is that the sim- pler the language lexicon is, the easier it will be to access and compare meaning of sentences. This as- sumption is justified by the multiple attempts at con- trolled natural languages (Schwitter, 2010) and es- pecially simplified forms of English. One of them, Basic English (Ogden, 1930), has been adopted by the Wikipedia Project as the preferred language of the Simple English Wikipedia1 and its sister project the Simple English Wiktionary2. Our method starts with acquiring paraphrases from the Simple English Wiktionary’s definitions. Using those, we generate variants of both sentences whose meanings are to be compared. Finally, we compute traditional lexical and semantic similarity measures on those two sets of variants to produce features to train a …

Content truncated to 500 words