Automatic Cloze-Questions Generation

Proceedings of Recent Advances in Natural Language Processing, pages 511–515, Hissar, Bulgaria, 7-13 September 2013. Automatic Cloze-Questions Generation Annamaneni Narendra, Manish Agarwal and Rakshit shah LTRC, IIIT-Hyderabad, India {narendra.annamaneni|manish agrwal|rakshit shah}@students.iiit.ac.in Abstract Cloze questions are questions containing sen- tences with one or more blanks and multiple choices listed to pick an answer from. In this work, we present an automatic Cloze Ques- tion Generation (CQG) system that generates a list of important cloze questions given an En- glish article. Our system is divided into three modules: sentence selection, keyword selec- tion and distractor selection. We also present evaluation guidelines to evaluate CQG sys- tems. Using these guidelines three evaluators report an average score of 3.18 (out of 4) on Cricket World Cup 2011 data. 1 Introduction Multiple choice questions (MCQs) have been proved efficient to judge students’ knowledge. Man- ual construction of such questions, however, is a time-consuming and labour-intensive task. Cloze questions (CQs) are fill-in-the-blank questions, where a sentence is given with one or more blanks in it with four alternatives to fill those blanks. As op- posed to MCQs where one has to generate the WH style question, CQs use a sentence with blanks to form a question. The sentence could be picked from a document on the topic avoiding the need to gener- ate a WH style question. As a result, automatic CQG has received a lot of research attention recently. 1. Zaheer Khan opened his account with three con- secutive maidens in the world-cup final. (a) Zaheer Khan (b) Lasith Malinga (c) Praveen Kumar (d) Munaf Patel In the above example CQ, the underlined word (referred to as keyword) Zaheer Khan is blanked out in the sentence and four alternatives are given. In area of cloze questions, (Sumita et. al., 2005; Lee and Seneff, 2007; Lin et. al., 2007; Pino et. al., 2009; Smith et. al., 2010) have mostly worked in the domain of English language learning. Cloze ques- tions have been generated to test students knowl- edge of English in using the correct verbs (Sumita et. al., 2005), prepositions (Lee and Seneff, 2007) and adjectives (Lin et. al., 2007) in sentences. Pino et. al. (2009) and Smith et. al. (2010) have gener- ated questions to teach and evaluate student’s vo- cabulary. Agarwal and Mannem (2011) have gener- ated factual cloze questions from a biology text book through heuristically weighted features. They do not use any external knowledge and rely only on infor- mation present in the document to generate the CQs with distractors. This restricts the possibilities dur- ing distractor selection and leads to poor distractors. In this work, we present an end-to-end automatic cloze question generating system which adopts a semi-structured approach to generate CQs by mak- ing use of a knowledge base extracted from a Cricket 1 portal. Also, unlike previous approaches we add context to the question sentence in the process of creating a CQ. This is done to disambiguate the question and avoid cases where there are multiple answers for a …

Content truncated to 500 words