Which is the Effective Way for Gaokao: Information Retrieval or Neural Networks?

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 111–120, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Which is the Effective Way for Gaokao: Information Retrieval or Neural Networks? Shangmin Guo†1, Xiangrong Zeng†1,2, Shizhu He1, Kang Liu1 and Jun Zhao1 1National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China 2University of Chinese Academy of Sciences, Beijing, 10049, China {shangmin.guo, xiangrong.zeng}@nlpr.ia.ac.cn {shizhu.he, kliu, jzhao}@nlpr.ia.ac.cn Abstract As one of the most important test of China, Gaokao is designed to be difficult enough to distinguish the excellent high school students. In this work, we detailed the Gaokao History Multiple Choice Ques- tions(GKHMC) and proposed two differ- ent approaches to address them using var- ious resources. One approach is based on entity search technique (IR approach), the other is based on text entailment approach where we specifically employ deep neu- ral networks(NN approach). The result of experiment on our collected real Gaokao questions showed that they are good at different categories of questions, i.e. IR approach performs much better at entity questions(EQs) while NN approach shows its advantage on sentence questions(SQs). Our new method achieves state-of-the-art performance and show that it’s indispens- able to apply hybrid method when partici- pating in the real-world tests. 1 Introduction Gaokao, namely the National College Entrance Examination, is the most important examination for Chinese senior high school students. Ev- ery college in China, no matter it is Top10 or Top100, would only accept the exam-takers whose Gaokao score is higher than its threshold score. As there are almost 10 million students take the examination every year, Gaokao needs to be dif- ficult enough to distinguish the excellent students. Therefore, it includes various types of questions such as multiple-choice questions, short-answer † Both of the two authors contributed equally to this paper. After the World War II, U.S. and Soviet Union are fighting against each other in politics, economics and military. To promote the development of economics in Socialist Countries, Soviet Union establish The Council for Mutual Economic Assistance. This is against A. Truman Doctrine B. Marshall Plan C. NATO D. Federal Republic of Germany Entity Question From Qin and Han Dynasties to Ming Dynasty, businessmen are always at the bottom of hierarchy. One reason for this is that the ruling class thought the businessmen A. are not engaged in production B. do not respect Confucianism C. do not respect the clan D. do not pay tax Sentence Question Figure 1: Examples of questions and their types. The upper one is an entity question. The lower one is a sentence question. questions and essays and it covers several dif- ferent subjects, like Chinese, Math, History and etc. In this work, we focus on Gaokao History Multiple Choice questions which is denoted as GKHMC. Both of the factoid question answering task and reading comprehension task are similar to GKHMC. But, the GKHMC questions have their own characteristics. A multiple-choice question …

Content truncated to 500 words