ExpanRL: Hierarchical Reinforcement Learning for Course Concept Expansion in MOOCs

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 770–780 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 770 ExpanRL: Hierarchical Reinforcement Learning for Course Concept Expansion in MOOCs Jifan Yu1, Chenyu Wang1, Gan Luo1, Lei Hou1,2,3∗, Juanzi Li1,2,3, Jie Tang1,2,3, Minlie Huang1,2,3, Zhiyuan Liu1,2,3 1Dept. of Computer SCi.& Tech., Tsinghua University, China 100084 2KIRC, Institute for Artificial Intelligence, Tsinghua University, China 100084 3Beijing National Research Center for Information Science and Technology, China 100084 {yujf18,luog18}@mails.tsinghua.edu.cn {houlei,lijuanzi,jietang,aihuang,liuzy}@tsinghua.edu.cn Abstract Within the prosperity of Massive Open On- line Courses (MOOCs), the education appli- cations that automatically provide extracurric- ular knowledge for MOOC users have be- come rising research topics. However, MOOC courses’ diversity and rapid updates make it more challenging to find suitable new knowl- edge for students. In this paper, we present Ex- panRL, an end-to-end hierarchical reinforce- ment learning (HRL) model for concept ex- pansion in MOOCs. Employing a two-level HRL mechanism of seed selection and con- cept expansion, ExpanRL is more feasible to adjust the expansion strategy to find new con- cepts based on the students’ feedback on ex- pansion results. Our experiments on nine novel datasets from real MOOCs show that ExpanRL achieves significant improvements over existing methods and maintain competi- tive performance under different settings. 1 Introduction The cognitive-driven theory has been widely used in practical teaching since Ausubel firstly proposed it in (Ausubel, 1968), which suggests educators provide new knowledge for students to motivate their learning continuously. In fact, in addition to the concepts taught in course, many related con- cepts are also attractive and worthy of learning. As shown in Figure 1, when a student studies the concept LSTM in “Deep Learning” course from Coursera1, many related concepts, including its pre- requisite concepts (RNN), related scientists (J¨urgen Schmidhuber) and its related applications (Machine Translation) can also benefit his/her further study. In traditional classrooms, these concepts are often considerately introduced by teachers. ∗Corresponding author. 1https://www.coursera.org Figure 1: An example of course-related concepts in the “Deep Learning” course from Coursera. However, in the era of Massive Open Online Courses (MOOCs), thousands of courses are pre- recorded for with millions of students with various backgrounds (Shah, 2019), which makes it infeasi- ble to pick out these essential concepts manually. Therefore, there is a clear need to automatically discover course-related concepts so that they can easily acquire additional knowledge and achieve better educational outcomes. This task is formally defined as Course Concept Expansion (Yu et al., 2019a), a special type of Concept Expansion or Set Expansion (Wang and Cohen, 2007), which refers to the task of expanding a small set of seed concepts into a complete set of concepts that belong to the same course or subject from external resources. Despite abundant efforts in related topics (He and Xin, 2011; Shen et al., 2017; Yan et al., 2019), existing methods still face three challenges when applied to MOOCs. First, distinct from …

Content truncated to 500 words