R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning

Proceedings of the 28th International Conference on Computational Linguistics, pages 1147–1157 Barcelona, Spain (Online), December 8-13, 2020 1147 R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning Irene Li1, Alexander Fabbri1, Swapnil Hingmire2 and Dragomir Radev1 1Yale University, USA 2Tata Consultancy Services Limited (TCS), India Abstract The task of concept prerequisite chain learning is to automatically determine the existence of prerequisite relationships among concept pairs. In this paper, we frame learning prerequisite relationships among concepts as an unsupervised task with no access to labeled concept pairs during training. We propose a model called the Relational-variational Graph AutoEncoder (R- VGAE) to predict concept relations within a graph consisting of concept and resource nodes. Results show that our unsupervised approach outperforms graph-based semi-supervised methods and other baseline methods by up to 9.77% and 10.47% in terms of prerequisite relation pre- diction accuracy and F1 score. Our method is notably the first graph-based model that attempts to make use of deep learning representations for the task of unsupervised prerequisite learning. We also expand an existing corpus which totals 1, 717 English Natural Language Processing (NLP)-related lecture slide files and manual concept pair annotations over 322 topics. 1 Introduction With the increasing amount of information available online, there is a rising need for structuring how one should process that information and learn knowledge efficiently in a reasonable order. As a result, recent work has tried to learn prerequisite relations among concepts, or which concept is needed to learn another concept within a concept graph (Liang et al., 2017; Gordon et al., 2016; AlSaad et al., 2018). Figure 1 shows an illustration of prerequisite chains as a directed graph. In such a graph, each node is a concept, and the direction of each edge indicates the prerequisite relation. Consider two concepts p and q, we define p → q as p is a prerequisite concept of q. For example, the concept Variational Autoencoders is a prerequisite concept of the concept Variational Graph Autoencoders. If someone wants to learn about the concept Variational Graph Autoencoders, the prerequisite concept Variation Autoencoder should appear in the prerequisite concept graph in order to create a proper study plan. Recent work has attempted to extract such prerequisite relationships from various types of materials including Wikipedia articles, university course dependencies or MOOCs (Massive Open Online Courses) (Pan et al., 2017; Gordon et al., 2016; Liang et al., 2017). However, these materials either need additional steps for pre-processing and cleaning, or contain too many noisy free-texts, bringing more challenges to prerequisite relation learning or extracting. Recently, Li et al. (2019) presented a collection of university lecture slide files mainly in NLP lectures with related prerequisite concept annotations. We expanded this dataset as we believe these lecture slides offer a concise yet comprehensive description of advanced topics. Deep models such as word embeddings (Mikolov et al., 2013) and more recently contextualized word embeddings (Devlin et al., 2018) have achieved great success in the NLP tasks as they demonstrate a stronger ability to represent …

Content truncated to 500 words