Improving Embeddings Representations for Comparing Higher Education Curricula: A Use Case in Computing

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11299 - 11307 December 7-11, 2022 ©2022 Association for Computational Linguistics Improving Embeddings Representations for Comparing Higher Education Curricula: A Use Case in Computing Jeffri Murrugarra-Llerena1, Fernando Alva-Manchego2, Nils Murrugarra-Llerena3 1Institute of Informatics, Federal University of Rio Grande do Sul (UFRGS), Brasil 2School of Computer Science and Informatics, Cardiff University, UK 3School of Computing, Weber State University, USA jeffri.mllerena@inf.ufrgs.br,alvamanchegof@cardiff.ac.uk, nmurrugarrallerena@weber.edu Abstract We propose an approach for comparing cur- ricula of study programs in higher education. Pre-trained word embeddings are fine-tuned in a study program classification task, where each curriculum is represented by the names and content of its courses. By combining met- ric learning with a novel course-guided atten- tion mechanism, our method obtains more ac- curate curriculum representations than strong baselines. Experiments on a new dataset with curricula of computing programs demonstrate the intuitive power of our approach via atten- tion weights, topic modeling, and embeddings visualizations. We also present a use case comparing computing curricula from USA and Latin America to showcase the capabilities of our improved embeddings representations. Our code and data are available in https:// github.com/Artcs1/DL_curriculas. 1 Introduction Several stakeholders in Education compare and as- sess curricula, such as Governments aiming to im- prove the competitiveness of their local programs (Adamson and Morris, 2007), or academics looking to analyze market offer and suggest new courses and innovations (Prietch, 2010; de Alburquerque et al., 2010; Macêdo, 2016). Despite their benefits, manual comparisons can be time-consuming, and are prone to biases from human assessors due to their personal beliefs and perspectives (Kawinti- ranon et al., 2016). This situation becomes more challenging when studying curricula related to pro- grams in computing due to their fast-paced evolu- tion, which may prevent suitable analysis and eval- uation by human intervention (Föll and Thiesse, 2021). To aid higher education stakeholders, we propose a method to automatically compare curric- ula based on the names and content of their courses, and test it on study programs in computing. Previous work has represented curricula using bag-of-words, and relied on clustering algorithms to identify groups of commonly-studied topics in data science (West, 2017), or to find links between study programs in different countries and across computing disciplines (Murrugarra-Llerena et al., 2011). Others have attempted to associate curric- ula to knowledge areas defined by international associations, such as ACM or IEEE (Shackelford et al., 2005), using standard TF-IDF representa- tions (Kawintiranon et al., 2016) or topic modelling (Matsuda et al., 2018; Föll and Thiesse, 2021). In this work, we follow the current trend in NLP applications and use pre-trained word embeddings, such as BERT (Devlin et al., 2019), to obtain better representations of textual curricula. We fine-tune these embeddings on a computing discipline classi- fication task, using a newly-collected dataset that includes study programs from accredited universi- ties in the USA and Latin America. We introduce a course-guided attention mechanism that allows the model to identify which courses are more rele- vant for each …

Content truncated to 500 words