Text Retrieval for Language Learners: Graded Vocabulary vs. Open Learner Model

Proceedings of Recent Advances in Natural Language Processing, pages 798–804 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_091 798 Text Retrieval for Language Learners: Graded Vocabulary vs. Open Learner Model John S. Y. Lee, Chak Yan Yeung Department of Linguistics and Translation City University of Hong Kong Hong Kong SAR, China jsylee@cityu.edu.hk, cyyeung91@gmail.com Abstract A text retrieval system for language learning returns reading materials at the appropriate dif- ficulty level for the user. The system typically maintains a learner model on the user’s vocab- ulary knowledge, and identifies texts that best fit the model. As the user’s language profi- ciency increases, model updates are necessary to retrieve texts with the corresponding lexi- cal complexity. We investigate an open learner model that allows user modification of its con- tent, and evaluate its effectiveness with respect to the amount of user update effort. We com- pare this model with the graded approach, in which the system returns texts at the optimal grade. When the user makes at least half of the expected updates to the open learner model, simulation results show that it outperforms the graded approach in retrieving texts that fit user preference for new-word density. 1 Introduction Since language learning requires extensive extra- curricular reading, learners can benefit from a text retrieval system that helps them identify suit- able reading materials from a pool of candidate texts (Brown and Eskenazi, 2004; Miltsakaki, 2009; Lee, 2021). The suitability of a text may depend on multiple factors, including the user’s reading interests (Heilman et al., 2007b), and the degree of matching between its difficulty and the user’s proficiency. This paper investigates the use of an open learner model (OLM) to predict the latter. By giving users more control over their learn- ing, OLMs have been shown to foster users’ con- fidence and reflection on their progress (Bull and Kay, 2007). These benefits are especially important for long-term or life-long learning activities (Kay and Kummerfeld, 2019), such as foreign language learning. We evaluate the effectiveness of an ed- itable OLM (Bull and Kay, 2010) — i.e., an OLM that allows the user not only to view but also to modify its content — for text retrieval for language learning. As their vocabulary expands, users can update the OLM so that the system continues to retrieve texts that are lexically challenging to them. Specifically, we address two research questions: Text retrieval performance (Q1): How accu- rately can an OLM identify reading materials with the desired density of new vocabulary, as specified by the user? User update effort (Q2): How frequently does the user need to edit the OLM in order to reap its benefits? Most previous research in computer-assisted lan- guage learning measured text retrieval performance through holistic evaluation (Heilman et al., 2007a), or in terms of users’ overall learning outcomes (Hsu et al., 2013). In answering Q1, we will directly eval- uate the density of new vocabulary in the retrieved texts. In addressing Q2, we will further consider how retrieval performance is affected by users’ vo- cabulary …

Content truncated to 500 words