Analysis and Detection of Reading Miscues for Interactive Literacy Tutors

Analysis and Detection of Reading Miscues for Interactive Literacy Tutors Katherine Lee, Andreas Hagen, Nicholas Romanyshyn, Sean Martin, Bryan Pellom Center for Spoken Language Research University of Colorado at Boulder pellom@cslr.colorado.edu Abstract The Colorado Literacy Tutor (CLT) is a technology-based literacy program, designed on the basis of cognitive theory and scientifically motivated reading research, which aims to improve literacy and student achievement in public schools. One of the critical components of the CLT is a speech recognition system which is used to track the child’s progress during oral reading and to provide sufficient information to detect reading miscues. In this paper, we extend on prior work by examining a novel labeling of children’s oral reading audio data in order to better understand the factors that contribute most significantly to speech recognition errors. While these events make up nearly 8% of the data, they are shown to account for approximately 30% of the word errors in a state-of-the-art speech recognizer. Next, we consider the problem of detecting miscues during oral reading. Using features derived from the speech recognizer, we demonstrate that 67% of reading miscues can be detected at a false alarm rate of 3%. 1 Introduction Pioneering research by MIT and CMU as well as more recent work by the IBM Watch-me-Read project has demonstrated human language technologies can play an effective role in systems designed to improve children’s reading abilities (McCandless, 1992; Mostow et al., 1994; Zue et al., 1996). In CMU’s Project LISTEN, for example, the tutor operates by prompting children to read individual sentences out loud. The tutor listens to the child using speech recognition and extracts features that can be used to detect oral reading miscues (Mostow et al., 2002; Tam et al. 2003). The most common miscues that children make while reading out loud are word substitutions, repetitions, and self-corrections with word omissions and insertions being less frequent (Fogarty et al. 2001). Upon detecting such reading errors, the tutor must provide appropriate feedback to the child. While the type of feedback and level of feedback is the current subject of much debate within the research community, recent results have shown that automated reading tutors can improve student achievement (Mostow et al., 2003). In fact, providing real time feedback by simply highlighting words as they are read out loud is the basis of at least one commercial product today1. Cole et al. (2003) and Wise et al. (in press) describe a new scientifically-based literacy program, Foundations to Fluency, in which a virtual tutor—a lifelike 3D computer model— interacts with children in multimodal learning tasks to teach them to read. A key component of this program is the Interactive Book, which combines real-time multilingual speech recognition, facial animation, and natural language understanding capabilities to teach children to read and comprehend text. Within the context of this reading program, Hagen et al. (2003) demonstrated an initial speech recognition system that provides real-time reading tracking for children. This work was later extended by Hagen et al. (2004) to incorporate …

Content truncated to 500 words