Joining-in-type Humanoid Robot Assisted Language Learning System

Joining-in-type Humanoid Robot Assisted Language Learning System AlBara Khalifa, Tsuneo Kato, Seiichi Yamamoto Graduate School of Science and Engineering, Doshisha University 1-3 Tatara Miyakodani, Kyotanabe-shi, Kyoto, 610-0394, Japan Email: albara.khalifa@gmail.com, seyamamo@mail.doshisha.ac.jp, tsukato@mail.doshisha.ac.jp Abstract Dialogue robots are attractive to people, and in language learning systems, they motivate learners and let them practice conversational skills in more realistic environment. However, automatic speech recognition (ASR) of the second language (L2) learners is still a challenge, because their speech contains not just pronouncing, lexical, grammatical errors, but is sometimes totally disordered. Hence, we propose a novel robot assisted language learning (RALL) system using two robots, one as a teacher and the other as an advanced learner. The system is designed to simulate multiparty conversation, expecting implicit learning and enhancement of predictability of learners' utterance through an alignment similar to "interactive alignment", which is observed in human-human conversation. We collected a database with the prototypes, and measured how much the alignment phenomenon observed in the database with initial analysis. Keywords: CALL, robot assisted language learning, interactive alignment, speech recognition 1. Introduction Today's globalization produces much more opportunities of communicating in second languages (L2s) than ever. Learning second languages (L2s) is getting important for a large number of people. As a convenient and economic self-learning method, computer assisted language learning (CALL) gathers high interests. Though the CALL systems started from a fixed and passive style, interactive CALL systems, such as translation game type (Rayner et al., 2010; Wang & Seneff, 2007) and dialogue game type (Seneff & Wang & Zhang, 2004; Brusk & Wik & Hjalmarsson, 2007; Ito et al., 2008), are actively studied in accordance with the advance of automatic speech recognition (ASR). The interactive CALL systems have advantages of encouraging learners to construct utterances on their own and giving corrective feedback. Such flexible interactive CALL systems are able to motivate learners much more than fixed and passive systems as well. To make the interactive CALL systems simulate human-human conversation more realistically, robot assisted language learning (RALL) has been proposed. Using a robot instead of a computer helps to add the dimension of using different modalities into the interaction. The modalities, such as gestures, nodding, face tracking, etc. raise the level of the experience closer to real life situation. Some studies verified the effectiveness of RALL in motivating learners (Lee et al., 2011), especially kids (Chang et al., 2010; Fridin, 2014; Keren & Fridin, 2014). If the RALL systems cover a wide range of topics, learners can train their conversational skill in L2 on the topics of their interest (Wilcock & Yamamoto, 2015). However, ASR of L2 speech is still a challenge because the speech is often made with poor pronunciation and contains grammatical errors. On this issue, CALL or RALL systems usually limit the range of possible learner utterances, for example, by setting the task as translation or dialogue in a predefined domain (Raux & Eskenazi, 2004). Regarding this issue, a phenomenon called "interactive alignment" occurs in human-human conversations. The "interactive alignment" is an …

Content truncated to 500 words