Using idiolects and sociolects to improve word prediction

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 318–327, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Using idiolects and sociolects to improve word prediction Wessel Stoop Centre for Language and Speech Technology Radboud University Nijmegen w.stoop@let.ru.nl Antal van den Bosch Centre for Language Studies Radboud University Nijmegen a.vandenbosch@let.ru.nl Abstract In this paper the word prediction system Soothsayer1 is described. This system pre- dicts what a user is going to write as he is keying it in. The main innovation of Soothsayer is that it not only uses idi- olects, the language of one individual per- son, as its source of knowledge, but also sociolects, the language of the social cir- cle around that person. We use Twitter for data collection and experimentation. The idiolect models are based on individ- ual Twitter feeds, the sociolect models are based on the tweets of a particular person and the tweets of the people he often com- municates with. The idea behind this is that people who often communicate start to talk alike; therefore the language of the friends of person x can be helpful in try- ing to predict what person x is going to say. This approach achieved the best re- sults. For a number of users, more than 50% of the keystrokes could have been saved if they had used Soothsayer. 1 Introduction The main aim of the study presented here is to show that the concepts of idiolect and sociolect, the language of one person and his or her so- cial circle, can be used to improve word predic- tion, the task of predicting what a user is going to type, as he is typing. Word prediction technol- ogy reduces the number of keystrokes we have to make, thus saving time and preventing mistakes. With the rise of smartphones word prediction has become widely known and used. Preceding this 1The system is available as an interactive demo at http://soothsayer.cls.ru.nl/ and its source code is publicly available at https://github.com/ woseseltops/soothsayer popularization, word prediction systems were al- ready developed up to three decades ago to as- sist people with speech and motor disabilities, like cerebral palsy or hemiplexia. By using a device equipped with word prediction technology, they can increase their communication rate consider- ably (Garay-Vitoria and Abascal, 2006). Indeed, most studies before the year 2000, when mobile phones were not widely used yet, targeted the dis- abled user group - Copestake (1997) even reports building a system for one individual user. More recent work targets a wider audience, but in this paper we return to the idea of using an individual’s own language to train individualized models. The concept of an idiolect, the language of a single person, is well-known, but rarely ever modelled or in some other way operationalized (Mollin, 2009; Barlow, 2010; Louwerse, 2004). Almost every claim in the field of linguistics con- cerns language as a whole; whether the subject of investigation is a particular syntactic construc- …

Content truncated to 500 words