The effect of domain and text type on text prediction quality

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 561–569, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics The effect of domain and text type on text prediction quality Suzan Verberne, Antal van den Bosch, Helmer Strik, Lou Boves Centre for Language Studies Radboud University Nijmegen s.verberne@let.ru.nl Abstract Text prediction is the task of suggesting text while the user is typing. Its main aim is to reduce the number of keystrokes that are needed to type a text. In this paper, we address the influence of text type and do- main differences on text prediction quality. By training and testing our text predic- tion algorithm on four different text types (Wikipedia, Twitter, transcriptions of con- versational speech and FAQ) with equal corpus sizes, we found that there is a clear effect of text type on text prediction qual- ity: training and testing on the same text type gave percentages of saved keystrokes between 27 and 34%; training on a differ- ent text type caused the scores to drop to percentages between 16 and 28%. In our case study, we compared a num- ber of training corpora for a specific data set for which training data is sparse: ques- tions about neurological issues. We found that both text type and topic domain play a role in text prediction quality. The best performing training corpus was a set of medical pages from Wikipedia. The second-best result was obtained by leave- one-out experiments on the test questions, even though this training corpus was much smaller (2,672 words) than the other cor- pora (1.5 Million words). 1 Introduction Text prediction is the task of suggesting text while the user is typing. Its main aim is to reduce the number of keystrokes that are needed to type a text, thereby saving time. Text prediction algo- rithms have been implemented for mobile devices, office software (Open Office Writer), search en- gines (Google query completion), and in special- needs software for writers who have difficulties typing (Garay-Vitoria and Abascal, 2006). In most applications, the scope of the prediction is the completion of the current word; hence the often- used term ‘word completion’. The most basic method for word completion is checking after each typed character whether the prefix typed since the last whitespace is unique according to a lexicon. If it is, the algorithm sug- gests to complete the prefix with the lexicon en- try. The algorithm may also suggest to complete a prefix even before the word’s uniqueness point is reached, using statistical information on the pre- vious context. Moreover, it has been shown that significantly better prediction results can be ob- tained if not only the prefix of the current word is included as previous context, but also previ- ous words (Fazly and Hirst, 2003) or characters (Van den Bosch and Bogers, 2008). In the current paper, we follow up on this work by addressing the influence of text type and do- main differences …

Content truncated to 500 words