Autonomous Self-Assessment of Autocorrections: Exploring Text Message Dialogues

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 710–719, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics Autonomous Self-Assessment of Autocorrections: Exploring Text Message Dialogues Tyler Baldwin Department of Computer Science and Engineering Michigan State University East Lansing, MI 48824 baldwi96@cse.msu.edu Joyce Y. Chai Department of Computer Science and Engineering Michigan State University East Lansing, MI 48824 jchai@cse.msu.edu Abstract Text input aids such as automatic correction systems play an increasingly important role in facilitating fast text entry and efficient com- munication between text message users. Al- though these tools are beneficial when they work correctly, they can cause significant communication problems when they fail. To improve its autocorrection performance, it is important for the system to have the capabil- ity to assess its own performance and learn from its mistakes. To address this, this pa- per presents a novel task of self-assessment of autocorrection performance based on interac- tions between text message users. As part of this investigation, we collected a dataset of au- tocorrection mistakes from true text message users and experimented with a rich set of fea- tures in our self-assessment task. Our exper- imental results indicate that there are salient cues from the text message discourse that al- low systems to assess their own behaviors with high precision. 1 Introduction The use of SMS text messaging is widespread and growing. Users of text messaging often rely on small mobile devices with limited user interfaces to com- municate with each other. To support efficient com- munication between users, many tools to aid text in- put such as automatic completion (autocompletion) and automatic correction (autocorrection) have be- come available. When they work correctly, these tools allow users to maintain clear communication while potentially increasing the rate at which they input their message, improving efficiency in com- munication. However, when these tools make a mis- take, they can cause problematic situations. Con- sider the following example: A1: Euthanasia doing tonight? B1: Euthanasia?! A2: I typed whatcha and stupid autotype. In this example, the automatic correction system on person A’s phone interpreted his attempt to write the word whatcha as an attempt to write euthanasia (due to the keyboard adjacency of the w and e keys, etc.). This completely changed the meaning of the message, which confused person B. Although this instance was eventually discovered and corrected, the natural flow of conversation was interrupted and the participants were forced to make extra effort to clarify this confusion. This example indicates that the cost of a mistake in autocorrection is potentially high. This is exacer- bated by the fact that users will often fail to notice these mistakes in a timely manner, due to their focus being on the keyboard (Paek et al., 2010) and the quick and casual conversation style of text messag- ing. Because of this, autocorrection systems must have high accuracy to be useful for text messaging. This example also indicates that, when an autocor- rection mistake happens …

Content truncated to 500 words