Context-Aware Abbreviation Expansion Using Large Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1261 - 1275 July 10-15, 2022 ©2022 Association for Computational Linguistics Context-Aware Abbreviation Expansion Using Large Language Models Shanqing Cai∗ Subhashini Venugopalan∗ Katrin Tomanek Ajit Narayanan Meredith Ringel Morris Michael P. Brenner Google Research {cais,vsubhashini}@google.com Abstract Motivated by the need for accelerating text en- try in augmentative and alternative communi- cation (AAC) for people with severe motor im- pairments, we propose a paradigm in which phrases are abbreviated aggressively as primar- ily word-initial letters. Our approach is to expand the abbreviations into full-phrase op- tions by leveraging conversation context with the power of pretrained large language mod- els (LLMs). Through zero-shot, few-shot, and fine-tuning experiments on four public conver- sation datasets, we show that for replies to the initial turn of a dialog, an LLM with 64B pa- rameters is able to accurately expand over 70% of phrases with abbreviation length up to 10, leading to an effective keystroke saving rate of up to 77% on these expansions. Including a small amount of context in the form of a single conversation turn more than doubles abbrevia- tion expansion accuracies compared to having no context, an effect that is more pronounced for longer phrases. Additionally, the robust- ness of the models against typo noise can be enhanced through fine-tuning on noisy data. 1 Introduction The prevalent paradigm of text entry on computing devices is sequential typing of characters. Word completion and prediction can theoretically save up to 40-50% keystrokes when 3-5 predictions are provided (Trnka and McCoy, 2008; Fowler et al., 2015). This reduces the motor and cogni- tive demand of entering text, especially on devices where typing is difficult, e.g., phones. In AAC use cases such as eye-gaze keyboards for severely motor-impaired individuals, the cost per keystroke is so high that there is a desire to save as many keystrokes as possible. Gaze-typing requires the user to precisely control the direction and timing of gaze for each keystroke, resulting in an extremely low text-entry speed of 8-10 words per minute and ∗equal contribution Figure 1: Our approach to abbreviation expansion based on an LLM with context compared to one without. The conversation context (e.g., a previous turn of conversation) along with the abbreviation of the intended phrase form the LLM’s input. Sampled continuations from the model are filtered to discard those that do not match the abbreviation. Top-5 options after sorting by frequency are presented. severely limiting real-time communication (Waller, 2019). A text-entry paradigm with substantially higher keystroke saving rate (KSR) can reduce mo- tor demand and thereby benefit AAC usage in real- time communication. aOne potential paradigm is "SMS language", spontaneously-evolved system for saving keystrokes in which each word is abbreviated as a single letter, such as in the well-known abbrevi- ations sg for sounds good and ttyl for talk to you later (Anjaneyulu, 2013). SMS language features a high KSR (75-80%), but is limited by its small closed …

Content truncated to 500 words