Phonetic and Visual Priors for Decipherment of Informal Romanization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8308–8319 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8308 Phonetic and Visual Priors for Decipherment of Informal Romanization Maria Ryskina1 Matthew R. Gormley2 Taylor Berg-Kirkpatrick3 1Language Technologies Institute, Carnegie Mellon University 2Machine Learning Department, Carnegie Mellon University 3Computer Science and Engineering, University of California, San Diego {mryskina,mgormley}@cs.cmu.edu tberg@eng.ucsd.edu Abstract Informal romanization is an idiosyncratic pro- cess used by humans in informal digital com- munication to encode non-Latin script lan- guages into Latin character sets found on common keyboards. Character substitution choices differ between users but have been shown to be governed by the same main princi- ples observed across a variety of languages— namely, character pairs are often associated through phonetic or visual similarity. We pro- pose a noisy-channel WFST cascade model for deciphering the original non-Latin script from observed romanized text in an unsuper- vised fashion. We train our model directly on romanized data from two languages: Egyp- tian Arabic and Russian. We demonstrate that adding inductive bias through phonetic and visual priors on character mappings substan- tially improves the model’s performance on both languages, yielding results much closer to the supervised skyline. Finally, we intro- duce a new dataset of romanized Russian, col- lected from a Russian social network website and partially annotated for our experiments.1 1 Introduction Written online communication poses a number of challenges for natural language processing sys- tems, including the presence of neologisms, code- switching, and the use of non-standard orthogra- phy. One notable example of orthographic varia- tion in social media is informal romanization2— speakers of languages written in non-Latin alpha- bets encoding their messages in Latin characters, for convenience or due to technical constraints (improper rendering of native script or keyboard 1The code and data are available at https://github. com/ryskina/romanization-decipherment 2Our focus on informal transliteration excludes formal settings such as pinyin for Mandarin where transliteration conventions are well established. хорошо xopowo horosho [Phonetic] [Visual] [Cyrillic] [Phonetically romanized] [Visually romanized] [Underlying Cyrillic] [Underlying Cyrillic] [Visually romanized] [Phonetically romanized] Figure 1: Example transliterations of a Russian word horoxo [horošo, ‘good’] (middle) based on phonetic (top) and visual (bottom) similarity, with character alignments displayed. The phonetic- visual dichotomy gives rise to one-to-many map- pings such as x /S/ → sh / w. layout incompatibility). An example of such a sen- tence can be found in Figure 2. Unlike named en- tity transliteration where the change of script rep- resents the change of language, here Latin charac- ters serve as an intermediate symbolic representa- tion to be decoded by another speaker of the same source language, calling for a completely differ- ent transliteration mechanism: instead of express- ing the pronunciation of the word according to the phonetic rules of another language, informal transliteration can be viewed as a substitution ci- pher, where each source character is replaced with a similar Latin character. In this paper, we focus on decoding informally romanized texts back into their original scripts. We view the …

Content truncated to 500 words