Semi-supervised Contextual Historical Text Normalization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7284–7295 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7284 Semi-supervised Contextual Historical Text Normalization Peter Makarov Simon Clematide Institute of Computational Linguistics University of Zurich, Switzerland makarov@cl.uzh.ch simon.clematide@cl.uzh.ch Abstract Historical text normalization, the task of map- ping historical word forms to their modern counterparts, has recently attracted a lot of interest (Bollmann, 2019; Tang et al., 2018; Lusetti et al., 2018; Bollmann et al., 2018; Robertson and Goldwater, 2018; Bollmann et al., 2017; Korchagina, 2017). Yet, virtu- ally all approaches suffer from the two lim- itations: 1) They consider a fully supervised setup, often with impractically large manually normalized datasets; 2) Normalization hap- pens on words in isolation. By utilizing a sim- ple generative normalization model and ob- taining powerful contextualization from the target-side language model, we train accurate models with unlabeled historical data. In re- alistic training scenarios, our approach often leads to reduction in manually normalized data at the same accuracy levels. 1 Introduction Text normalization is the task of mapping texts written in some non-standard variety of language L (a dialect or an earlier diachronic form) to some standardized form, typically the official mod- ern standard variety of L (Table 1). Examples include the normalization of informal English- language tweets (Han and Baldwin, 2011); quasi- phonetic transcriptions of dialectal Swiss German (Samardˇzi´c et al., 2015); and historical documents such as religious texts in 15th-century Icelandic (Bollmann et al., 2011; Pettersson et al., 2013b; Ljubeˇsi´c et al., 2016, inter alia). Text can to a large extent be normalized by replacing non-standard words with their standard counterparts. Because of this often-made as- sumption, this task is also known as “lexical” or “spelling normalization” (Han and Baldwin, 2011; Tang et al., 2018). There has been a lot of interest in historical and dialectal text normalization over the past years. Earlier works attempt type-level normalization by way of search for standardized words (Pettersson et al., 2013a; Bollmann, 2012). More recently, the dominant approach casts the problem as prob- abilistic type-level character transduction. Most commonly, a fully-supervised machine translation system transduces words in isolation (Bollmann, 2019). The use of context is limited to employ- ing a target-side language model for an improved, contextualized decoding (Ljubeˇsi´c et al., 2016; Etxeberria et al., 2016; Jurish, 2010). In this paper, we develop simple approaches to semi-supervised contextualized text normaliza- tion. On the example of historical text normal- ization, we show that one can reduce the amount of supervision by leveraging unlabeled histori- cal text and utilizing context at training. Our methods build on familiar techniques for semi- supervised learning such as generative modeling and expectation–maximization and unify previous work (search, noisy channel, contextualized de- coding, neural character-level transduction) in a simple setup. We experimentally validate the strength of our models on a suite of historical datasets. In ad- dition to the token-level supervision scenario, we show benefits of a more economic supervision by a word-type normalization dictionary. 2 Historical …

Content truncated to 500 words