Historical Text Normalization with Delayed Rewards

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1614–1619 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1614 Historical Text Normalization with Delayed Rewards Simon Flachs, Marcel Bollmann, Anders Søgaard Department of Computer Science University of Copenhagen Denmark {flachs,marcel,soegaard}@di.ku.dk Abstract Training neural sequence-to-sequence models with simple token-level log-likelihood is now a standard approach to historical text normal- ization, albeit often outperformed by phrase- based models. Policy gradient training enables direct optimization for exact matches, and while the small datasets in historical text nor- malization are prohibitive of from-scratch re- inforcement learning, we show that policy gra- dient fine-tuning leads to significant improve- ments across the board. Policy gradient train- ing, in particular, leads to more accurate nor- malizations for long or unseen words. 1 Introduction Historical text normalization is a common ap- proach to making historical documents accessible and searchable. It is a challenging problem, since most historical texts were written without fixed spelling conventions, and spelling is therefore at times idiosyncratic (Piotrowski, 2012). Traditional approaches to historical text nor- malization relied on hand-written rules, but re- cently, several authors have proposed neural mod- els for historical text normalization (Bollmann and Søgaard, 2016; Bollmann, 2018; Tang et al., 2018). Such models are trained using character- level maximum-likelihood training, which is in- consistent with the objective of historical text normalization; namely, transduction into modern, searchable word forms. The discrepancy between character-level loss and our word-level objective means that model decision costs are biased. Our objective, however, is reflected by the standard evaluation metric, which is computed as the frac- tion of benchmark words that are translated cor- rectly. In order to mitigate the discrepancy between the optimization method and the task objective, work has been carried out on using reinforcement learning to optimize directly for the evaluation metric (Ranzato et al., 2016; Shen et al., 2016). Reinforcement learning enables direct optimiza- tion of exact matches or other non-decomposable metrics, computing updates based on delayed re- wards rather than token-level error signals. This paper contrasts maximum likelihood training and training with delayed rewards, in the context of sequence-to-sequence historical text normaliza- tion (Bollmann et al., 2017). Contributions We show that training with de- layed rewards achieves better performance than maximum likelihood training across six different historical text normalization benchmarks; and that training with delayed rewards is particularly help- ful for long words, words where maximum likeli- hood training leads to predicting long words, and for unseen words. We note that our approach dif- fers from other applications in the NLP literature in using the mean reward as our baseline, and in comparing different reward functions; we also fine-tune relying only on rewards, rather than a mixture of cross entropy loss and rewards. 2 Historical text normalization datasets Historical text normalization datasets are rare and typically rather small. Most of them are based on collections of medieval documents. In our experi- ments, we include six historical text normalization datasets: the English, Hungarian, Icelandic, …

Content truncated to 500 words