Context-Aware Text Normalisation for Historical Dialects

Proceedings of the 28th International Conference on Computational Linguistics, pages 1023–1036 Barcelona, Spain (Online), December 8-13, 2020 1023 Context-Aware Text Normalisation for Historical Dialects Maria Sukhareva Applied Computational Linguistics Lab (ACoLi) Goethe University Frankfurt sukhareva.maria@gmail.com Abstract Context-aware historical text normalisation is a severely under-researched area. To fill the gap we propose a context-aware normalisation approach that relies on the state-of-the-art methods in neural machine translation and transfer learning. We propose a multidialect normaliser with a context-aware reranking of the candidates. The reranker relies on a word-level n-gram language model that is applied to the five best normalisation candidates. The results are evaluated on the historical multidialect datasets of German, Spanish, Portuguese and Slovene. We show that incorporating dialectal information into the training leads to an accuracy improvement on all the datasets. The context-aware reranking gives further improvement over the baseline. For three out of six datasets, we reach a significantly higher accuracy than reported in the previous studies. The other three results are comparable with the current state-of-the-art. The code for the reranker is published as open-source1. 1 Introduction Historical languages rarely have enough resources to satisfy the needs of state-of-the-art supervised machine learning methods. Manual labeling of large amounts of training data is frequently out-of-the- question as it demands rare and expensive expertise of historical linguists. Historical dialects, however, are frequently grammatically and lexically similar to their modern high-resource descendants. Thus, being able to normalise the historical spelling to the modern standards can enable processing of historical dialects with NLP tools trained on modern languages. Modern high-resource languages are well-standardized and have prescriptive grammar, spelling and lexica usage standards. Historical languages, on the contrary, did not have such standardization. While modern German also has a plenty of dialects, there exists its standardized variety i.e. Standard High German. Middle High German (MHG), on the contrary, does not have contemporary standardization. When we deal with a historical text, we deal with a historical dialect (e.g. Middle High German, Middle Low German) that text is written in. Even more, there is a plenty of variability within the dialects caused by the lack of orthographic standards, geographical differences, the exact time when a text was written etc. The normalisation task presupposes that there exists a standard to which the historical variants are normalised. As historical languages do not have such a standard, the closes possible goal for historical dialect normalisation would be their modern standardized descendants. Throughout centuries, languages undergo systematic phonological changes such as the Great Vowel Shift in English, the first and second consonant shifts in Germanic dialects or the first and second palatal- ization in Slavonic languages. This can be captured systematically by machine learning algorithms and applied to unseen words. Thus, the current state-of-the-art approaches to the historical normalisation rely on statistical or neural machine translation methods and define the task as a problem of translating between characters or substrings (Mansfield et al., 2019) instead of words. Historical texts pose an additional challenge of non-standardized orthography. Spelling variations can …

Content truncated to 500 words