Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4253–4259 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4253 Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration Bálint Csanády, András Lukács Department of Computer Science, AI Research Group Institute of Mathematics, Eötvös Loránd University Pázmány Péter stny. 1/c, 1036 Budapest, Hungary csbalint@protonmail.ch, lukacs@cs.elte.hu Abstract Diacritics restoration has become a ubiquitous task in the Latin-alphabet-based English-dominated Internet language environ- ment. In this paper, we describe a small footprint 1D dilated convolution-based approach which operates on a character-level. We find that neural networks based on 1D dilated convolutions are competitive alternatives to solutions based on recurrent neural networks or linguistic modeling for the task of diacritics restoration. Our approach surpasses the performance of similarly sized models and is also competitive with larger models. A special feature of our solution is that it even runs locally in a web browser. We also provide a working example of this browser-based implementation. Our model is evaluated on different corpora, with emphasis on the Hungarian language. We performed comparative measurements about the generalization power of the model in relation to three Hungarian corpora. We also analyzed the errors to understand the limitation of corpus-based self-supervised training. Keywords: diacritics restoration, 1D convolutional neural network, A-TCN, small footprint, Hungarian 1. Introduction Many languages, including most European languages, have alphabets where some of the characters are de- rived from base characters using diacritical marks. The goal of diacritics restoration is to restore diacritical marks, given an input text which does not contain (or only partially contains) the proper diacritical marks. Diacritics restoration is a practical task on the Inter- net, where the absence of diacritical marks can still be prominent. Diacritics restoration is a useful preprocessing step for many NLP tasks, e.g. question answering (Abdelnasser et al., 2014). On the other hand, diacritics restoration is an important tool for language revitalization (Galla, 2009), thus contributing to linguistic diversity, the liter- acy of endangered languages, and the maintenance of their digital presence (Kornai, 2013). This can be ef- fectively supported by language-independent diacritics restoration tools. Nevertheless, we consider only liv- ing languages where large corpora based on the Latin alphabet are available (omitting such exciting cases as Celtic languages or poetry marking). Diacritical marks appear in certain Slavic languages (Czech, Slovak, Pol- ish), some Finno-Ugric languages (Finnish, Hungarian, Latvian), Romanian, Turkish, and, most intensively, in Vietnamese. Approaches to diacritics restoration have evolved from rule-based and statistical solutions to the application of machine learning models (Yarowsky, 1999). The latter approach can be broken down into solutions using fixed or learned representations. All solutions with learned representations seem to be based on neural networks connected to the models used in NLP, lately recurrent neural networks models (Hucko and Lacko, 2018) be- ing replaced by transformers (Laki and Yang, 2020; Náplava et al., 2021). In such cases, models used for machine translation are often used to correct diacritical marks (Novák and Siklósi, 2015). Another …

Content truncated to 500 words