Diacritics Restoration Using Neural Networks

Diacritics Restoration Using Neural Networks Jakub Náplava, Milan Straka, Pavel Straňák, Jan Hajič Institute of Formal and Applied Linguistics Charles University, Faculty of Mathematics and Physics Malostranské náměstí 25, Prague, Czech Republic {naplava,straka,stranak,hajic}@ufal.mff.cuni.cz Abstract In this paper, we describe a novel combination of a character-level recurrent neural-network based model and a language model applied to diacritics restoration. In many cases in the past and still at present, people often replace characters with diacritics with their ASCII counterparts. Despite the fact that the resulting text is usually easy to understand for humans, it is much harder for further computational processing. This paper opens with a discussion of applicability of restoration of diacritics in selected languages. Next, we present a neural network-based approach to diacritics generation. The core component of our model is a bidirectional recurrent neural network operating at a character level. We evaluate the model on two existing datasets consisting of four European languages. When combined with a language model, our model reduces the error of current best systems by 20% to 64%. Finally, we propose a pipeline for obtaining consistent diacritics restoration datasets for twelve languages and evaluate our model on it. All the code is available under open source license on https://github.com/arahusky/diacritics_restoration. Keywords: neural networks, diacritics, diacritics generation, error correction 1. Introduction When writing emails, tweets or texts in certain languages, people for various reasons sometimes write without diacrit- ics. When using Latin script, they replace characters with diacritics (e.g. c with acute or caron) by the underlying ba- sic character without diacritics. Practically speaking, they write in ASCII. We offer several possible reasons for this phenomenon: • Historically, many devices offered only an English key- board and/or ASCII encoding (for example older mobile phones and SMS). • Before Unicode became widespread, there were encod- ing problems among platforms and even among pro- grams on the same platform, and many people still have this in mind. • Even though text encoding is rarely a problem any more and all modern devices offer native keyboards, some problems persist. In situations of frequent code switch- ing between English and a language with a substantially different keyboard layout, it is very hard to touch type in both layouts. It is much easier to type both languages using the same layout, although one of them without proper diacritics. • In some circumstances typing with diacritical marks is significantly slower than using just basic latin char- acters. The most common example is on-screen key- boards on mobile devices. These keyboards do not in- clude the top row (numerical on US English), so lan- guages that use that row for accented characters are much slower to type. Naturally, users type without ex- plicit accents and rely on the auto-completion systems. However, these systems are usually simple, unigram- based, and based on the word form ambiguity for a given language (cf. Table 1), which introduces many errors. Postponing the step of diacritics generation would be beneficial both for typing speed and accuracy. • For example …

Content truncated to 500 words