GM-RKB WikiText Error Correction Task and Baselines

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2424–2430 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2424 GM-RKB WikiText Error Correction Task and Baselines Gabor Melli, Abdelrhman Eldallal, Bassim Lazem, Olga Moreira gmelli@acm.org, abdelrhman.d@aucegypt.edu, lazem@live.com, olga.moreira@gmail.com Abstract We introduce the GM-RKB WikiText Error Correction Task for the automatic detection and correction of typographical errors in WikiText annotated pages. The included corpus is based on a snapshot of the GM-RKB domain-specific semantic wiki consisting of a large collection of concepts, personages, and publications primary centered on data mining and machine learning research topics. Numerous Wikipedia pages were also included as additional training data in the task’s evaluation process. The corpus was then automatically updated to synthetically include realistic errors to produce a training and evaluation ground truth comparison. We designed and evaluated two supervised baseline WikiFixer error correction methods: (1) a naive approach based on a maximum likelihood character-level language model; (2) and an advanced model based on a sequence-to-sequence (seq2seq) neural network architecture. Both error correction models operated at a character level. When compared against an off-the-shelf word-level spell checker these methods showed a significant improvement in the task’s performance – with the seq2seq-based model correcting a higher number of errors than it introduced. Finally, we published our data and code at https://github.com/GM-RKB/LREC-2020. Keywords: Wiki Text, Typographical Error Correction, Character-level Language Models, seq2seq-based Error Correction 1. Introduction Wikis are one of the most popular resources for online col- laboration and knowledge exchange on which users can write and edit content directly through a web browser. Be- cause wiki software engines are easy to use and deploy, they have become the natural choice for creating and growing a large number of modern knowledge bases, dictionaries, academic libraries, and encyclopedias. Mediawiki, the wiki engine that runs the well-known Wikipedia (Junghans et al., 2008; Kr¨otzsch et al., 2006), for instance, currently powers a multitude of interactive websites including the online se- mantic wiki that this paper is focused on, the GM-RKB 1. Similar to the creation of websites in the early days of the Web; when websites content was crafted with HTML code by hand than transformed by a browser rendering engine into web pages, wikis content is created using WikiText (Dohrn and Riehle, 2011) written and edited predominantly by humans then parsed and rendered by a wiki engine (Dohrn and Riehle, 2011; Junghans et al., 2008). WikiText is a simplified markup language that facilitates annotation of text documents. To create an internal link between anno- tated words (concept mentions) and a target wiki pages in Mediawiki markup, the editor simply needs to use double square brackets (“[[” and “]]”). For instance, the following wikitext: “A [[Character-Level Seq2Seq Training Algorithm|character-level seq2seq algorithm]] is a [[seq2seq algorithm]] that is a [[character-level NNet algorithm]].” will be displayed as “A character-level seq2seq algorithm is a seq2seq algorithm that is a character-level NNet algo- rithm” and link the concept mentions to the corresponding wiki entries. 1http://GM-RKB.com …

Content truncated to 500 words