Wiktionary Normalization of Translations and Morphological Information

Proceedings of the 28th International Conference on Computational Linguistics, pages 4683–4692 Barcelona, Spain (Online), December 8-13, 2020 4683 Wiktionary Normalization of Translations and Morphological Information Winston Wu and David Yarowsky Department of Computer Science Center for Language and Speech Processing Johns Hopkins University {wswu,yarowsky}@jhu.edu Abstract We extend the Yawipa Wiktionary Parser (Wu and Yarowsky, 2020) to extract and normalize translations from etymology glosses, and morphological form-of relations, resulting in 300K unique translations and over 4 million instances of 168 annotated morphological relations. We propose a method to identify typos in translation annotations. Using the extracted morphological data, we develop multilingual neural models for predicting three types of word formation— clipping, contraction, and eye dialect—and improve upon a standard attention baseline by using copy attention. 1 Introduction Wiktionary is a large, free multilingual dictionary with a wealth of information. Yawipa (Wu and Yarowsky, 2020), henceforth W&Y, is a recent Wiktionary parser billed as “comprehensive and extensible.” It has the ability to extract numerous types information from Wiktionary, including pronunciations, part of speech, translations, etymology, and a wide range of word relations, and normalize it into an easy to process tabular format. In particular, one of Yawipa’s innovations over existing parsers was extracting translations from the definition section of a dictionary definition. Confirming its easy extensibility and improving upon its comprehensiveness, we extend Yawipa’s extraction and normalization of Wiktionary in two directions: we extract translations from an unusual source, etymology glosses, and we extract morphological relations as annotated by form-of relations. This results in an addition of 282,092 new unique translations and 4,027,201 extracted morphological relations (from the 2020-04 English Wiktionary XML dump). We present an analysis that enables us to find typos in translation annotations. Using the extracted morphological data, we experiment with several new low-resource (1.5K instances) multilingual prediction tasks on clipping, contraction, and eye dialect. Our experiments with neural sequence-to- sequence models show that using copy attention can improve performance by up to 52% over a model with a standard attention mechanism. 2 Related Work Though Wiktionary has existed since 2002, only until very recently has there been a surge of interest in using Wiktionary. Navarro et al. (2009) was one of the first to examine Wiktionary as a resource for NLP. This paper builds upon Yawipa (Wu and Yarowsky, 2020), an open-source, extensible Wiktionary parsing framework written in Julia with support for parsing a wide variety of data from multiple language editions of Wiktionary into a structured machine-readable format. Yawipa’s goal is to be comprehensive and extensible. To that end, Yawipa goes beyond existing parsers in extracting and normalizing information, such as etymology and translations, that exist outside of structured Wiktionary markup (we further this goal in this paper), and it facilitates the creation of new parsers for other Wiktionary editions. In the literature, there are similar Wiktionary parsing efforts (e.g. knoWitiary (Nastase and Strapparava, 2015), DBnary (S´erasset, 2015), and ENGLAWI (Sajous et al., 2020)), but with different goals and coverage. This work is licensed under a Creative …

Content truncated to 500 words