Tools for The Production of Analogical Grids and a Resource of N-gram Analogical Grids in 11 Languages

Tools for The Production of Analogical Grids and a Resource of N-gram Analogical Grids in 11 Languages Rashel Fam, Yves Lepage Graduate School of Information, Production, and Systems Waseda University 2-7 Hibikino, Wakamatsu-ku, Kitakyushu-shi, 808-0135 Fukuoka-ken, Japan {fam.rashel@fuji., yves.lepage@}waseda.jp Abstract We release a Python module containing several tools to build analogical grids from words contained in a corpus. The module implements several previously presented algorithms. The tools are language-independent. This permits their use with any language and any writing system. We hope that the tools will ease research in morphology by allowing researchers to automatically obtain structured representations of the vocabulary contained in corpora or linguistic data. We also release analogical grids built on the vocabularies contained in 1,000 corresponding lines of the 11 different language versions of the Europarl corpus v.3. The grids were built on N-grams of different lengths, from words to 6-grams. We hope that the use of structured parallel data will foster research in comparative linguistics. Keywords: Analogy, morphology, analogical grids 1. Introduction Paradigm tables are known for their usefulness in learn- ing conjugation or declension when studying a language. Such paradigm tables are the result of grammatical tradi- tion or thorough linguistic formalization. They are com- monly found in dictionaries and constructed from lexemes and exponents, like the one shown in Figure 2 (left) which is taken from a French & English dictionary (Mansion, 1981, grey section, p. 1). Analogical grids are not paradigm tables, but they also give a compact view on the organization of a lexicon up to a certain extent (see below, Section 2.1.). Analogical grids are the result of an empirical procedure. They may be seen as a preliminary step towards the production of paradigm tables. Figure 2 (right) shows an example of an analogical grid in English. They can be used to study the productivity of a language (Singh and Ford, 2000; Neuvel and Fulop, 2002; Hathout, 2008). Fam and Lepage (2016) performed such an analysis across 12 languages using analogical grids built from the Bible corpus (Christodouloupoulos, 2015). As another example of use, Hathout (2009) showed how to produce the French word form rectification by analogy from the neighboring word forms fructifier, fructification, and rectifier in the same series (see Figure 1). fructifier —————— family fructification ———– series series ———– rectifier family —————— rectification fructifier : fructification :: rectifier : rectification Figure 1: Producing a new word form from neighboring word forms. Example taken from (Hathout, 2009) This paper introduces the release of a Python module which implements previously presented algorithms that automat- ically build analogical grids. We also release analogical clusters and analogical grids produced from a parallel cor- pus on 11 European languages using this Python module. 2. Main Usages of the Tools Released We release an implementation of previously presented algo- rithms to produce analogical grids as a Python 2 module1 called Nlg. The various algorithms have been presented elsewhere (Lepage, 1998; Lepage, 2014; Fam and Lepage, 2016). One particular program called Words2Grids in this module …

Content truncated to 500 words