Bilingual Lexicon Extraction at the Morpheme Level Using Distributional Analysis

Bilingual Lexicon Extraction at the Morpheme Level Using Distributional Analysis Amir HAZEM and B´eatrice DAILLE LINA, Universit´e de Nantes. 2 rue de la houssini`ere 44000 Nantes, France. amir.hazem@univ-nantes.fr, beatrice.daille@univ-nantes.fr Abstract Bilingual lexicon extraction from comparable corpora is usually based on distributional methods when dealing with single word terms (SWT). These methods often treat SWT as single tokens without considering their compositional property. However, many SWT are compositional (composed of roots and affixes) and this information, if taken into account, can be very useful to match translational pairs, especially for infrequent terms where distributional methods often fail. For instance, the English compound xenograft which is composed of the root xeno and the lexeme graft can be translated into French compositionally by aligning each of its elements (xeno with x´eno and graft with greffe) resulting in the translation: x´enogreffe. In this paper, we experiment several distributional modellings at the morpheme level that we apply to perform compositional translation to a subset of French and English compounds. We show promising results using distributional analysis at the root and affix levels. We also show that the adapted approach significantly improve bilingual lexicon extraction from comparable corpora compared to the approach at the word level. Keywords: Bilingual lexicon extraction, comparable corpora, morphemes, compound term, distributional analysis 1. Introduction Nowadays comparable corpora are widely used in many applications of natural language processing, particularly in bilingual terminology extraction where parallel corpora are a scarce resource (Rapp, 1995; Fung, 1995; Chiao and Zweigenbaum, 2002; Laroche and Langlais, 2010). In the task of bilingual lexicon extraction from comparable cor- pora, the acquisition of translational pairs is mainly based on the Harris’ distributional hypothesis (Harris, 1954) which states that words with similar meaning tend to oc- cur in similar contexts (hypothesis that has been extended to the bilingual scenario). It is well-known that the efficiency of distributional meth- ods heavily depends on the quality and the size of compara- ble corpora (Morin et al., 2007). If the quality of bilingual lexicons can always be improved by using more data, this is true only if the training data is reasonably well-matched to the desired output (Morin and Hazem, 2014). In the case of specialised comparable corpora, the amount of data is lim- ited and often small (Rapp, 1995; Morin et al., 2007). This presents a problem for distributional methods that fail to ex- tract infrequent single-word terms (SWTs) as well as multi- word terms (MWTs). In the case of MWTs, it has become a standard practice to apply the principle of composition- ality on its parts to extract their corresponding translations (Robitaille et al., 2006; Daille and Morin, 2008; Delpech et al., 2012b). While in the case of SWTs, distributional methods often treat them as single tokens without consid- ering their compositional property (Rapp, 1995; Chiao and Zweigenbaum, 2002; Laroche and Langlais, 2010). One can note that a reasonable amount of SWTs are compound terms consisting of a combination of two (or more) lexical elements to form a unit of meaning …

Content truncated to 500 words