Identifying Word Translations in Non-Parallel Texts

Identifying Word Translations in Non-Parallel Texts Reinhard Rapp ISSCO, Universit6 de Gen~ve 54 route des Acacias Gen~ve, Switzerland rapp@divsun.unige.ch Abstract Common algorithms for sentence and word-alignment allow the automatic iden- tification of word translations from paxalhl texts. This study suggests that the identi- fication of word translations should also be possible with non-paxMlel and even unre- lated texts. The method proposed is based on the assumption that there is a corre- lation between the patterns of word co- occurrences in texts of different languages. 1 Introduction In a number of recent studies it has been shown that word translations can be automatically derived from the statistical distribution of words in bilingual pax- allel texts (e. g. Catizone, Russell & Warwick, 1989; Brown et al., 1990; Dagan, Church & Gale, 1993; Kay & Rbscheisen, 1993). Most of the proposed algorithms first conduct an alignment of sentences, i. e. those palxs of sentences axe located that are translations of each other. In a second step a word alignment is performed by analyzing the correspon- dences of words in each pair of sentences. The results achieved with these algorithms have been found useful for the compilation of dictionaries, for checking the consistency of terminological usage in translations, and for assisting the terminological work of translators and interpreters. However, despite serious efforts in the compilation of corpora (Church & Mercer, 1993; Armstrong & Thompson, 1995) the availability of a large enough paxallel corpus in a specific field and for a given pair of languages will always be the exception, not the rule. Since the acquisition of non-paxallel texts is usually much easier, it would be desirable to have a program that can determine the translations of words from comparable or even unrelated texts. 2 Approach It is assumed that there is a correlation between the co-occurrences of words which are translations of each other. If - for example - in a text of one language two words A and B co-occur more often than expected from chance, then in a text of an- other language those words which axe translations of A and B should also co-occur more frequently than expected. This assumption is reasonable for parallel texts. However, in this paper it is further assumed that the co-occurrence patterns in original texts axe not fundamentally different from those in translated texts. Starting from an English vocabulary of six words and the corresponding German translations, table la and b show an English and a German co-occurrence mat~x. In these matrices the entries belonging to those pairs of words that in texts co-occur more fre- quently than expected have been marked with a dot. In general, word order in the lines and columns of a co-occurrence matrix is independent of each other, but for the purpose of this paper can always be as- sumed to be equal without loss of generality. If now the word order of the English matrix is per- muted until the resulting pattern of dots is most sim- ilar to that …

Content truncated to 500 words