A language-independent method for the alignement of parallel corpora

A language-independent method for the alignement of parallel corpora Nguy˜ên Thi. Minh Huy`ên1 and Mathias Rossignol2 1 Hanoi University of Sciences 334 Nguy˜ên Trãi, Hà Nô.i, Vietnam huyenntm@vnu.edu.vn 2 International Research Center MICA HUT - CNRS/UMI 2954 - INPG 1, Ð.ai C`ô Viê.t, Hà Nô.i, Vietnam mathias.rossignol@mica.edu.vn Abstract. The automatic alignment of parallel corpora is a very rich source of information for automatic translation, multilingual document indexing, information retrieval, etc. The rapid growth of the use of “ minority ” languages in online documents makes it necessary to develop methods that can easily adapt to any language. We present an evolution over previous works, notably by Church and Gale [1], that performs the alignement of parallel texts in any language without any need for information concerning these languages, as is often the case in existing systems. We also introduce a more experimental system, that shows promise for the alignment of degraded translations. The systems have taken part to the ARCADE II evaluation campaign [2], of which we present the results. 1 Introduction As more and more countries and cultures develop a popular use of the Internet, the amount of distinct languages composing the information found online has grown tremendously in the last five years. Once a mostly English-speaking medium totally based on the latin alphabet, Internet has now become a place where information is represented in radically different ways, in languages that for example may not even share the basic notion of “ word ”. Being able to find relevant information for a given need in any language, whatever language the need is expressed in, is one of the challenges associated with that evolution. A much explored way to solve it is, of course, automatic translation; the problem of that approach is that each new language to take into account involves a huge amount of human work for the creation of lexicons, grammars, etc., which soon becomes un-tractable as the number of languages increases. But full text translation is not necessary to know that two documents address a same topic: often, a few known keywords or statistical properties of the considered texts are sufficient for that task. The interest of that more “ loose ” approach is that the information needed can be automatically acquired from training data presented as aligned parallel corpora. Parallel corpora are instances of a same text written in different languages, and aligning them means exposing the links between portions of text bearing the same meaning. The interest of that information for the automatic acquisition of knowledge concerning semantic equivalences between languages is of course very important. Although parallel corpora are quite frequent– for example, multilingual news outlets –, aligning them still requires work. Doing so manually is extremely tedious and time-consuming, and studies have therefore been conducted in order to do it automatically. Church and Gale [1] were among the first to propose a system to perform the task of sentence- level parallel text alignment. Given two texts carrying the same message, they make the …

Content truncated to 500 words