Using Confidence Bands for Parallel Texts Alignment

Using Confidence Bands for Parallel Texts Alignment António RIBEIRO Departamento de Informática Faculdade de Ciências e Tecnologia Universidade Nova de Lisboa Quinta da Torre P-2825-114 Monte da Caparica Portugal ambar@di.fct.unl.pt Gabriel LOPES Departamento de Informática Faculdade de Ciências e Tecnologia Universidade Nova de Lisboa Quinta da Torre P-2825-114 Monte da Caparica Portugal gpl@di.fct.unl.pt João MEXIA Departamento de Matemática Faculdade de Ciências e Tecnologia Universidade Nova de Lisboa Quinta da Torre P-2825-114 Monte da Caparica Portugal Abstract This paper describes a language independent method for alignment of parallel texts that makes use of homograph tokens for each pair of languages. In order to filter out tokens that may cause misalignment, we use confidence bands of linear regression lines instead of heuristics which are not theoreti- cally supported. This method was originally inspired on work done by Pascale Fung and Kathleen McKeown, and Melamed, provid- ing the statistical support those authors could not claim. Introduction Human compiled bilingual dictionaries do not cover every term translation, especially when it comes to technical domains. Moreover, we can no longer afford to waste human time and effort building manually these ever changing and in- complete databases or design language specific applications to solve this problem. The need for an automatic language independent task for equivalents extraction becomes clear in multi- lingual regions like Hong Kong, Macao, Quebec, the European Union, where texts must be translated daily into eleven languages, or even in the U.S.A. where Spanish and English speaking communities are intermingled. Parallel texts (texts that are mutual transla- tions) are valuable sources of information for bilingual lexicography. However, they are not of much use unless a computational system may find which piece of text in one language corre- sponds to which piece of text in the other lan- guage. In order to achieve this, they must be aligned first, i.e. the various pieces of text must be put into correspondence. This makes the translations extraction task easier and more reli- able. Alignment is usually done by finding correspondence points – sequences of characters with the same form in both texts (homographs, e.g. numbers, proper names, punctuation marks), similar forms (cognates, like Region and Região in English and Portuguese, respectively) or even previously known translations. Pascale Fung and Kathleen McKeown (1997) present an alignment algorithm that uses term translations as correspondence points between English and Chinese. Melamed (1999) aligns texts using correspondence points taken either from orthographic cognates (Michel Simard et al., 1992) or from a seed translation lexicon. However, although the heuristics both ap- proaches use to filter noisy points may be intui- tively quite acceptable, they are not theoretically supported by Statistics. The former approach considers a candidate correspondence point reliable as long as, among some other constraints, “[...] it is not too far away from the diagonal [...]” (Pascale Fung and Kathleen McKeown, 1997, p.72) of a rectangle whose sides sizes are proportional to the lengths of the texts in each language (henceforth, ‘the golden translation diagonal’). The latter ap- proach uses …

Content truncated to 500 words