Improved Lexical Alignment by Combining Multiple Reified Alignments

Improved Lexical Alignment by Combining Multiple Reified Alignments Dan Tufi� Institute for Artificial Intelligence 13, “13 Septembrie”, 050711, Bucharest 5, Romania tufis@racai.ro Radu Ion Institute for Artificial Intelligence 13, “13 Septembrie”, 050711, Bucharest 5, Romania radu@racai.ro Alexandru Ceau�u Institute for Artificial Intelligence 13, “13 Septembrie”, 050711, Bucharest 5, Romania alceausu@racai.ro Dan �tef�nescu Institute for Artificial Intelligence 13, “13 Septembrie”, 050711, Bucharest 5, Romania danstef@racai.ro Abstract We describe a word alignment platform which ensures text pre-processing (to- kenization, POS-tagging, lemmatization, chunking, sentence alignment) as re- quired by an accurate word alignment. The platform combines two different methods, producing distinct alignments. The basic word aligners are described in some details and are individually evalu- ated. The union of the individual align- ments is subject to a filtering post- processing phase. Two different filtering methods are also presented. The evalua- tion shows that the combined word alignment contains 10.75% less errors than the best individual aligner. 1 Introduction It is almost a truism that more decision makers, working together, are likely to find a better solu- tion than when working alone. Dieterich (1998) discusses conditions under which different deci- sions (in his case classifications) may be com- bined for obtaining a better result. Essentially, a successful automatic combination method would require comparable performance for the decision makers and, additionally, that they should not make similar errors. This idea has been exploited by various NLP researchers in language model- ling, statistical POS tagging, parsing, etc. We developed two quite different word align- ers, driven by two distinct objectives: the first one was motivated by a project aiming at the de- velopment of an interlingually aligned set of wordnets while the other one was developed within an SMT ongoing project. The first one was used for validating, against a multilingual corpus, the interlingual synset equivalences and also for WSD experiments. Although, initially, it was concerned only with open class words re- corded in a wordnet, turning it into an “all words” aligner was not a difficult task. This word aligner, called YAWA is described in sec- tion 3. A quite different approach from the one used by YAWA, is implemented in our second word aligner, called MEBA, described in section 4. It is a multiple parameter and multiple step algo- rithm using relevance thresholds specific to each parameter, but different from each step to the other. The implementation of MEBA was strongly influenced by the notorious five IBM models described in (Brown et al. 1993). We used GIZA++ (Och and Ney 2000; Och and Ney, 2003) to estimate different parameters of the MEBA aligner. The alignments produced by MEBA were compared to the ones produced by YAWA and evaluated against the Gold Standard (GS)1 anno- tations used in the Word Alignment Shared Tasks (Romanian-English track) organized at HLT-NAACL2003 (Mihalcea and Pedersen 2003). Given that the two aligners are based on quite different models and that their F-measures are comparable, it was quite a natural idea to com- bine their results and hope for an improved …

Content truncated to 500 words