NeurAlign: Combining Word Alignments Using Neural Networks

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 65–72, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics NeurAlign: Combining Word Alignments Using Neural Networks Necip Fazil Ayan, Bonnie J. Dorr and Christof Monz Department of Computer Science University of Maryland College Park, MD 20742 {nfa,bonnie,christof}@umiacs.umd.edu Abstract This paper presents a novel approach to combining different word alignments. We view word alignment as a pattern classifi- cation problem, where alignment combi- nation is treated as a classifier ensemble, and alignment links are adorned with lin- guistic features. A neural network model is used to learn word alignments from the individual alignment systems. We show that our alignment combination approach yields a significant 20-34% relative er- ror reduction over the best-known align- ment combination technique on English- Spanish and English-Chinese data. 1 Introduction Parallel texts are a valuable resource in natural lan- guage processing and essential for projecting knowl- edge from one language onto another. Word-level alignment is a critical component of a wide range of NLP applications, such as construction of bilingual lexicons (Melamed, 2000), word sense disambigua- tion (Diab and Resnik, 2002), projection of language resources (Yarowsky et al., 2001), and statistical ma- chine translation. Although word-level aligners tend to perform well when there is sufficient training data, the quality decreases as the size of training data de- creases. Even with large amounts of training data, statistical aligners have been shown to be suscepti- ble to mis-aligning phrasal constructions (Dorr et al., 2002) due to many-to-many correspondences, mor- phological language distinctions, paraphrased and free translations, and a high percentage of function words (about 50% of the tokens in most texts). This paper presents a novel approach to align- ment combination, NeurAlign, that treats each align- ment system as a black box and merges their outputs. We view word alignment as a pattern classification problem and treat alignment combination as a classi- fier ensemble (Hansen and Salamon, 1990; Wolpert, 1992). The ensemble-based approach was devel- oped to select the best features of different learning algorithms, including those that may not produce a globally optimal solution (Minsky, 1991). We use neural networks to implement the classifier-ensemble approach, as these have previ- ously been shown to be effective for combining clas- sifiers (Hansen and Salamon, 1990). Neural nets with 2 or more layers and non-linear activation func- tions are capable of learning any function of the feature space with arbitrarily small error. Neural nets have been shown to be effective with (1) high- dimensional input vectors, (2) relatively sparse data, and (3) noisy data with high within-class variability, all of which apply to the word alignment problem. The rest of the paper is organized as follows: In Section 2, we describe previous work on improv- ing word alignments and use of classifier ensembles in NLP. Section 3 gives a brief overview of neu- ral networks. In Section 4, we present a new ap- proach, NeurAlign, that learns how to combine indi- vidual word alignment …

Content truncated to 500 words