Word Alignment in English-Hindi Parallel Corpus Using Recency-Vector Approach: Some Studies

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 649–656, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Word Alignment in English-Hindi Parallel Corpus Using Recency-Vector Approach: Some Studies Niladri Chatterjee Department of Mathematics Indian Institute of Technology Delhi Hauz Khas, New Delhi INDIA - 110016 niladri iitd@yahoo.com Saumya Agrawal Department of Mathematics Indian Institute of Technology Kharagpur, West Bengal INDIA - 721302 saumya agrawal2000@yahoo.co.in Abstract Word alignment using recency-vector based approach has recently become pop- ular. One major advantage of these tech- niques is that unlike other approaches they perform well even if the size of the par- allel corpora is small. This makes these algorithms worth-studying for languages where resources are scarce. In this work we studied the performance of two very popular recency-vector based approaches, proposed in (Fung and McKeown, 1994) and (Somers, 1998), respectively, for word alignment in English-Hindi parallel cor- pus. But performance of the above al- gorithms was not found to be satisfac- tory. However, subsequent addition of some new constraints improved the perfor- mance of the recency-vector based align- ment technique significantly for the said corpus. The present paper discusses the new version of the algorithm and its per- formance in detail. 1 Introduction Several approaches including statistical tech- niques (Gale and Church, 1991; Brown et al., 1993), lexical techniques (Huang and Choi, 2000; Tiedemann, 2003) and hybrid techniques (Ahren- berg et al., 2000), have been pursued to design schemes for word alignment which aims at estab- lishing links between words of a source language and a target language in a parallel corpus. All these schemes rely heavily on rich linguistic re- sources, either in the form of huge data of parallel texts or various language/grammar related tools, such as parser, tagger, morphological analyser etc. Recency vector based approach has been pro- posed as an alternative strategy for word align- ment. Approaches based on recency vectors typ- ically consider the positions of the word in the corresponding texts rather than sentence bound- aries. Two algorithms of this type can be found in (Fung and McKeown, 1994) and (Somers, 1998). The algorithms first compute the position vector Vw for the word w in the text. Typically, Vw is of the form ⟨p1p2 . . . pk⟩, where the pis indicate the positions of the word w in a text T. A new vector Rw, called the recency vector, is computed using the position vector Vw, and is defined as ⟨p2−p1, p3−p2, . . . , pk −pk−1⟩. In order to com- pute the alignment of a given word in the source language text, the recency vector of the word is compared with the recency vector of each target language word and the similarity between them is measured by computing a matching cost associ- ated with the recency vectors using dynamic pro- gramming. The target language word having the least cost is selected as the aligned word. The results given in the above references show that the …

Content truncated to 500 words