GIZA++

AbstractOur statistical machine translation system that uses large Japanese-English parallel sentences and long phrase tables is described. We collected 698,973 Japanese-English parallel sentences, and we used long phrase tables. Also, …...

The alignment problem—establishing links between corresponding phrases in two related sentences—is as important in natural language inference (NLI) as it is in machine transla- tion (MT). But the tools and …...

We present a method to align words in a bitext that combines elements of a tra- ditional statistical approach with linguis- tic knowledge. We demonstrate this ap- proach for Arabic-English, …...

We describe the LIU systems for English- German and German-English translation in the WMT09 shared task. We focus on two methods to improve the word align- ment: (i) by applying …...

Word alignment plays a central role in statisti- cal MT (SMT) since almost all SMT systems extract translation rules from word aligned parallel training data. While most SMT systems use …...

While Inversion Transduction Grammar (ITG) has regained more and more attention in recent years, it still suffers from the major obstacle of speed. We propose a discriminative ITG prun- ing …...

In this paper we present a na¨ıve approach to tackle the problem of cross-lingual WSD and cross-lingual lexical substitu- tion which correspond to the Task #2 and #3 of the …...

We explore the possibility of using Stochastic Bracketing Linear Inversion Transduction Grammars for a full-scale German–English translation task, both on their own and in conjunction with align- ments induced with …...

AbstractWord alignment is an important step for machine translation systems. Although the alignment performance between grammatically similar languages is reported to be very high in many studies, the case is …...

This paper describes QCRI-MES’s sub- mission on the English-Russian dataset to the Eighth Workshop on Statistical Ma- chine Translation. We generate improved word alignment of the training data by incorporating …...

AbstractThis paper presents a method for verb phrase (VP) alignment in an English-French parallel corpus and its use for improving statistical machine translation (SMT) of verb tenses. The method starts …...

AbstractIt is well known that word aligned parallel corpora are valuable linguistic resources. Since many factors affect automatic alignment quality, manual post-editing may be required in some applications. While there …...

Clustering words is a widely used technique in statistical natural language processing. It re- quires syntactic, semantic, and contextual fea- tures. Especially, semantic clustering is gain- ing a lot of …...

Expectation-maximization algorithms, such as those implemented in GIZA++ pervade the field of unsupervised word alignment. However, these algorithms have a problem of over-fitting, leading to “garbage collector effects,” where rare …...

We describe a new technique for improv- ing statistical machine translation training by adopting scores from a recent crosslin- gual semantic frame based evaluation met- ric, XMEANT, as outside probabilities …...

AbstractDespite its original goal to jointly learn to align and translate, prior researches suggest that Transformer captures poor word alignments through its attention mechanism. In this paper, we show that …...

AbstractWord alignment was once a core unsupervised learning task in natural language processing because of its essential role in training statistical machine translation (MT) models. Although unnecessary for training neural …...

Word alignments identify translational correspondences between words in a parallel sentence pair and are used, for instance, to learn bilingual dictionaries, to train statistical machine trans- lation systems or to …...

AbstractA popular natural language processing task decades ago, word alignment has been dominated until recently by GIZA++, a statistical method based on the 30-year-old IBM models. New methods that outperform …...

AbstractWord alignment is essential for the downstream cross-lingual language understanding and generation tasks. Recently, the performance of the neural word alignment models has exceeded that of statistical models. However, they …...