Articles

In this study, we perform an investigation of coreference resolution in the biomed- ical literature. We compare a state-of- the-art general system with a purpose- built system, demonstrating that the use of domain-specific knowledge results in dramatic improvement. However, perfor- mance of the system is still modest, with recall a …...

As larger and more diverse parallel texts become available, how can we lever- age heterogeneous data to train robust machine translation systems that achieve good translation quality on various test domains? This challenge has been ad- dressed so far by repurposing techniques developed for domain adaptation, such as linear mixture …...

In phrase-based statistical machine trans- lation systems, variation in grammatical structures between source and target lan- guages can cause large movements of phrases. Modeling such movements is cru- cial in achieving translations of long sen- tences that appear natural in the target lan- guage. We explore generative learning approach to …...

Scalable discriminative training methods are now broadly available for estimating phrase-based, feature-rich translation mod- els. However, the sparse feature sets typi- cally appearing in research evaluations are less attractive than standard dense features such as language and translation model probabilities: they often overfit, do not gen- eralize, or require complex …...

In this work, we tackle the problem of language and translation models domain- adaptation without explicit bilingual in- domain training data. In such a scenario, the only information about the domain can be induced from the source-language test corpus. We explore unsupervised adaptation, where the source-language test corpus is combined …...

Despite its potential to improve lexical selection, most state-of-the-art machine translation systems take only minimal con- textual information into account. We cap- ture context with a topic model over dis- tributional profiles built from the context words of each translation unit. Topic dis- tributions are inferred for each transla- tion …...

High-quality parallel data is crucial for a range of multilingual applications, from tuning and evaluating machine translation systems to cross-lingual annotation pro- jection. Unfortunately, automatically ob- tained parallel data (which is available in relative abundance) tends to be quite noisy. To obtain high-quality parallel data, we introduce a crowdsourcing paradigm …...

We present the UvA-ILLC submission of the BEER metric to WMT 14 metrics task. BEER is a sentence level metric that can incorporate a large number of features combined in a linear model. Novel con- tributions are (1) efficient tuning of a large number of features for maximizing corre- lation …...

We present an effective technique to easily augment GHKM-style syntax-based ma- chine translation systems (Galley et al., 2006) with phrase pairs that do not comply with any syntactic well-formedness con- straints. Non-syntactic phrase pairs are distinguished from syntactic ones in or- der to avoid harming effects. We apply our technique …...

This paper describes a machine translation metric submitted to the WMT14 Metrics Task. It is a simple modification of the standard BLEU metric using a monolin- gual alignment of reference and test sen- tences. The alignment is computed as a minimum weighted maximum bipartite matching of the translated and the …...

We present novel automatic metrics for machine translation evaluation that use discourse structure and convolution ker- nels to compare the discourse tree of an automatic translation with that of the hu- man reference. We experiment with five transformations and augmentations of a base discourse tree representation based on the rhetorical …...

In previous work we showed that when us- ing an SMT model trained on old-domain data to translate text in a new-domain, most errors are due to unseen source words, unseen target translations, and in- accurate translation model scores (Irvine et al., 2013a). In this work, we target er- rors …...

Based on the last year’s DCU-CASIST participation on WMT metrics task, we further improve our model in the follow- ing ways: 1) parameter tuning 2) support languages other than English. We tuned our system on all the data of WMT 2010, 2012 and 2013. The tuning results as well as …...

This paper describes the LAYERED met- ric which is used for the shared WMT’14 metrics task. Various metrics exist for MT evaluation: BLEU (Papineni, 2002), ME- TEOR (Alon Lavie, 2007), TER (Snover, 2006) etc., but are found inadequate in quite a few language settings like, for ex- ample, in case …...

As described in this paper, we pro- pose a new automatic evaluation met- ric for machine translation. Our met- ric is based on chunking between the reference and candidate translation. Moreover, we apply a prize based on sentence-length to the metric, dissim- ilar from penalties in BLEU or NIST. We …...

This paper describes Meteor Universal, re- leased for the 2014 ACL Workshop on Statistical Machine Translation. Meteor Universal brings language specific evalu- ation to previously unsupported target lan- guages by (1) automatically extracting lin- guistic resources (paraphrase tables and function word lists) from the bitext used to train MT systems …...

In this paper we present VERTa, a lin- guistically-motivated metric that com- bines linguistic features at different lev- els. We provide the linguistic motivation on which the metric is based, as well as describe the different modules in VERTa and how they are combined. Finally, we describe the two versions …...

BLEU is the de facto standard machine translation (MT) evaluation metric. How- ever, because BLEU computes a geo- metric mean of n-gram precisions, it of- ten correlates poorly with human judg- ment on the sentence-level. There- fore, several smoothing techniques have been proposed. This paper systemati- cally compares 7 smoothing …...

This paper describes Parmesan, our sub- mission to the 2014 Workshop on Sta- tistical Machine Translation (WMT) met- rics task for evaluation English-to-Czech translation. We show that the Czech Me- teor Paraphrase tables are so noisy that they actually can harm the performance of the metric. However, they can be …...

This paper describes the UPC submissions to the WMT14 Metrics Shared Task: UPC- IPA and UPC-STOUT. These metrics use a collection of evaluation measures in- tegrated in ASIYA, a toolkit for machine translation evaluation. In addition to some standard metrics, the two submissions take advantage of novel metrics that consider …...