Articles

This paper describes the latest version of the ATEC metric for automatic MT evaluation, with parameters optimized for word choice and word order, the two fundamental features of language that the metric relies on. The former is assessed by matching at various linguistic levels and weighting the informa- tiveness of …...

We present a unified approach to perform- ing minimum risk training and minimum Bayes risk (MBR) decoding with BLEU in a phrase-based model. Key to our ap- proach is the use of a Gibbs sampler that allows us to explore the entire probabil- ity distribution and maintain a strict prob- …...

We present TESLA-M and TESLA, two novel automatic machine translation eval- uation metrics with state-of-the-art perfor- mances. TESLA-M builds on the suc- cess of METEOR and MaxSim, but em- ploys a more expressive linear program- ming framework. TESLA further exploits parallel texts to build a shallow seman- tic representation. We …...

We present the MT-NCD and MT-mNCD machine translation evaluation metrics as submission to the machine transla- tion evaluation shared task (MetricsMATR 2010). The metrics are based on nor- malized compression distance (NCD), a general information theoretic measure of string similarity, and evaluated against hu- man judgments from the WMT08 shared …...

This paper describes the joint submission of Universitat Polit`ecnica de Catalunya and Uni- versitat de Barcelona to the Metrics MaTr 2010 evaluation challenge, in collaboration with ELDA/ELRA. Our work is aimed at widening the scope of current automatic evaluation measures from sentence to document level. Preliminary ex- periments, based on …...

The ability to measure the quality of word order in translations is an important goal for research in machine translation. Cur- rent machine translation metrics do not adequately measure the reordering perfor- mance of translation systems. We present a novel metric, the LRscore, which di- rectly measures reordering success. The …...

BBN submitted system combination out- puts for Czech-English, German-English, Spanish-English, French-English, and All- English language pairs. All combinations were based on confusion network decod- ing. An incremental hypothesis alignment algorithm with flexible matching was used to build the networks. The bi-gram de- coding weights for the single source lan- guage …...

RWTH participated in the System Combi- nation task of the Fifth Workshop on Sta- tistical Machine Translation (WMT 2010). For 7 of the 8 language pairs, we com- bine 5 to 13 systems into a single con- sensus translation, using additional n-best reranking techniques in two of these lan- guage …...

This paper describes the JHU system combination scheme that was used in the WMT 2010 submission. The in- cremental alignment scheme of (Karakos et.al, 2008) was used for confusion net- work generation. The system order in the alignment of each sentence was learned using SVMs, following the work of (Karakos …...

We describe DCU’s LFG dependency- based metric submitted to the shared eval- uation task of WMT-MetricsMATR 2010. The metric is built on the LFG F-structure- based approach presented in (Owczarzak et al., 2007). We explore the following improvements on the original metric: 1) we replace the in-house LFG parser with …...

This paper describes our submission to the WMT10 Shared Evaluation Task and MetricsMATR10. We present a version of the METEOR-NEXT metric with para- phrase tables for five target languages. We describe the creation of these paraphrase tables and conduct a tuning experiment that demonstrates consistent improvement across all languages over …...

In this paper we describe machine learn- ing experiments that aim to characterise the content selection process for distin- guishing descriptions. Our experiments are based on two large corpora of human- produced descriptions of objects in rela- tively small visual scenes; the referring ex- pressions are annotated with their seman- …...

n-best parse reranking is an important technique for improving the accuracy of statistical parsers. Reranking is not con- strained by the dynamic programming re- quired for tractable parsing, so arbitrary features of each parse may be considered. We adapt the reranking features and methodology used by Charniak and John- son …...

In this paper we motivate the need for a corpus for the development and testing of summarisation systems for evidence- based medicine. We describe the corpus which we are currently creating, and show its applicability by evaluating several sim- ple query-based summarisation techniques using a small fragment of the corpus. …...

Early use of corpora for language learn- ing has included analysis of word usage via concordancing. In addition, some at- tempts have been made to use readabil- ity criteria for recommending reading to learners. In this paper we discuss var- ious tools and approaches for enhanced language learning support, including …...

This paper introduces a novel user clas- sification task in the context of web user forums. We present a definition of four basic user characteristics and an anno- tated dataset. We outline a series of ap- proaches for predicting user characteris- tics, utilising aggregated post features and user/thread network analysis …...

Pathology reports are used to store infor- mation about cells and tissues of a patient, and they are crucial to monitor the health of individuals and population groups. In this work we present an evaluation of su- pervised text classification models for the prediction of relevant categories in pathol- ogy …...

Preceding a phrase-based statistical ma- chine translation (PSMT) system by a syntactically-informed reordering prepro- cessing step has been shown to improve overall translation performance compared to a baseline PSMT system. However, the improvement is not seen for every sen- tence. We use a lattice input to a PSMT system in …...

This research focuses on improving in- formation access over troubleshooting- oriented technical user forums via thread- level analysis. We describe a modular task formulation and novel dataset, and go on to describe a series of preliminary classifi- cation experiments over the data. We find that a class composition strategy achieves …...

Written contracts are a fundamental framework for commercial and coopera- tive transactions and relationships. Lim- ited research has been published on the ap- plication of machine learning and natural language processing (NLP) to contracts. In this paper we report the classification of components of contract texts using ma- chine learning …...