Automatic Evaluation Metrics

Often, the training procedure for statisti- cal machine translation models is based on maximum likelihood or related criteria. A general problem of this approach is that there is only a …...

. The first research on integrating controlled language data in an Example-Based Machine Translation (EBMT) system was published in [Gough & Way, 2003]. We improve on their sub-sentential alignment algorithm …...

Recent studies suggest that machine learn- ing can be applied to develop good auto- matic evaluation metrics for machine trans- lated sentences. This paper further ana- lyzes aspects of learning …...

AbstractIn emergency situations it is crucial that instructions are straightforward to understand. For this reason a controlled language for crisis management (CLCM), based on psycholinguistic studies of human comprehension under …...

Automatic evaluation metrics are fundamen- tally important for Machine Translation, al- lowing comparison of systems performance and efficient training. Current evaluation met- rics fall into two classes: heuristic approaches, like …...

AbstractExisting personalized dialogue models use human designed persona descriptions to improve dialogue consistency. Collecting such descriptions from existing dialogues is expensive and requires hand-crafted feature designs. In this paper, we …...

AbstractUnsupervised text style transfer is full of challenges due to the lack of parallel data and difficulties in content preservation. In this paper, we propose a novel neural approach to …...

AbstractMany automatic evaluation metrics have been proposed to score the overall quality of a response in open-domain dialogue. Generally, the overall quality is comprised of various aspects, such as relevancy, …...

AbstractWe performed a detailed error analysis in domain-specific neural machine translation (NMT) for the English and Japanese language pair with fine-grained manual annotation. Despite its importance for advancing NMT technologies, …...

AbstractIn this paper, we address the task of news-image captioning, which generates a description of an image given the image and its article body as input. This task is more …...

AbstractIn this paper, we study review generation given a set of attribute identifiers which are user ID, product ID and rating. This is a difficult subtask of natural language generation …...

AbstractThis paper studies the effects of word-level linguistic annotations in under-resourced neural machine translation, for which there is incomplete evidence in the literature. The study covers eight language pairs, different …...

The lack of reliable automatic evaluation met- rics is a major impediment to the development of open-domain dialogue systems. Various reference-based metrics have been proposed to calculate a score between …...

AbstractChinese couplet generation aims to generate a pair of clauses (usually generating a subsequent clause given an antecedent one) with certain rules (e.g., morphological and syntactical symmetry) adhered and has …...

AbstractAutomatic evaluation metrics are essential for the rapid development of open-domain dialogue systems as they facilitate hyper-parameter tuning and comparison between models. Although recently proposed trainable conversation-level metrics have shown …...

AbstractThis paper presents the results of the DELA Project. We describe the testing of context span for document-level evaluation, construction of a document-level corpus, and context position, as well as …...

AbstractDriven by deep learning breakthroughs, natural language generation (NLG) models have been at the center of steady progress in the last few years, with a ubiquitous task influence. However, since …...

AbstractThis paper describes Tencent’s multilingual machine translation systems for the WMT22 shared task on Large-Scale Machine Translation Evaluation for African Languages. We participated in the constrained translation track in which …...

AbstractThe use of automatic evaluation metrics to assess Machine Translation (MT) quality is well established in the translation industry. Whereas it is relatively easy to cover the word- and character-based …...

AbstractWe present FRMT, a new dataset and evaluation benchmark for Few-shot Region-aware Machine Translation, a type of style-targeted translation. The dataset consists of professional translations from English into two regional …...

AbstractThis work presents an unsupervised method of selecting filters and threshold values for the OpusFilter parallel corpus cleaning toolbox. The method clusters sentence pairs into noisy and clean categories and …...