Confidence Estimation
Related Papers
Confidence estimation for translation prediction
The purpose of this work is to investigate the use of machine learning approaches for confi- dence estimation within a statistical machine translation application. Specifically, we at- tempt to learn …...
Relaxed Marginal Inference and its Application to Dependency Parsing
Recently, relaxation approaches have been successfully used for MAP inference on NLP problems. In this work we show how to extend the relaxation approach to marginal inference used in conditional …...
Are You Sure? Confidence in Prediction of Dependency Tree Edges
We describe and evaluate several methods for estimating the confidence in the per-edge cor- rectness of a predicted dependency parse. We show empirically that the confidence is asso- ciated with …...
A Study on Bootstrapping Bilingual Vector Spaces from Non-Parallel Data (and Nothing Else)
We present a new language pair agnostic ap- proach to inducing bilingual vector spaces from non-parallel data without any other re- source in a bootstrapping fashion. The pa- per systematically …...
Novel elicitation and annotation schemes for sentential and sub-sentential alignments of bitexts
AbstractResources for evaluating sentence-level and word-level alignment algorithms are unsatisfactory. Regarding sentence alignments, the existing data is too scarce, especially when it comes to difficult bitexts, containing instances of non-literal …...
Disentangling ASR and MT Errors in Speech Translation
The main aim of this paper is to investigate automatic quality assessment for spoken language translation (SLT). More precisely, we investigate SLT errors that can be due to transcription (ASR) …...
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing
AbstractIn selective prediction, a classifier is allowed to abstain from making predictions on low-confidence examples. Though this setting is interesting and important, selective prediction has rarely been examined in natural …...
Learning Confidence for Transformer-based Neural Machine Translation
AbstractConfidence estimation aims to quantify the confidence of the model prediction, providing an expectation of success. A well-calibrated confidence estimate enables accurate failure prediction and proper risk measurement when given …...
A Close Look into the Calibration of Pre-trained Language Models
AbstractPre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty. We take a close look into this problem, aiming to answer two questions: (1) Do PLMs …...