Метрики Качества
Related Papers
EVALIEX — A Proposal for an Extended Evaluation Methodology for Information Extraction Systems
AbstractAssessing the correctness of extracted data requires performance evaluation, which is accomplished by calculating quality metrics. The evaluation process must cope with the challenges posed by information extraction and natural …...
A Survey of Current Datasets for Vision and Language Research
Integrating vision and language has long been a dream in work on artificial intel- ligence (AI). In the past two years, we have witnessed an explosion of work that brings …...
Integrating Meaning into Quality Evaluation of Machine Translation
AbstractMachine translation (MT) quality is evaluated through comparisons between MT outputs and the human translations (HT). Traditionally, this evaluation relies on form related features (e.g. lexicon and syntax) and ignores …...
Behavior of Modern Pre-trained Language Models Using the Example of Probing Tasks
AbstractModern transformer-based language models are revolutionizing NLP. However, existing studies into language modelling with BERT have been mostly limited to English-language material and do not pay enough attention to the …...
Bias Mitigation in Machine Translation Quality Estimation
AbstractMachine Translation Quality Estimation (QE) aims to build predictive models to assess the quality of machine-generated translations in the absence of reference translations. While state-of-the-art QE models have been shown …...