Меры оценки
Related Papers
On Learning more Appropriate Selectional Restrictions
We present some variations affecting the association measure and thresholding on a technique for learning Selectional Re- strictions from on-line corpora. It uses a wide-coverage noun taxonomy and a statistical …...
Evaluation Tool for Rule-based Anaphora Resolution Methods
In this paper we argue that comparative evaluation in anaphora resolution has to be performed using the same pre-processing tools and on the same set of data. The paper proposes …...
Semantic Similarity for Detecting Recognition Errors in Automatic Speech Transcripts
Browsing through large volumes of spoken audio is known to be a challenging task for end users. One way to alleviate this prob- lem is to allow users to gist …...
CDER: Efficient MT Evaluation Using Block Movements
Most state-of-the-art evaluation measures for machine translation assign high costs to movements of word blocks. In many cases though such movements still result in correct or almost correct sentences. In …...
SemEval-2007 Task 08: Metonymy Resolution at SemEval-2007
We provide an overview of the metonymy resolution shared task organised within SemEval-2007. We describe the problem, the data provided to participants, and the evaluation measures we used to assess …...
Assessing Dialog System User Simulation Evaluation Measures Using Human Judges
Previous studies evaluate simulated dialog corpora using evaluation measures which can be automatically extracted from the dialog systems’ logs. However, the validity of these automatic measures has not been fully …...
Topic Identification for Fine-Grained Opinion Analysis
Within the area of general-purpose fine- grained subjectivity analysis, opinion topic identification has, to date, received little attention due to both the difficulty of the task and the lack of …...
Evaluating Multilingual Question Answering Systems at CLEF
AbstractThe paper offers an overview of the key issues raised during the seven years activity of the Multilingual Question Answering Track at the Cross Language Evaluation Forum (CLEF). The general …...
Discriminative Language Models as a Tool for Machine Translation Error Analysis
In this paper, we propose a new method for effective error analysis of machine translation (MT) systems. In previous work on error analysis of MT, error trends are often shown …...
Tw-StAR at SemEval-2018 Task 1: Preprocessing Impact on Multi-label Emotion Classification
AbstractIn this paper, we describe our contribution in SemEval-2018 contest. We tackled task 1 “Affect in Tweets”, subtask E-c “Detecting Emotions (multi-label classification)”. A multilabel classification system Tw-StAR was developed …...
Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection
AbstractThe cognitive mechanisms needed to account for the English past tense have long been a subject of debate in linguistics and cognitive science. Neural network models were proposed early on, …...
An Evaluation Method for Diachronic Word Sense Induction
AbstractThe task of Diachronic Word Sense Induction (DWSI) aims to identify the meaning of words from their context, taking the temporal dimension into account. In this paper we propose an …...
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features
AbstractKnowledge-grounded dialogue systems are intended to convey information that is based on evidence provided in a given source text. We discuss the challenges of training a generative neural dialogue model …...