Бағалау парадигмасы
Related Papers
A Rough Set Formalization of Quantitative Evaluation with Ambiguity
AbstractIn this paper, we present the founding elements of a formal model of the evaluation paradigm in natural language processing. We propose an abstract model of objective quantitative evaluation based …...
Corpus and Evaluation of Handwriting Recognition of Historical Genealogical Records
AbstractOver the last few decades, significant strides have been made in handwriting recognition (HR), which is the automatic transcription of handwritten documents. HR often focuses on modern handwritten material, but …...
Interpretable Question Answering on Knowledge Bases and Text
AbstractInterpretability of machine learning (ML) models becomes more relevant with their increasing adoption. In this work, we address the interpretability of ML based question answering (QA) models on a combination …...
A Prioritization Model for Suicidality Risk Assessment
AbstractWe reframe suicide risk assessment from social media as a ranking problem whose goal is maximizing detection of severely at-risk individuals given the time available. Building on measures developed for …...
Evaluation Paradigms in Question Answering
AbstractQuestion answering (QA) primarily descends from two branches of research: (1) Alan Turing’s investigation of machine intelligence at Manchester University and (2) Cyril Cleverdon’s comparison of library card catalog indices …...
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models
AbstractCoherent discourse is distinguished from a mere collection of utterances by the satisfaction of a diverse set of constraints, for example choice of expression, logical relation between denoted events, and …...
Evaluating the Diversity, Equity, and Inclusion of NLP Technology: A Case Study for Indian Languages
AbstractIn order for NLP technology to be widely applicable, fair, and useful, it needs to serve a diverse set of speakers across the world’s languages, be equitable, i.e., not unduly …...
Factual or Contextual? Disentangling Error Types in Entity Description Generation
AbstractIn the task of entity description generation, given a context and a specified entity, a model must describe that entity correctly and in a contextually-relevant way. In this task, as …...