Оценка Производительности
Related Papers
A Formal Basis for Performance Evaluation of Natural Language Understanding Systems
level. The definition of concrete measures is then obtained by assigning actual values to the functional parameters of the abstract definition; some particular cases are shown and discussed in detail. …...
Acquiring Knowledge From Encyclopedic Texts
A computational model for the acquisition of knowledge from encyclopedic texts is de- scribed. The model has been implemented in a program, called SNOWY, that reads unedited texts from The …...
A Simple Hybrid Aligner for Generating Lexical Correspondences in Parallel Texts
We present an algorithm for bilingual word alignment that extends previous work by treating multi-word candidates on a par with single words, and combining some simple assumptions about the translation …...
Towards a Noise-Tolerant, Representation-Independent Mechanism for Argument Interpretation
We describe a mechanism for the interpretation of arguments, which can cope with noisy conditions in terms of wording, beliefs and argument struc- ture. This is achieved through the application …...
Structuring Knowledge for Reference Generation: A Clustering Algorithm
This paper discusses two problems that arise in the Generation of Referring Expressions: (a) numeric-valued attributes, such as size or location; (b) perspective-taking in reference. Both problems, it is argued, …...
Bilingually Motivated Domain-Adapted Word Segmentation for Statistical Machine Translation
We introduce a word segmentation ap- proach to languages where word bound- aries are not orthographically marked, with application to Phrase-Based Statis- tical Machine Translation (PB-SMT). In- stead of using …...
On the Use of Comparable Corpora to Improve SMT performance
We present a simple and effective method for extracting parallel sentences from comparable corpora. We employ a sta- tistical machine translation (SMT) system built from small amounts of parallel texts …...
Automatic Detection of Syllable Boundaries in Spontaneous Speech
AbstractThis paper presents the outline and performance of an automatic syllable boundary detection system. The syllabification of phonemes is performed with a rule-based system, implemented in a Java program. Phonemes …...
EVALIEX — A Proposal for an Extended Evaluation Methodology for Information Extraction Systems
AbstractAssessing the correctness of extracted data requires performance evaluation, which is accomplished by calculating quality metrics. The evaluation process must cope with the challenges posed by information extraction and natural …...
Sketch Algorithms for Estimating Point Queries in NLP
Many NLP tasks rely on accurate statis- tics from large corpora. Tracking com- plete statistics is memory intensive, so recent work has proposed using compact approx- imate “sketches” of frequency …...
A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic
AbstractThis paper presents a multi-dialect, multi-genre, human annotated corpus of dialectal Arabic. We collected utterances in five Arabic dialects: Levantine, Gulf, Egyptian, Iraqi and Maghrebi. We scraped newspaper websites for …...
Benchmarking Twitter Sentiment Analysis Tools
AbstractTwitter has become one of the quintessential social media platforms for user-generated content. Researchers and industry practitioners are increasingly interested in Twitter sentiments. Consequently, an array of commercial and freely …...
Combining String and Context Similarity for Bilingual Term Alignment from Comparable Corpora
Automatically compiling bilingual dictio- naries of technical terms from comparable corpora is a challenging problem, yet with many potential applications. In this paper, we exploit two independent observations about term …...
Rapid Development of a Corpus with Discourse Annotations using Two-stage Crowdsourcing
We present a novel approach for rapidly developing a corpus with discourse annotations using crowdsourcing. Although discourse annotations typically require much time and cost owing to their complex nature, we …...
Lisbon: Evaluating TurboSemanticParser on Multiple Languages and Out-of-Domain Data
As part of the SemEval-2015 shared task on Broad-Coverage Semantic Dependency Pars- ing, we evaluate the performace of our last year’s system (TurboSemanticParser) on mul- tiple languages and out-of-domain data. …...
BUTknot at SemEval-2016 Task 5: Supervised Machine Learning with Term Substitution Approach in Aspect Category Detection
This paper describes an approach used to solve Aspect Category Detection (Subtask 1, Slot 1) of SemEval 2016 Task 5. The core of the pre- sented system is based on …...
Evaluating anaphora and coreference resolution to improve automatic keyphrase extraction
AbstractIn this paper we analyze the effectiveness of using linguistic knowledge from coreference and anaphora resolution for improving the performance for supervised keyphrase extraction. In order to verify the impact …...
A Statistical, Grammar-Based Approach to Microplanning
AbstractAlthough there has been much work in recent years on data-driven natural language generation, little attention has been paid to the fine-grained interactions that arise during microplanning between aggregation, surface …...
Delta vs. N-Gram Tracing: Evaluating the Robustness of Authorship Attribution Methods
Delta measures are a well-established and popular family of authorship attribution methods, especially for literary texts. N-gram tracing is a novel method for authorship attribution designed for very short texts, …...
Detecting Frames in News Headlines and Its Application to Analyzing News Framing Trends Surrounding U.S. Gun Violence
AbstractDifferent news articles about the same topic often offer a variety of perspectives: an article written about gun violence might emphasize gun control, while another might promote 2nd Amendment rights, …...
Image Position Prediction in Multimodal Documents
AbstractConventional multimodal tasks, such as caption generation and visual question answering, have allowed machines to understand an image by describing or being asked about it in natural language, often via …...
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
AbstractMany datasets have been shown to contain incidental correlations created by idiosyncrasies in the data collection process. For example, sentence entailment datasets can have spurious word-class correlations if nearly all …...
Learning to Update Natural Language Comments Based on Code Changes
AbstractWe formulate the novel task of automatically updating an existing natural language comment based on changes in the body of code it accompanies. We propose an approach that learns to …...
Social Web Observatory: A Platform and Method for Gathering Knowledge on Entities from Different Textual Sources
AbstractWithin this work we describe a framework for the collection and summarization of information from the Web in an entity-driven manner. The framework consists of a set of appropriate workflows …...
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets
AbstractIdeally Open-Domain Question Answering models should exhibit a number of competencies, ranging from simply memorizing questions seen at training time, to answering novel question formulations with answers seen during training, …...
Automatic Correction of Syntactic Dependency Annotation Differences
AbstractAnnotation inconsistencies between data sets can cause problems for low-resource NLP, where noisy or inconsistent data cannot be easily replaced. We propose a method for automatically detecting annotation mismatches between …...
Discovering Language-neutral Sub-networks in Multilingual Language Models
AbstractMultilingual pre-trained language models transfer remarkably well on cross-lingual downstream tasks. However, the extent to which they learn language-neutral representations (i.e., shared representations that encode similar phenomena across languages), and …...
HECTOR: A Hybrid TExt SimplifiCation TOol for Raw Texts in French
AbstractReducing the complexity of texts by applying an Automatic Text Simplification (ATS) system has been sparking interest inthe area of Natural Language Processing (NLP) for several years and a number …...
Tweet Review Mining focusing on Celebrities by Machine Reading Comprehension based on BERT
In this paper, we propose a method of min- ing tweets that represent reviews on celebrities by the machine reading comprehension model based on BERT. The purpose of this paper …...
BERT Shows Garden Path Effects
AbstractGarden path sentences (i.e. “the horse raced past the barn fell”) are sentences that readers initially incorrectly parse, requiring partial or total re-analysis of the sentence structure. Given human difficulty …...
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
AbstractLong-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. The de facto paradigm of LFQA necessitates two procedures: information retrieval, which searches for relevant supporting …...