Оценка Производительности

metric ID: performanceevaluation

Related Papers

A Formal Basis for Performance Evaluation of Natural Language Understanding Systems

Authors: nan

Publication: TIPSTER (1984)

DOI: nan

level. The definition of concrete measures is then obtained by assigning actual values to the functional parameters of the abstract definition; some particular cases are shown and discussed in detail. …...

36218

Acquiring Knowledge From Encyclopedic Texts

Authors: nan

Publication: TIPSTER (1994)

DOI: nan

A computational model for the acquisition of knowledge from encyclopedic texts is de- scribed. The model has been implemented in a program, called SNOWY, that reads unedited texts from The …...

40942

A Simple Hybrid Aligner for Generating Lexical Correspondences in Parallel Texts

Authors: nan

Publication: TIPSTER (1998)

DOI: nan

We present an algorithm for bilingual word alignment that extends previous work by treating multi-word candidates on a par with single words, and combining some simple assumptions about the translation …...

35348

Towards a Noise-Tolerant, Representation-Independent Mechanism for Argument Interpretation

Authors: nan

Publication: TIPSTER (2002)

DOI: nan

We describe a mechanism for the interpretation of arguments, which can cope with noisy conditions in terms of wording, beliefs and argument struc- ture. This is achieved through the application …...

6764

Structuring Knowledge for Reference Generation: A Clustering Algorithm

Authors: nan

Publication: TIPSTER (2006)

DOI: nan

This paper discusses two problems that arise in the Generation of Referring Expressions: (a) numeric-valued attributes, such as size or location; (b) perspective-taking in reference. Both problems, it is argued, …...

43748

Bilingually Motivated Domain-Adapted Word Segmentation for Statistical Machine Translation

Authors: nan

Publication: TIPSTER (2009)

DOI: nan

We introduce a word segmentation ap- proach to languages where word bound- aries are not orthographically marked, with application to Phrase-Based Statis- tical Machine Translation (PB-SMT). In- stead of using …...

43321

On the Use of Comparable Corpora to Improve SMT performance

Authors: nan

Publication: TIPSTER (2009)

DOI: nan

We present a simple and effective method for extracting parallel sentences from comparable corpora. We employ a sta- tistical machine translation (SMT) system built from small amounts of parallel texts …...

43501

Automatic Detection of Syllable Boundaries in Spontaneous Speech

Authors: nan

Publication: TIPSTER (2010)

DOI: nan

AbstractThis paper presents the outline and performance of an automatic syllable boundary detection system. The syllabification of phonemes is performed with a rule-based system, implemented in a Java program. Phonemes …...

8368

EVALIEX — A Proposal for an Extended Evaluation Methodology for Information Extraction Systems

Authors: nan

Publication: TIPSTER (2012)

DOI: nan

AbstractAssessing the correctness of extracted data requires performance evaluation, which is accomplished by calculating quality metrics. The evaluation process must cope with the challenges posed by information extraction and natural …...

13942

Sketch Algorithms for Estimating Point Queries in NLP

Authors: nan

Publication: TIPSTER (2012)

DOI: nan

Many NLP tasks rely on accurate statis- tics from large corpora. Tracking com- plete statistics is memory intensive, so recent work has proposed using compact approx- imate “sketches” of frequency …...

44087

A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic

Authors: nan

Publication: TIPSTER (2014)

DOI: nan

AbstractThis paper presents a multi-dialect, multi-genre, human annotated corpus of dialectal Arabic. We collected utterances in five Arabic dialects: Levantine, Gulf, Egyptian, Iraqi and Maghrebi. We scraped newspaper websites for …...

16122

Benchmarking Twitter Sentiment Analysis Tools

Authors: nan

Publication: TIPSTER (2014)

DOI: nan

AbstractTwitter has become one of the quintessential social media platforms for user-generated content. Researchers and industry practitioners are increasingly interested in Twitter sentiments. Consequently, an array of commercial and freely …...

16480

Combining String and Context Similarity for Bilingual Term Alignment from Comparable Corpora

Authors: nan

Publication: TIPSTER (2014)

DOI: nan

Automatically compiling bilingual dictio- naries of technical terms from comparable corpora is a challenging problem, yet with many potential applications. In this paper, we exploit two independent observations about term …...

34374

Rapid Development of a Corpus with Discourse Annotations using Two-stage Crowdsourcing

Authors: nan

Publication: TIPSTER (2014)

DOI: nan

We present a novel approach for rapidly developing a corpus with discourse annotations using crowdsourcing. Although discourse annotations typically require much time and cost owing to their complex nature, we …...

6019

Lisbon: Evaluating TurboSemanticParser on Multiple Languages and Out-of-Domain Data

Authors: nan

Publication: TIPSTER (2015)

DOI: nan

As part of the SemEval-2015 shared task on Broad-Coverage Semantic Dependency Pars- ing, we evaluate the performace of our last year’s system (TurboSemanticParser) on mul- tiple languages and out-of-domain data. …...

20822

BUTknot at SemEval-2016 Task 5: Supervised Machine Learning with Term Substitution Approach in Aspect Category Detection

Authors: nan

Publication: TIPSTER (2016)

DOI: nan

This paper describes an approach used to solve Aspect Category Detection (Subtask 1, Slot 1) of SemEval 2016 Task 5. The core of the pre- sented system is based on …...

22538

Evaluating anaphora and coreference resolution to improve automatic keyphrase extraction

Authors: nan

Publication: TIPSTER (2016)

DOI: nan

AbstractIn this paper we analyze the effectiveness of using linguistic knowledge from coreference and anaphora resolution for improving the performance for supervised keyphrase extraction. In order to verify the impact …...

4537

A Statistical, Grammar-Based Approach to Microplanning

Authors: nan

Publication: TIPSTER (2017)

DOI: nan

AbstractAlthough there has been much work in recent years on data-driven natural language generation, little attention has been paid to the fine-grained interactions that arise during microplanning between aggregation, surface …...

41447

Delta vs. N-Gram Tracing: Evaluating the Robustness of Authorship Attribution Methods

Authors: nan

Publication: TIPSTER (2018)

DOI: nan

Delta measures are a well-established and popular family of authorship attribution methods, especially for literary texts. N-gram tracing is a novel method for authorship attribution designed for very short texts, …...

17498

Detecting Frames in News Headlines and Its Application to Analyzing News Framing Trends Surrounding U.S. Gun Violence

Authors: nan

Publication: TIPSTER (2019)

DOI: nan

AbstractDifferent news articles about the same topic often offer a variety of perspectives: an article written about gun violence might emphasize gun control, while another might promote 2nd Amendment rights, …...

41701

Image Position Prediction in Multimodal Documents

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractConventional multimodal tasks, such as caption generation and visual question answering, have allowed machines to understand an image by describing or being asked about it in natural language, often via …...

14456

Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractMany datasets have been shown to contain incidental correlations created by idiosyncrasies in the data collection process. For example, sentence entailment datasets can have spurious word-class correlations if nearly all …...

32208

Learning to Update Natural Language Comments Based on Code Changes

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractWe formulate the novel task of automatically updating an existing natural language comment based on changes in the body of code it accompanies. We propose an approach that learns to …...

38112

Social Web Observatory: A Platform and Method for Gathering Knowledge on Entities from Different Textual Sources

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractWithin this work we describe a framework for the collection and summarization of information from the Web in an entity-driven manner. The framework consists of a set of appropriate workflows …...

15243

Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractIdeally Open-Domain Question Answering models should exhibit a number of competencies, ranging from simply memorizing questions seen at training time, to answering novel question formulations with answers seen during training, …...

42754

Automatic Correction of Syntactic Dependency Annotation Differences

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractAnnotation inconsistencies between data sets can cause problems for low-resource NLP, where noisy or inconsistent data cannot be easily replaced. We propose a method for automatically detecting annotation mismatches between …...

14239

Discovering Language-neutral Sub-networks in Multilingual Language Models

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractMultilingual pre-trained language models transfer remarkably well on cross-lingual downstream tasks. However, the extent to which they learn language-neutral representations (i.e., shared representations that encode similar phenomena across languages), and …...

30451

HECTOR: A Hybrid TExt SimplifiCation TOol for Raw Texts in French

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractReducing the complexity of texts by applying an Automatic Text Simplification (ATS) system has been sparking interest inthe area of Natural Language Processing (NLP) for several years and a number …...

14861

Tweet Review Mining focusing on Celebrities by Machine Reading Comprehension based on BERT

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

In this paper, we propose a method of min- ing tweets that represent reviews on celebrities by the machine reading comprehension model based on BERT. The purpose of this paper …...

28346

BERT Shows Garden Path Effects

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractGarden path sentences (i.e. “the horse raced past the barn fell”) are sentences that readers initially incorrectly parse, requiring partial or total re-analysis of the sentence structure. Given human difficulty …...

42780

WebCPM: Interactive Web Search for Chinese Long-form Question Answering

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractLong-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. The de facto paradigm of LFQA necessitates two procedures: information retrieval, which searches for relevant supporting …...

38298