Model Performance

metric ID: modelperformance

Related Papers

Evaluating and Combining Approaches to Selectional Preference Acquisition

Authors: nan

Publication: TIPSTER (2003)

DOI: nan

Previous work on the induction of se- lectional preferences has been mainly carried out for English and has concen- trated almost exclusively on verbs and their direct objects. In this …...

43456

Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training

Authors: nan

Publication: TIPSTER (2019)

DOI: nan

AbstractOne way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients. However, doing so damages the gradient and degrades the model’s performance. …...

9139

Learning Explicit and Implicit Structures for Targeted Sentiment Analysis

Authors: nan

Publication: TIPSTER (2019)

DOI: nan

AbstractTargeted sentiment analysis is the task of jointly predicting target entities and their associated sentiment information. Existing research efforts mostly regard this joint task as a sequence labeling problem, building …...

8971

Discontinuous Constituent Parsing as Sequence Labeling

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractThis paper reduces discontinuous parsing to sequence labeling. It first shows that existing reductions for constituent parsing as labeling do not support discontinuities. Second, it fills this gap and proposes …...

30667

DualTKB: A Dual Learning Bridge between Text and Knowledge Base

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractIn this work, we present a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases (KBs). We investigate the impact of weak …...

29757

Ruler: Data Programming by Demonstration for Document Labeling

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractData programming aims to reduce the cost of curating training data by encoding domain knowledge as labeling functions over source data. As such it not only requires domain expertise but …...

32298

What Can We Learn from Collective Human Opinions on Natural Language Inference Data?

Authors: nan

Publication: TIPSTER (2020)

DOI: nan

AbstractDespite the subjective nature of many NLP tasks, most NLU evaluations have focused on using the majority label with presumably high agreement as the ground truth. Less attention has been …...

29715

Augmenting Transformers with KNN-Based Composite Memory for Dialog

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractVarious machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable …...

26898

Does Typological Blinding Impede Cross-Lingual Sharing?

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractBridging the performance gap between high- and low-resource languages has been the focus of much previous work. Typological features from databases such as the World Atlas of Language Structures (WALS) …...

42855

Evaluating Neural Model Robustness for Machine Comprehension

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractWe evaluate neural model robustness to adversarial attacks using different types of linguistic unit perturbations – character and word, and propose a new method for strategic sentence-level perturbations. We experiment …...

42604

Exploring Distantly-Labeled Rationales in Neural Network Models

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractRecent studies strive to incorporate various human rationales into neural networks to improve model performance, but few pay attention to the quality of the rationales. Most existing methods distribute their …...

8240

Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractWhile state-of-the-art NLP models have been achieving the excellent performance of a wide range of tasks in recent years, important questions are being raised about their robustness and their underlying …...

8852

Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractText classification is a significant branch of natural language processing, and has many applications including document classification and sentiment analysis. Unsurprisingly, those who do text classification are concerned with the …...

21635

Handling Variance of Pretrained Language Models in Grading Evidence in the Medical Literature

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractIn this paper, we investigate the utility of modern pretrained language models for the evidence grading system in the medical literature based on the ALTA 2021 shared task. We benchmark …...

28410

Semantic Parsing of Disfluent Speech

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractSpeech disfluencies are prevalent in spontaneous speech. The rising popularity of voice assistants presents a growing need to handle naturally occurring disfluencies. Semantic parsing is a key component for understanding …...

42664

UAntwerp at SemEval-2021 Task 5: Spans are Spans, stacking a binary word level approach to toxic span detection

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractThis paper describes the system developed by the Antwerp Centre for Digital humanities and literary Criticism [UAntwerp] for toxic span detection. We used a stacked generalisation ensemble of five component …...

25488

UoB at SemEval-2021 Task 5: Extending Pre-Trained Language Models to Include Task and Domain-Specific Information for Toxic Span Prediction

Authors: nan

Publication: TIPSTER (2021)

DOI: nan

AbstractToxicity is pervasive in social media and poses a major threat to the health of online communities. The recent introduction of pre-trained language models, which have achieved state-of-the-art results in …...

25744

Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractIn this paper we describe our work towards building a generic framework for both multi-modal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) …...

25079

Contrast Sets for Stativity of English Verbs in Context

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractFor the task of classifying verbs in context as dynamic or stative, current models approach human performance, but only for particular data sets. To better understand the performance of such …...

4863

Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractNLP models that process multiple texts often struggle in recognizing corresponding and salient information that is often differently phrased, and consolidating the redundancies across texts. To facilitate research of such …...

21298

Towards Better Semantic Understanding of Mobile Interfaces

Authors: nan

Publication: TIPSTER (2022)

DOI: nan

AbstractImproving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users. To stimulate research in this direction, we release …...

4732

COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractA characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of …...

42837

Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractIn recent years, progress in NLU has been driven by benchmarks. These benchmarks are typically collected by crowdsourcing, where annotators write examples based on annotation instructions crafted by dataset creators. …...

42959

How Many Data Samples is an Additional Instruction Worth?

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractRecently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they …...

33374

Visual Spatial Reasoning

Authors: nan

Publication: TIPSTER (2023)

DOI: nan

AbstractSpatial relations are a basic part of human cognition. However, they are expressed in natural language in a variety of ways, and previous work has suggested that current vision-and-language models …...

26987