Model Performance
Related Papers
Evaluating and Combining Approaches to Selectional Preference Acquisition
Previous work on the induction of se- lectional preferences has been mainly carried out for English and has concen- trated almost exclusively on verbs and their direct objects. In this …...
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training
AbstractOne way to reduce network traffic in multi-node data-parallel stochastic gradient descent is to only exchange the largest gradients. However, doing so damages the gradient and degrades the model’s performance. …...
Learning Explicit and Implicit Structures for Targeted Sentiment Analysis
AbstractTargeted sentiment analysis is the task of jointly predicting target entities and their associated sentiment information. Existing research efforts mostly regard this joint task as a sequence labeling problem, building …...
Discontinuous Constituent Parsing as Sequence Labeling
AbstractThis paper reduces discontinuous parsing to sequence labeling. It first shows that existing reductions for constituent parsing as labeling do not support discontinuities. Second, it fills this gap and proposes …...
DualTKB: A Dual Learning Bridge between Text and Knowledge Base
AbstractIn this work, we present a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases (KBs). We investigate the impact of weak …...
Ruler: Data Programming by Demonstration for Document Labeling
AbstractData programming aims to reduce the cost of curating training data by encoding domain knowledge as labeling functions over source data. As such it not only requires domain expertise but …...
What Can We Learn from Collective Human Opinions on Natural Language Inference Data?
AbstractDespite the subjective nature of many NLP tasks, most NLU evaluations have focused on using the majority label with presumably high agreement as the ground truth. Less attention has been …...
Augmenting Transformers with KNN-Based Composite Memory for Dialog
AbstractVarious machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable …...
Does Typological Blinding Impede Cross-Lingual Sharing?
AbstractBridging the performance gap between high- and low-resource languages has been the focus of much previous work. Typological features from databases such as the World Atlas of Language Structures (WALS) …...
Evaluating Neural Model Robustness for Machine Comprehension
AbstractWe evaluate neural model robustness to adversarial attacks using different types of linguistic unit perturbations – character and word, and propose a new method for strategic sentence-level perturbations. We experiment …...
Exploring Distantly-Labeled Rationales in Neural Network Models
AbstractRecent studies strive to incorporate various human rationales into neural networks to improve model performance, but few pay attention to the quality of the rationales. Most existing methods distribute their …...
Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis
AbstractWhile state-of-the-art NLP models have been achieving the excellent performance of a wide range of tasks in recent years, important questions are being raised about their robustness and their underlying …...
Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification
AbstractText classification is a significant branch of natural language processing, and has many applications including document classification and sentiment analysis. Unsurprisingly, those who do text classification are concerned with the …...
Handling Variance of Pretrained Language Models in Grading Evidence in the Medical Literature
AbstractIn this paper, we investigate the utility of modern pretrained language models for the evidence grading system in the medical literature based on the ALTA 2021 shared task. We benchmark …...
Semantic Parsing of Disfluent Speech
AbstractSpeech disfluencies are prevalent in spontaneous speech. The rising popularity of voice assistants presents a growing need to handle naturally occurring disfluencies. Semantic parsing is a key component for understanding …...
UAntwerp at SemEval-2021 Task 5: Spans are Spans, stacking a binary word level approach to toxic span detection
AbstractThis paper describes the system developed by the Antwerp Centre for Digital humanities and literary Criticism [UAntwerp] for toxic span detection. We used a stacked generalisation ensemble of five component …...
UoB at SemEval-2021 Task 5: Extending Pre-Trained Language Models to Include Task and Domain-Specific Information for Toxic Span Prediction
AbstractToxicity is pervasive in social media and poses a major threat to the health of online communities. The recent introduction of pre-trained language models, which have achieved state-of-the-art results in …...
Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework
AbstractIn this paper we describe our work towards building a generic framework for both multi-modal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) …...
Contrast Sets for Stativity of English Verbs in Context
AbstractFor the task of classifying verbs in context as dynamic or stative, current models approach human performance, but only for particular data sets. To better understand the performance of such …...
Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations
AbstractNLP models that process multiple texts often struggle in recognizing corresponding and salient information that is often differently phrased, and consolidating the redundancies across texts. To facilitate research of such …...
Towards Better Semantic Understanding of Mobile Interfaces
AbstractImproving the accessibility and automation capabilities of mobile devices can have a significant positive impact on the daily lives of countless users. To stimulate research in this direction, we release …...
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models
AbstractA characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of …...
Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions
AbstractIn recent years, progress in NLU has been driven by benchmarks. These benchmarks are typically collected by crowdsourcing, where annotators write examples based on annotation instructions crafted by dataset creators. …...
How Many Data Samples is an Additional Instruction Worth?
AbstractRecently introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. Instruction-tuned models have significantly outperformed multitask learning models (without instruction); however they …...
Visual Spatial Reasoning
AbstractSpatial relations are a basic part of human cognition. However, they are expressed in natural language in a variety of ways, and previous work has suggested that current vision-and-language models …...