BERT Large
Related Papers
BERT for Coreference Resolution: Baselines and Analysis
AbstractWe apply BERT to coreference resolution, achieving a new state of the art on the GAP (+11.5 F1) and OntoNotes (+3.9 F1) benchmarks. A qualitative analysis of model predictions indicates …...
Learning to Ask Unanswerable Questions for Machine Reading Comprehension
AbstractMachine reading comprehension with unanswerable questions is a challenging task. In this work, we propose a data augmentation technique by automatically generating relevant unanswerable questions according to an answerable question …...
BERT-QE: Contextualized Query Expansion for Document Re-ranking
AbstractQuery expansion aims to mitigate the mismatch between the language used in a query and in a document. However, query expansion methods can suffer from introducing non-relevant information when expanding …...
KDELAB at SemEval-2020 Task 12: A System for Estimating Aggression of Tweets Using Two Layers of BERT Features
AbstractIn recent years, with the development of social network services and video distribution services, there has been a sharp increase in offensive posts. In this paper, we present our approach …...
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
AbstractNatural Language Processing (NLP) has recently achieved great success by using huge pre-trained models with hundreds of millions of parameters. However, these models suffer from heavy model sizes and high …...
Rad-SpatialNet: A Frame-based Resource for Fine-Grained Spatial Relations in Radiology Reports
AbstractThis paper proposes a representation framework for encoding spatial language in radiology based on frame semantics. The framework is adopted from the existing SpatialNet representation in the general domain with …...
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level
Larger language models have higher accu- racy on average, but are they better on ev- ery single instance (datapoint)? Some work suggests larger models have higher out-of- distribution robustness, while …...
Are Rotten Apples Edible? Challenging Commonsense Inference Ability with Exceptions
Previous studies have argued that pre-trained language models encode commonsense rela- tional knowledge (e.g. that apples are ed- ible). However, simultaneous work has re- vealed that such models are often …...
DLJUST at SemEval-2021 Task 7: Hahackathon: Linking Humor and Offense
AbstractHumor detection and rating poses interesting linguistic challenges to NLP; it is highly subjective depending on the perceptions of a joke and the context in which it is used. This …...
Posterior Differential Regularization with f-divergence for Improving Model Robustness
AbstractWe address the problem of enhancing model robustness through regularization. Specifically, we focus on methods that regularize the model posterior difference between clean and noisy inputs. Theoretically, we provide a …...
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
AbstractThe Lottery Ticket Hypothesis suggests that an over-parametrized network consists of ”lottery tickets”, and training a certain collection of them (i.e., a subnetwork) can match the performance of the full …...
COCO-DR: Combating Distribution Shift in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning
AbstractWe present a new zero-shot dense retrieval (ZeroDR) method, COCO-DR, to improve the generalization ability of dense retrieval by combating the distribution shifts between source training tasks and target scenarios. …...
Andronicus of Rhodes at SemEval-2023 Task 4: Transformer-Based Human Value Detection Using Four Different Neural Network Architectures
AbstractThis paper presents our participation to the “Human Value Detection shared task (Kiesel et al., 2023), as “Andronicus of Rhodes. We describe the approaches behind each entry in the official …...
Should You Mask 15% in Masked Language Modeling?
AbstractMasked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rate has been widely …...