MNLI

AbstractA machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases. …...

AbstractLarge-scale natural language inference (NLI) datasets such as SNLI or MNLI have been created by asking crowdworkers to read a premise and write three new hypotheses, one for each possible …...

AbstractModels that perform well on a training domain often fail to generalize to out-of-domain (OOD) examples. Data augmentation is a common method used to prevent overfitting and improve OOD generalization. …...

AbstractPre-trained Transformer-based neural architectures have consistently achieved state-of-the-art performance in the Natural Language Inference (NLI) task. Since NLI examples encompass a variety of linguistic, logical, and reasoning phenomena, it remains …...

AbstractDespite the subjective nature of many NLP tasks, most NLU evaluations have focused on using the majority label with presumably high agreement as the ground truth. Less attention has been …...

Larger language models have higher accu- racy on average, but are they better on ev- ery single instance (datapoint)? Some work suggests larger models have higher out-of- distribution robustness, while …...

AbstractNeural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks. Minority examples, i.e., examples that contradict the spurious correlations present in the …...

AbstractRecent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to understand human-like syntax, at least to some extent. We provide …...