Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?

8408

Knowledge Triples

Source Entity	Relationship	Target Entity
Bayesian Leaderboard Model Method	analyzes	Leaderboard Material
Bayesian Leaderboard Model Method	detects	Overfitting Detection Task
Bayesian Leaderboard Model Method	guides	Annotation Task
Bayesian Leaderboard Model Method	identifies	Informative Example Identification Task
Bayesian Leaderboard Model Method	predicts	Ranking Reliability Metric
Benchmark Task Task	evaluates	Natural Language Processing OtherEntity