Случайные разбиения
Related Papers
We Need to Talk about Standard Splits
AbstractIt is standard practice in speech & language technology to rank systems according to their performance on a test set held out for evaluation. However, few researchers apply statistical tests …...
We Need To Talk About Random Splits
Abstract(CITATION) argued for using random splits rather than standard splits in NLP experiments. We argue that random splits, like standard splits, lead to overly optimistic performance estimates. We can also …...
Benchmarking Long-tail Generalization with Likelihood Splits
AbstractIn order to reliably process natural language, NLP systems must generalize to the long tail of rare utterances. We propose a method to create challenging benchmarks that require generalizing to …...
It’s about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits
AbstractNew events emerge over time influencing the topics of rumors in social media. Current rumor detection benchmarks use random splits as training, development and test sets which typically results in …...