Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1325–1335 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1325 Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation Aparna Elangovan1, Jiayuan He1,2, and Karin Verspoor1 1The University of Melbourne, Australia 2RMIT University, Australia aparnae@student.unimelb.edu.au {estrid.he, karin.verspoor}@unimelb.edu.au Abstract Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). However, the presence of overlap between the train and test datasets can lead to inflated results, inadvertently evaluat- ing the model’s ability to memorize and inter- preting it as the ability to generalize. In ad- dition, such data sets may not provide an ef- fective indicator of the performance of these methods in real world scenarios. We identify leakage of training data into test data on sev- eral publicly available datasets used to evalu- ate NLP tasks, including named entity recog- nition and relation extraction, and study them to assess the impact of that leakage on the model’s ability to memorize versus generalize. 1 Introduction Shared tasks that provide publicly available datasets in order to evaluate and compare the per- formance of different methods on the same task and data are common in NLP. Held-out test sets are typically provided, enabling assessment of the generalizability of different methods to previously unseen data. These datasets have played a key role in driving progress in NLP, by defining focus tasks and by making annotated data available to the broader community, in particular in specialized domains such as biomedicine where data can be dif- ficult to obtain, and quality data annotations require the detailed work of domain experts. Examples of tasks where benchmark data sets exist include open domain question answering (QA) (Berant et al., 2013; Joshi et al., 2017) and biomedical named entity recognition (Smith et al., 2008) . In the context of machine learning models, ef- fectiveness is typically determined by the model’s ability to both memorize and generalize (Chatterjee, 2018). A model that has huge capacity to memo- rize will often work well in real world applications, particularly where large amounts of training data are available (Daelemans et al., 2005). The ability of a model to generalize relates to how well the model performs when it is applied on data that may be different from the data used to train the model, in terms of e.g. the distribution of vocabulary or other relevant vocabulary. The ability to memorize, taken to the extreme, can be considered equivalent to an exact match lookup table (Chatterjee, 2018) and the ability to generalize captures how well it can deal with degrees of variations from the lookup table. An effective combination of memorization and generalization can be achieved where a model selectively memorizes only those aspects or fea- tures that matter in solving a target objective given an input, allowing it to generalize better and to be less susceptible to noise. When there is considerable overlap …

Content truncated to 500 words