When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11473 - 11487 December 7-11, 2022 ©2022 Association for Computational Linguistics When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems Elias Stengel-Eskin1∗ Emmanouil Antonios Platanios2 Adam Pauls2 Sam Thomson2 Hao Fang2 Benjamin Van Durme2 Jason Eisner2 Yu Su2 1Johns Hopkins University 2Microsoft Semantic Machines Abstract In natural language understanding (NLU) pro- duction systems, users’ evolving needs neces- sitate the addition of new features over time, indexed by new symbols added to the mean- ing representation space. This requires addi- tional training data and results in ever-growing datasets. We present the first systematic in- vestigation of this incremental symbol learn- ing scenario. Our analysis reveals a trou- bling quirk in building broad-coverage NLU systems: as the training dataset grows, per- formance on the new symbol often decreases if we do not accordingly increase its training data. This suggests that it becomes more dif- ficult to learn new symbols with a larger train- ing dataset. We show that this trend holds for multiple mainstream models on two common NLU tasks: intent recognition and semantic parsing. Rejecting class imbalance as the sole culprit, we reveal that the trend is closely as- sociated with an effect we call source signal dilution, where strong lexical cues for the new symbol become diluted as the training dataset grows. Selectively dropping training exam- ples to prevent dilution often reverses the trend, showing the over-reliance of mainstream neu- ral NLU models on simple lexical cues.1 1 Introduction Broad-coverage natural language understanding (NLU) systems that simultaneously support a wide range of user requests are critical for developing general-purpose natural language interfaces. Such systems are already being deployed and reach mil- lions of users worldwide: As of 2021, Amazon Alexa contains more than 80,000 skills (Vailshery, 2021), and Microsoft has deployed a new conver- sational interface for Outlook that uses over 300 composable functions to represent fine-grained se- *Work done as an intern at Microsoft Semantic Machines. 1Code, models, and data are available at https://aka. ms/nlu-incremental-symbol-learning. Figure 1: Overall and per-symbol test accuracy for learning a new symbol from a fixed number of training examples (30 for intent, 100 for SMCalFlow) when the overall training dataset grows (x-axis). Each line rep- resents fixing a different symbol. As training data size increases, overall accuracy (averaged across symbols) increases but new symbol accuracy often decreases. mantics in task-oriented user-agent dialogues (Se- mantic Machines et al., 2020; Burrage, 2021). These broad-coverage NLU systems do not ac- quire their full capability on day one: new features (e.g., intents or functions), and the new training ex- amples for learning them, are added incrementally. However, there has been little research on the data and learning dynamics during such incremental de- velopment. The wide deployment and high cost of such systems make this research critical, and we aim to call attention to this important problem. We consider two prototypical NLU tasks: in- tent recognition and semantic parsing. NLU is …

Content truncated to 500 words