Improving Generalization in Coreference Resolution via Adversarial Training

Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM), pages 192–197 Minneapolis, June 6–7, 2019. c⃝2019 Association for Computational Linguistics 192 Improving Generalization in Coreference Resolution via Adversarial Training Sanjay Subramanian University of Pennsylvania sanjayssub34@gmail.com Dan Roth University of Pennsylvania danroth@seas.upenn.edu Abstract In order for coreference resolution systems to be useful in practice, they must be able to generalize to new text. In this work, we demonstrate that the performance of the state-of-the-art system decreases when the names of PER and GPE named entities in the CoNLL dataset are changed to names that do not occur in the training set. We use the technique of adversarial gradient-based training to retrain the state-of-the-art system and demonstrate that the retrained system achieves higher performance on the CoNLL dataset (both with and without the change of named entities) and the GAP dataset. 1 Introduction Through the use of neural networks, performance on the task of coreference resolution has increased significantly over the last few years. Still, neu- ral systems trained on the standard coreference dataset have issues with generalization, as shown by (Moosavi and Strube, 2018). One way to improve the understanding of how a system overfits a dataset is to study the change in the system’s performance when the dataset is modified slightly in a focused and relevant man- ner. We take this approach by modifying the test set so that each PER and GPE (person and geopo- litical entity) named entity is different from those seen in training. In other words, we ensure that there is no leakage of PER and GPE named entities from the training set into the test set. We demon- strate that the performance of the (Lee et al., 2018) system, which is the current state-of-the-art, de- creases when the named entities are replaced. An example of a replacement that causes the system to make an error is given in Table 1. Motivated by these issues of generalization, this paper aims to improve the training process of neu- Original: But Dirk Van Dongen , president of the National Association of Wholesaler - Distributors , said that last month ’s rise “ is n’t as bad an omen ” as the 0.9 % figure suggests . “ If you examine the data carefully , the increase is concentrated in energy and motor vehicle prices , rather than being a broad - based advance in the prices of consumer and industrial goods , ” he explained . Replacement: Replace Dick Van Dongen with Vendemiaire Van Korewdit. Table 1: An excerpt from the CoNLL test set. The coreference between the two highlighted mentions is correctly predicted by the (Lee et al., 2018) system, but after the specified replacement, the system incorrectly resolves “he” to a different name occurring outside this excerpt. ral coreference systems. Various regularization techniques have been proposed for improving the generalization capability of neural networks, in- cluding dropout (Srivastava et al., 2014) and ad- versarial training (Goodfellow et al., 2015; Miy- ato et al., …

Content truncated to 500 words