What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue

Proceedings of NAACL-HLT 2019, pages 3772–3783 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3772 What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue Laura Aina∗ Carina Silberer∗ Ionut-Teodor Sorodoc∗ Matthijs Westera∗ Gemma Boleda Universitat Pompeu Fabra Barcelona, Spain {firstname.lastname}@upf.edu Abstract Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why. In this paper we analyze the behavior of two re- cently proposed entity-centric models in a ref- erential task, Entity Linking in Multi-party Di- alogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entity- centric naturally fosters good architectural de- cisions. However, we also show that these models do not really build entity representa- tions and that they make poor use of linguis- tic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when de- ployed. 1 Introduction Modeling reference to entities is arguably crucial for language understanding, as humans use language to talk about things in the world. hypothesis in recent work on referential tasksA such as co-reference resolution and entity link- ing (Haghighi and Klein, 2010; Clark and Manning, 2016; Henaff et al., 2017; Aina et al., 2018; Clark et al., 2018) is that encouraging models to learn and use entity representations will help them better carry out referential tasks. To illustrate, creating an entity representation with the relevant information upon reading a woman should make it easier to ∗denotes equal contribution. JOEY TRIBBIANI (183): ”...see Ross, because I think you love her .” 335 183 335 306 Figure 1: Character identification: example. resolve a pronoun mention like she.1 In the men- tioned work, several models have been proposed that incorporate an explicit bias towards entity representations. Such entity-centric models have shown empirical success, but we still know little about what it is that they effectively learn to model. In this analysis paper, we adapt two previous entity-centric models (Henaff et al., 2017; Aina et al., 2018) for a recently proposed referential task and show that, despite their strengths, they are still very far from modeling entities.2 The task is character identification on multi- party dialogue as posed in SemEval 2018 Task 4 (Choi and Chen, 2018).3 Models are given dia- logues from the TV show Friends and asked to link entity mentions (nominal expressions like I, she or the woman) to the characters to which they refer in each case. Figure 1 shows an example, where the mentions Ross and you are linked to entity 335, mention I to entity 183, …

Content truncated to 500 words