PeTra: A Sparsely Supervised Memory Model for People Tracking

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5415–5428 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5415 PeTra: A Sparsely Supervised Memory Model for People Tracking Shubham Toshniwal1, Allyson Ettinger2, Kevin Gimpel1, Karen Livescu1 1Toyota Technological Institute at Chicago 2Department of Linguistics, University of Chicago {shtoshni, kgimpel, klivescu}@ttic.edu, aettinger@uchicago.edu Abstract We propose PeTra, a memory-augmented neu- ral network designed to track entities in its memory slots. PeTra is trained using sparse annotation from the GAP pronoun resolution dataset and outperforms a prior memory model on the task while using a simpler architec- ture. We empirically compare key modeling choices, finding that we can simplify several aspects of the design of the memory module while retaining strong performance. To mea- sure the people tracking capability of memory models, we (a) propose a new diagnostic evalu- ation based on counting the number of unique entities in text, and (b) conduct a small scale human evaluation to compare evidence of peo- ple tracking in the memory logs of PeTra rela- tive to a previous approach. PeTra is highly ef- fective in both evaluations, demonstrating its ability to track people in its memory despite being trained with limited annotation. 1 Introduction Understanding text narratives requires maintaining and resolving entity references over arbitrary-length spans. Current approaches for coreference resolu- tion (Clark and Manning, 2016b; Lee et al., 2017, 2018; Wu et al., 2019) scale quadratically (without heuristics) with length of text, and hence are im- practical for long narratives. These models are also cognitively implausible, lacking the incrementality of human language processing (Tanenhaus et al., 1995; Keller, 2010). Memory models with finite memory and online/quasi-online entity resolution have linear runtime complexity, offering more scal- ability, cognitive plausibility, and interpretability. Memory models can be viewed as general prob- lem solvers with external memory mimicking a Turing tape (Graves et al., 2014, 2016). Some of the earliest applications of memory networks in language understanding were for question an- swering, where the external memory simply stored all of the word/sentence embeddings for a docu- ment (Sukhbaatar et al., 2015; Kumar et al., 2016). To endow more structure and interpretability to memory, key-value memory networks were intro- duced by Miller et al. (2016). The key-value archi- tecture has since been used for narrative understand- ing and other tasks where the memory is intended to learn to track entities while being guided by vary- ing degrees of supervision (Henaff et al., 2017; Liu et al., 2018a,b, 2019a). We propose a new memory model, PeTra, for entity tracking and coreference resolution, inspired by the recent Referential Reader model (Liu et al., 2019a) but substantially simpler. Experiments on the GAP (Webster et al., 2018) pronoun resolu- tion task show that PeTra outperforms the Refer- ential Reader with fewer parameters and simpler architecture. Importantly, while Referential Reader performance degrades with larger memory, PeTra improves with increase in memory capacity (before saturation), which should enable tracking of a larger number of entities. We conduct experiments to …

Content truncated to 500 words