NE-Table: A Neural key-value table for Named Entities

Proceedings of Recent Advances in Natural Language Processing, pages 980–993, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_114 980 NE-Table: A Neural Key-Value Table for Named Entities Janarthanan Rajendran∗ University of Michigan rjana@umich.edu Jatin Ganhotra∗ IBM Research jatinganhotra@us.ibm.com Xiaoxiao Guo IBM Research xiaoxiao.guo@ibm.com Mo Yu IBM Research yum@us.ibm.com Satinder Singh University of Michigan baveja@umich.edu Lazaros Polymenakos† Amazon Alexa-AI Research polyml@amazon.com Abstract Many Natural Language Processing (NLP) tasks depend on using Named Entities (NEs) that are contained in texts and in external knowledge sources. While this is easy for humans, the present neural methods that rely on learned word embed- dings may not perform well for these NLP tasks, especially in the presence of Out- Of-Vocabulary (OOV) or rare NEs. In this paper, we propose a solution for this prob- lem, and present empirical evaluations on: a) a structured Question-Answering task, b) three related Goal-Oriented dialog tasks, and c) a Reading-Comprehension task1, which show that the proposed method can be effective in dealing with both in-vocabulary and OOV NEs. 1 Introduction We come across Named Entities (NEs) in many Natural Language Processing (NLP) tasks. In tasks such as Question-Answering (QA) and goal- oriented dialog, NEs play a crucial role in task completion. Examples include QA systems for retrieving information about courses offered at a university, and dialog systems that perform restau- rant reservation, flight ticket booking, and so on. In many cases, these tasks also involve inter- action with external knowledge sources such as DataBases (DB) which could have a large num- ber of NEs. In these tasks NEs include people’s names, restaurant names, locations etc. ∗ Equal Contribution † This work was done when the author was at IBM Re- search, NY. 1We create extended versions of dialog bAbI tasks 1,2 and 4 and OOV versions of the CBT test set - https: //github.com/IBM/ne-table-datasets/ There has been a lot of interest in building neu- ral methods for NLP tasks. Past work has devel- oped multiple methods for addressing the unique challenges to neural methods posed by NEs. One straightforward method is to add each and every NE (including those in the DB) to the vocabu- lary. This method has been evaluated for only syn- thetic or small tasks (Neelakantan et al., 2015). For real world tasks, especially those with large DBs, this causes an explosion in the vocabulary size and hence the number of parameters to learn. There is also the problem of not being able to learn good neural embeddings for individual NEs, as individual NEs (e.g., a particular phone num- ber) generally occur only a few times in a dataset. Another previously proposed method is to encode all the NEs with random embeddings and keep them fixed throughout (Yin et al., 2015), but here we lose the meaning associated with the neural embeddings and risk interference and correlation with others in unexpected ways. A third method is to first recognize the NE-type with either NE taggers (Finkel et al., 2005) or en- tity linkers (Cucerzan, 2007; Guo et al., 2013), …

Content truncated to 500 words