Identifying Named Entities as they are Typed

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 976–988 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 976 Identifying Named Entities as they are Typed Ravneet Singh Arora Chen-Tse Tsai Daniel Preot¸iuc-Pietro Bloomberg {rarora62,ctsai54,dpreotiucpie}@bloomberg.net Abstract Identifying named entities in written text is an essential component of the text processing pipeline used in applications such as text ed- itors to gain a better understanding of the se- mantics of the text. However, the typical ex- perimental setup for evaluating Named Entity Recognition (NER) systems is not directly ap- plicable to systems that process text in real time as the text is being typed. Evaluation is performed on a sentence level assuming the end-user is willing to wait until the en- tire sentence is typed for entities to be iden- tified and further linked to identifiers or co- referenced. We introduce a novel experimental setup for NER systems for applications where decisions about named entity boundaries need to be performed in an online fashion. We study how state-of-the-art methods perform under this setup in multiple languages and propose adaptations to these models to suit this new ex- perimental setup. Experimental results show that the best systems that are evaluated on each token after its typed, reach performance within 1–5 F1 points of systems that are evaluated at the end of the sentence. These show that entity recognition can be performed in this setup and open up the development of other NLP tools in a similar setup. 1 Introduction Automatically identifying named entities such as organizations, people and locations is a key com- ponent in processing written text as it aids with understanding the semantics of the text. Named entity recognition is used as a pre-processing step to subsequent tasks such as linking named entities to concepts in a knowledge graph, identifying the salience of an entity to the text, identifying coref- erential mentions, computing sentiment towards an entity, in question answering or for extracting relations. Figure 1: An example of the proposed task and evalua- tion setup. After the word ‘Foreign’ is typed, the model immediately predicts an NER label for this word, only using left context (‘A spokesman for’) and the word it- self. The prediction is then compared against the gold label to compute token-level F1 score. This token’s pre- diction will not be changed, even if the model’s internal prediction for it can be revised later as more tokens are typed. Identifying named entities as they are typed bene- fits any system that processes text on the fly. Exam- ples of such applications include: a) News editors – where named entities can be highlighted, suggested (auto-completion), co-referenced or linked as the editor is typing; b) auto-correct – where named entities that are just typed are less likely to need correction as they may come from a different lan- guage or be out-of-vocabulary (OOV); c) simulta- neous machine translation – where translation of OOV named entities requires different approaches; d) …

Content truncated to 500 words