Global Attention for Name Tagging

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 86–96 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 86 Global Attention for Name Tagging Boliang Zhang, Spencer Whitehead, Lifu Huang and Heng Ji Computer Science Department Rensselaer Polytechnic Institute {zhangb8,whites5,huangl7,jih}@rpi.edu Abstract Many name tagging approaches use local con- textual information with much success, but fail when the local context is ambiguous or lim- ited. We present a new framework to improve name tagging by utilizing local, document- level, and corpus-level contextual informa- tion. We retrieve document-level context from other sentences within the same document and corpus-level context from sentences in other topically related documents. We propose a model that learns to incorporate document- level and corpus-level contextual informa- tion alongside local contextual information via global attentions, which dynamically weight their respective contextual information, and gating mechanisms, which determine the in- fluence of this information. Extensive exper- iments on benchmark datasets show the ef- fectiveness of our approach, which achieves state-of-the-art results for Dutch, German, and Spanish on the CoNLL-2002 and CoNLL- 2003 datasets.1. 1 Introduction Name tagging, the task of automatically identify- ing and classifying named entities in text, is often posed as a sentence-level sequence labeling prob- lem where each token is labeled as being part of a name of a certain type (e.g., location) or not (Chin- chor and Robinson, 1997; Tjong Kim Sang and De Meulder, 2003). When labeling a token, lo- cal context (i.e., surrounding tokens) is crucial be- cause the context gives insight to the semantic meaning of the token. However, there are many in- stances in which the local context is ambiguous or lacks sufficient content. For example, in Figure 1, the query sentence discusses “Zywiec” selling a pose:1The programs are publicly available for research pur- https://github.com/boliangz/global_ attention_ner product and profiting from these sales, but the lo- cal contextual information is ambiguous as more than one entity type could be involved in a sale. As a result, the baseline model mistakenly tags “Zywiec” as a person (PER) instead of the cor- rect tag, which is organization (ORG). If the model has access to supporting evidence that provides ad- ditional, clearer contextual information, then the model may use this information to correct the mis- take given the ambiguous local context. Baseline: So far this year [PER Zywiec], whose full name is Zaklady Piwowarskie w Zywcu SA, has netted six million zlotys on sales of 224 million zlotys. So far this year [ORG Zywiec], whose full name is Zaklady Piwowarskie w Zywcu SA, has netted six million zlotys on sales of 224 million zlotys. Our model (Document­level + Corpus­level Attention): Van Boxmeer also said [ORG Zywiec] would be boosted by its recent shedding of soft drinks which only accounted for about three percent of the firm's overall sales and for which 7.6 million zlotys in provisions had already been made. Polish brewer [ORG Zywiec]'s 1996 profit slump may last into next year due in part to hefty …

Content truncated to 500 words