Deep Learning Framework for Measuring the Digital Strategy of Companies from Earnings Calls

Proceedings of the 28th International Conference on Computational Linguistics, pages 88–99 Barcelona, Spain (Online), December 8-13, 2020 88 Span-based Joint Entity and Relation Extraction with Attention-based Span-specific and Contextual Semantic Representations Bin Ji†, Jie Yu†, Shasha Li∗, Jun Ma, Qingbo Wu, Yusong Tan, Huijun Liu∗ College of Computer, National University of Defense Technology, Changsha, China {jibin, yj, shashali, majun}@nudt.edu.cn {qingbowu, yusongtan, liuhuijun}@nudt.edu.cn Abstract Span-based joint extraction models have shown their efficiency on entity recognition and rela- tion extraction. These models regard text spans as candidate entities and span tuples as candidate relation tuples. Span semantic representations are shared in both entity recognition and relation extraction, while existing models cannot well capture semantics of these candidate entities and relations. To address these problems, we introduce a span-based joint extraction framework with attention-based semantic representations. Specially, attentions are utilized to calculate seman- tic representations, including span-specific and contextual ones. We further investigate effects of four attention variants in generating contextual semantic representations. Experiments show that our model outperforms previous systems and achieves state-of-the-art results on ACE2005, CoNLL2004 and ADE. 1 Introduction This paper considers intra-sentence joint entity and relation extraction. For joint extraction mode can alleviate cascading errors and promote information utilization compared to pipelined one, this mode has drawn much attention. Typically, the joint extraction task is solved by sequence tagging based methods (Zheng et al., 2017; Chi et al., 2019). Rather than using sequence tagging based methods, recent works attempt to solve the task with span- based joint extraction mode (Dixit and Al-Onaizan, 2019; Luan et al., 2019). Typically, this mode first processes sentence text into text spans, which are span-based candidate entities (”spans” for short); then, calculates span semantic representations and performs span classification on them to obtain predicted entities; next, forms span-based candidate relation (”relation” for short) tuples with spans, and calculates relation semantic representations with corresponding span semantic representations; at last, performs relation classification on relation semantic representations and obtains predicted relation triples. This mode further improves joint extraction performance, whereas exists three problems. First, different tokens in span should contribute differently to span representation, which we call span- specific features. But existing methods treat each span token equally important (Eberts and Ulges, 2019) or just consider span head and tail tokens (Dixit and Al-Onaizan, 2019), ignoring these sig- nificant features. Take the span ”a Palestinian youth” in sentence 1 of Figure 1 as an example, the ”youth” should contribute much greater to the span representation than ”a” and ”Palestinian” when classifying the span into ”PER”. Second, local contextual information of relation tuples is omit- ted (Luan et al., 2018; Dixit and Al-Onaizan, 2019) or just calculated by max pooling way (Eberts and Ulges, 2019) when performing relation classification, which do not sufficiently capture informa- tion contained in it. Whereas local context may contain crucial information to help predict the rela- tions that relation tuples hold. A case study is shown in sentence 2 of Figure 1, the ”ownership” (in red font) can greatly help to determine …

Content truncated to 500 words