Weakly Supervised Attention Networks for Entity Recognition

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6268–6273, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6268 Weakly Supervised Attention Networks for Entity Recognition Barun Patra⇤, Joel Ruben Antony Moniz⇤ {barunpatra95, jramoniz}@gmail.com Abstract The task of entity recognition has tradition- ally been modelled as a sequence labelling task. However, this usually requires a large amount of fine-grained data annotated at the token level, which in turn can be expensive and cumbersome to obtain. In this work, we aim to circumvent this requirement of word-level an- notated data. To achieve this, we propose a novel architecture for entity recognition from a corpus containing weak binary presence/ab- sence labels, which are relatively easier to ob- tain. We show that our proposed weakly su- pervised model, trained solely on a multi-label classification task, performs reasonably well on the task of entity recognition, despite not having access to any token-level ground truth data. 1 Introduction Entity Recognition frequently finds use as a first step in numerous downstream NLP tasks (Wang and Xue, 2017; Lee et al., 2018; Liang et al., 2018). Traditionally, it has been posed as a se- quence labeling task (Lample et al., 2016; Ma and Hovy, 2016), which in turn requires corpora with token-based annotations. A key drawback of this formulation, however, lies in its dependence on corpora annotated at the token-level, which can of- ten be tedious to obtain and expensive to annotate. One potential way of overcoming this limitation is to move towards a method that utilizes a weaker form of supervision that is easier to obtain. In this work, we focus on one such form of weak su- pervision: binary labels that indicate the presence of an entity type. The cognitive load of selecting whether an entity type is present or not is usually less than that of actually highlighting and annotat- ing spans with their correct entity types. It also stands to reason that providing these binary labels ⇤Equal Contribution might be faster. Both these properties are particu- larly advantageous for a human-in-the-loop setup in a user facing task, since a user is more likely to answer a yes/no question than to provide the annotated entity spans. This, in turn, facilitates cheaper and faster data collection; be it explicitly in the form of feedback questions, or implicitly from mined user logs (for example, clicked search engine results for queries related to ”movies” are likely to contain the entity in question (Xu et al., 2009)). In this work, we make first steps towards moving away from span-based corpora, relying solely on binary presence/absence classification labels for extracting entities. We propose a novel attention-based model that, though trained on a multi-label classification task, can be used for en- tity recognition. We show the efficacy of our pro- posed model on the widely used 2003 CoNLL dataset. Our model achieves reasonable perfor- mance without having access to token-level anno- tations. We …

Content truncated to 500 words