Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1841–1848 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1841 Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers Brian Lester, Daniel Pressel, Amy Hemmeter, Sagnik Ray Choudhury, and Srinivas Bangalore Interactions, Ann Arbor MI 48104 {blester,dpressel,ahemmeter,schoudhury,sbangalore} @interactions.com Abstract Current state-of-the-art models for named en- tity recognition (NER) are neural models with a conditional random field (CRF) as the final layer. Entities are represented as per-token la- bels with a special structure in order to decode them into spans. Current work eschews prior knowledge of how the span encoding scheme works and relies on the CRF learning which transitions are illegal and which are not to fa- cilitate global coherence. We find that by con- straining the output to suppress illegal transi- tions we can train a tagger with a cross-entropy loss twice as fast as a CRF with differences in F1 that are statistically insignificant, effec- tively eliminating the need for a CRF. We ana- lyze the dynamics of tag co-occurrence to ex- plain when these constraints are most effective and provide open source implementations of our tagger in both PyTorch and TensorFlow. 1 Introduction Named entity recognition (NER) is the task of find- ing phrases of interest in text that map to real world entities such as organizations (“ORG”) or locations (“LOC”). This is normally cast as a sequence label- ing problem where each token is assigned a label that represents its entity type. Multi-token enti- ties are handled by having special “Beginning” and “Inside” indicators that specify which tokens start, continue, or change the type of an entity. Rati- nov and Roth (2009) show that the IOBES tagging scheme, where entity spans must begin with a “B” token, end with an “E“ token and where single to- ken entities are labeled with an “S”, performs better than the traditional BIO scheme. The IOBES tag- ging scheme dictates that some token sequences are illegal. For example, one cannot start an entity with an “E” tag (such as a transition from an “O”, mean- ing it is outside of an entity, to “E-ORG”) nor can they change types in the middle of an entity—for example, transitioning from “I-ORG” to “I-LOC”. Most approaches to NER rely on the model learn- ing which transitions are legal from the training data rather than injecting prior knowledge of how the encoding scheme works. It is conventional wisdom that, for NER, mod- els with a linear-chain conditional random field (CRF) (Lafferty et al., 2001) layer perform better than those without, yielding relative performance increases between 2 and 3 percent in F1 (Ma and Hovy, 2016; Lample et al., 2016). A CRF with Viterbi decoding promotes, but does not guarantee, global coherence while simple greedy decoding does not (Collobert et al., 2011). Therefore, in a bidirectional LSTM (biLSTM) model with a CRF layer, illegal transitions are rare compared to mod- els that select the best scoring tag for each token. Due to the …

Content truncated to 500 words