AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6019–6026, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6019 AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network Xinyu Wang⋄‡, Yong Jiang†∗, Nguyen Bach†, Tao Wang†, Zhongqiang Huang†, Fei Huang†, Kewei Tu⋄∗ ⋄School of Information Science and Technology, ShanghaiTech University ⋄Shanghai Engineering Research Center of Intelligent Vision and Imaging ⋄University of Chinese Academy of Sciences †DAMO Academy, Alibaba Group Abstract The linear-chain Conditional Random Field (CRF) model is one of the most widely-used neural sequence labeling approaches. Exact probabilistic inference algorithms such as the forward-backward and Viterbi algorithms are typically applied in training and prediction stages of the CRF model. However, these al- gorithms require sequential computation that makes parallelization impossible. In this pa- per, we propose to employ a parallelizable ap- proximate variational inference algorithm for the CRF model. Based on this algorithm, we design an approximate inference network that can be connected with the encoder of the neu- ral CRF model to form an end-to-end network, which is amenable to parallelization for faster training and prediction. The empirical results show that our proposed approaches achieve a 12.7-fold improvement in decoding speed with long sentences and a competitive accu- racy compared with the traditional CRF ap- proach. 1 Introduction Sequence labeling assigns each token with a la- bel in a sequence. Tasks such as Named Entity Recognition (NER) (Sundheim, 1995), Part-Of- Speech (POS) tagging (DeRose, 1988) and chunk- ing (Tjong Kim Sang and Buchholz, 2000) can all be formulated as sequence labeling tasks. BiLSTM- CRF (Huang et al., 2015; Lample et al., 2016; Ma and Hovy, 2016) is one of the most successful neu- ral sequence labeling architectures. It feeds pre- trained (contextual) word representations into a single layer bi-directional LSTM (BiLSTM) en- coder to extract contextual features and then feeds ∗ Yong Jiang and Kewei Tu are the corresponding authors. ‡: This work was conducted when Xinyu Wang was interning at Alibaba DAMO Academy. ⋄: {wangxy1, tukw}@shanghaitech.edu.cn, {yongjiang.jy, nguyen.bach, leeo.wangt,†: z.huang, f.huang}@alibaba-inc.com these features into a CRF (Lafferty et al., 2001) de- coder layer to produce final predictions. The CRF layer is a linear-chain structure that models the rela- tion between neighboring labels. In the traditional CRF approach, exact probabilistic inference algo- rithms such as the forward-backward and Viterbi algorithms are applied for training and prediction respectively. In many sequence labeling tasks, the CRF layer leads to better results than the simpler method of predicting each label independently. In practice, we sometimes require very fast se- quence labelers for training (e.g., on huge datasets like WikiAnn (Pan et al., 2017)) and prediction (e.g. for low latency online serving). The BiLSTM encoder and the CRF layer both contain sequential computation and require O(n) time over n input words even when parallelized on GPU. A com- mon practice to improve the speed of the encoder is to replace the BiLSTM with a CNN structure (Collobert et al., 2011; Strubell et al., 2017), dis- till larger …

Content truncated to 500 words