Neuralizing Regular Expressions for Slot Filling

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9481–9498 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9481 Neuralizing Regular Expressions for Slot Filling Chengyue Jiang⋄, Zijian Jin†, Kewei Tu⋄∗ ⋄ School of Information Science and Technology, ShanghaiTech University Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences Shanghai Engineering Research Center of Intelligent Vision and Imaging † New York University {jiangchy, tukw}@shanghaitech.edu.cn zj2076@nyu.edu Abstract Neural models and symbolic rules such as reg- ular expressions have their respective merits and weaknesses. In this paper, we study the integration of the two approaches for the slot filling task by converting regular expressions into neural networks. Specifically, we first con- vert regular expressions into a special form of finite-state transducers, then unfold its approx- imate inference algorithm as a bidirectional re- current neural model that performs slot filling via sequence labeling. Experimental results show that our model has superior zero-shot and few-shot performance and stays competi- tive when there are sufficient training data. 1 Introduction Neural approaches almost dominate recent natural language processing (NLP) research. The tremen- dous success of neural networks benefits from their strong capability of learning from a large amount of annotated data. On the other hand, systems based on symbolic rules, while being no longer the mainstream approach to NLP research, are still very widely used in practice because of their in- terpretability, trustworthiness and decent zero-shot performance in data-scarce scenarios. Therefore, how to integrate neural and symbolic approaches while retaining their respective advantages is be- coming an active research direction. In this paper, we try to integrate neural networks with regular expressions (RE), one of the most widely used forms of symbolic rules. We focus on the slot filling task (SF), a typical application scenario of REs that aims to identify words in a sentence that carry specific information. For exam- ple, given the sentence “show me the flights from New York to Dallas.”, SF aims to tag the span “New York” as the content of slot fr.city (the departing city). An RE specifies text patterns of both con- ∗Corresponding author. tents and contexts and uses capturing groups to mark target contents. Existing methods that integrate neural networks and symbolic rules either use rule outputs as pseudo-labels to distill knowledge into neural net- works (Zhang et al., 2018; Hu et al., 2016), or use rule outputs to influence attention weights or output logits of neural networks (Luo et al., 2018; Wang et al., 2019; Li and Srikumar, 2019). These in- tegration methods can outperform both rules and pure neural networks when there are sufficient train- ing data. However, since they still require train- ing, their performance is well below that of rules in zero-shot and low-resource settings. More re- cently, Jiang et al. (2020) proposed a novel method that converts REs into neural networks, which can match the performance of the original REs without training and can compete with previous integra- tion methods when …

Content truncated to 500 words