PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

Proceedings of the 29th International Conference on Computational Linguistics, pages 5327–5334 October 12–17, 2022. 5327 PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling Guanting Dong1∗, Daichi Guo1∗, Liwen Wang1∗, Xuefeng Li1∗, Zechen Wang1, Chen Zeng1, Keqing He2, Jinzheng Zhao3, Hao Lei1, Xinyue Cui1, Yi Huang4, Junlan Feng4, Weiran Xu1∗ 1Beijing University of Posts and Telecommunications, Beijing, China 2Meituan Group, Beijing, China 3University of Surrey, UK 4China Mobile Research Institute Abstract Most existing slot filling models tend to mem- orize inherent patterns of entities and corre- sponding contexts from training data. How- ever, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed seman- tic structure awareness transferring method for training perturbation-robust slot filling mod- els. Specifically, we introduce two MLM-based training strategies to respectively learn contex- tual semantic structure and word distribution from unsupervised language perturbation cor- pus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts. 1 Introduction The slot filling (SF) task in the goal-oriented dialog system aims to identify task-related slot types in certain domains for understanding user utterances. Traditional supervised slot filling models and se- quence labeling methods (Liu and Lane, 2015, 2016; Goo et al., 2018; Niu et al., 2019; He et al., 2020a,b; Wang et al., 2022a) have shown remark- able performance. However, these models tend to memorize inherent patterns of entities and contexts (Wang et al., 2022b; Lin et al., 2021). Faced with uncertainty and diversity of human language ex- pression, the perturbation of entities and contexts will lead to a decrease in the generalization ability of the SF model, which hinders its further applica- tion in practical dialog scenarios. ∗The first four authors contribute equally. Weiran Xu is the corresponding author. Email: dongguanting@bupt.edu.cn Excuse me, could you tell me which hotel around is cheap O O O O O O O B-TYPE O O B-PRICE Simplification Original I need a cheap hotel O O O O B-TYPE Paraphrase I want to proceed with the cheaper hotel O O O O O O O B-TYPE False True Figure 1: The impact of diverse spoken language pertur- bations on the slot filling system in real scenarios. Due to the variety of expression habits, users may not interact with the dialogue system abiding by a rigid input mode in real dialog scenarios. In- stead, the expression styles of users would be of high lexical and syntactic diversity while users ex- press their intentions. An interesting finding is that, every expression retains the key semantic informa- tion of the sentence to ensure consistency of the intention, but it inevitably damages the semantic structure of the context. …

Content truncated to 500 words