Semantic-aware Contrastive Learning for More Accurate Semantic Parsing

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4040 - 4052 December 7-11, 2022 ©2022 Association for Computational Linguistics Semantic-aware Contrastive Learning for More Accurate Semantic Parsing Shan Wu1,2, Chunlei Xin1,2, Bo Chen1, Xianpei Han1,3,∗, Le Sun1,3 1Chinese Information Processing Laboratory 2University of Chinese Academy of Sciences, Beijing, China 3State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China {wushan2018,chunlei2021,chenbo,xianpei,sunle}@iscas.ac.cn Abstract Since the meaning representations are detailed and accurate annotations which express fine- grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an autoregressive fashion. In this pa- per, we propose a semantic-aware contrastive learning algorithm, which can learn to dis- tinguish fine-grained meaning representations and take the overall sequence-level seman- tic into consideration. Specifically, a multi- level online sampling algorithm is proposed to sample confusing and diverse instances. Three semantic-aware similarity functions are designed to accurately measure the distance between meaning representations as a whole. And a ranked contrastive loss is proposed to pull the representations of the semantic- identical instances together and push negative instances away. Experiments on two stan- dard datasets show that our approach achieves significant improvements over MLE baselines and gets state-of-the-art performances by sim- ply applying semantic-aware contrastive learn- ing on a vanilla SEQ2SEQ model. 1 Introduction Semantic parsing aims to translate natural lan- guage utterances into formal meaning represen- tations(MRs), which has attracted much attention for many years (Wong and Mooney, 2007; Kate et al., 2005; Lu et al., 2008; Guo et al., 2019). Recent studies mostly treat semantic parsing as a neural sequence to sequence translation task via encoder-decoder frameworks (Dong and Lap- ata, 2016; Jia and Liang, 2016; Rabinovich et al., 2017; Chen et al., 2018; Zhao et al., 2020; Shao et al., 2020). To train neural semantic parsers, most studies employ Maximum Likelihood Estimation, which ∗Corresponding Author ���: what is the river of the state with the highest place? ���: ���, (���������������( ��� ), ���������(���, ���), ���������������(���), ���������(���, ���), ℎ������ℎ���������(���)) ���’: ���, (���������������( ��� ), ���������(���, ���), ���������������(���), ���������(���, ���), ℎ������ℎ���������(���)) ���: the player with no less than 3 turnover ���: ������������������������( ��� ��� (���, ���������_���������������������������, ≥ , 3), ������������������) ���’: ������������������������( ��� ��� (���, ���������_���������������������������, < , 3), ������������������) (b) (a) Figure 1: Meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics. (a) the correct MR y and the predicted y′ have similar token sequences but with very different semantics; (b) one token error (from ≥ to <) will reverse the semantics of MR. optimizes the probabilities of the tokens in an au- toregressive fashion. By decomposing sequence tasks into tokens, MLE training is good at n-gram based generation tasks such as machine translation and paraphrasing, but overlooks the semantics in a whole level. Unfortunately, meaning representation is for- mal and detailed annotation, that is, it should be viewed as a sequence-level whole possessing fine- grained semantics. Such features make it hard to train accurate semantic parsers …

Content truncated to 500 words