DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue

38656

Authors

nan

Abstract

AbstractModern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated multilingual models are less robust for semantic parsing compared to other tasks. In global markets such as India and Latin America, robust multilingual semantic parsing is critical as codeswitching between languages is prevalent for bilingual users. In this work we dramatically improve the zero-shot performance of a multilingual and codeswitched semantic parsing system using two stages of multilingual alignment. First, we show that contrastive alignment pretraining improves both English performance and transfer efficiency. We then introduce a constrained optimization approach for hyperparameter-free adversarial alignment during finetuning. Our Doubly Aligned Multilingual Parser (DAMP) improves mBERT transfer performance by 3x, 6x, and 81x on the Spanglish, Hinglish and Multilingual Task Oriented Parsing benchmarks respectively and outperforms XLM-R and mT5-Large using 3.2x fewer parameters.

Keywords

nan

Full Content

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 322–337 July 9-14, 2023 ©2023 Association for Computational Linguistics AMR-based Network for Aspect-based Sentiment Analysis Fukun Ma1, Xuming Hu1, Aiwei Liu1, Yawen Yang1, Shuang Li1, Philip S. Yu2, Lijie Wen1∗ 1Tsinghua University, 2University of Illinois Chicago 1{mfk22,hxm19,liuaw20,yyw19,lisa18}@mails.tsinghua.edu.cn 2psyu@uic.edu, 1wenlj@tsinghua.edu.cn Abstract Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment classification task. Many recent works have used dependency trees to extract the relation between aspects and con- texts and have achieved significant improve- ments. However, further improvement is lim- ited due to the potential mismatch between the dependency tree as a syntactic structure and the sentiment classification as a seman- tic task. To alleviate this gap, we replace the syntactic dependency tree with the seman- tic structure named Abstract Meaning Repre- sentation (AMR) and propose a model called AMR-based Path Aggregation Relational Net- work (APARN) to take full advantage of se- mantic structures. In particular, we design the path aggregator and the relation-enhanced self- attention mechanism that complement each other. The path aggregator extracts seman- tic features from AMRs under the guidance of sentence information, while the relation- enhanced self-attention mechanism in turn im- proves sentence features with refined seman- tic information. Experimental results on four public datasets demonstrate 1.13% average F1 improvement of APARN in ABSA when com- pared with state-of-the-art baselines.1 1 Introduction Recent years have witnessed growing popularity of the sentiment analysis tasks in natural language processing (Li and Hovy, 2017; Birjali et al., 2021). Aspect-based sentiment analysis (ABSA) is a fine- grained sentiment analysis task to recognize the sentiment polarities of specific aspect terms in a given sentence (Jiang et al., 2011; Li et al., 2018; Seoh et al., 2021; Zhang et al., 2022a). For exam- ple, here is a restaurant review “All the money went into the interior decoration, none of it went to the chefs” and the sentiment polarity of two aspects 1The code will be available at https://github.com/THU- BPM/APARN. *Corresponding Author. amaze -01 we dish so ARG1 ARG0 how at amazed the were We dish was cop nsubj mark advcl small small dep advmod det nsubj domain degree Dependency Tree: AMR: Sentence: We were amazed at how small the dish was. Figure 1: Comparison of the dependency tree and the AMR. The aspect is red and the opinion term is blue. “interior decoration” and “chefs” are positive and negative, respectively. Thus, ABSA can precisely recognize the corresponding sentiment polarity for any aspect, different from allocating a general sen- timent polarity to a sentence in sentence-level sen- timent analysis. The key challenge for ABSA is to capture the relation between an aspect and its context, espe- cially opinion terms. In addition, sentences with multiple aspects and several opinion terms make the problem more complex. To this end, some pre- vious studies (Wang et al., 2016; Chen et al., 2017; Gu et al., 2018; Du et al., 2019; Liang et al., 2019; Xing et al., 2019) have devoted the main …

Content truncated to 500 words

Labeled Text

AbstractModern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated multilingual models are less robust for semantic parsing compared to other tasks. In global markets such as India and Latin America, robust multilingual semantic parsing is critical as codeswitching between languages is prevalent for bilingual users. In this work we dramatically improve the zero-shot performance of a multilingual and codeswitched semantic parsing system using two stages of multilingual alignment. First, we show that contrastive alignment pretraining improves both English performance and transfer efficiency. We then introduce a constrained optimization approach for hyperparameter-free adversarial alignment during finetuning. Our Doubly Aligned Multilingual Parser (DAMP) improves mBERT transfer performance by 3x, 6x, and 81x on the Spanglish, Hinglish and Multilingual Task Oriented Parsing benchmarks respectively and outperforms XLM-R and mT5-Large using 3.2x fewer parameters.

Publication Details

Paper ID:
38656
Year:
2023
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan