MetaQA: Combining Expert Agents for Multi-Skill Question Answering
42736Authors
nan
Abstract
AbstractThe recent explosion of question-answering (QA) datasets and models has increased the interest in the generalization of models across multiple domains and formats by either training on multiple datasets or combining multiple models. Despite the promising results of multi-dataset models, some domains or QA formats may require specific architectures, and thus the adaptability of these models might be limited. In addition, current approaches for combining models disregard cues such as question-answer compatibility. In this work, we propose to combine expert agents with a novel, flexible, and training-efficient architecture that considers questions, answer predictions, and answer-prediction confidence scores to select the best answer among a list of answer predictions. Through quantitative and qualitative experiments, we show that our model i) creates a collaboration between agents that outperforms previous multi-agent and multi-dataset approaches, ii) is highly data-efficient to train, and iii) can be adapted to any QA format. We release our code and a dataset of answer predictions from expert agents for 16 QA datasets to foster future research of multi-agent systems.
Keywords
Full Content
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 328–339 May 2-6, 2023 ©2023 Association for Computational Linguistics Prompt Tuning with Contradictory Intentions for Sarcasm Recognition Yiyi Liu, Ruqing Zhang, Yixing Fan, Jiafeng Guo, Xueqi Cheng CAS Key Lab of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, Beijing, China {liuyiyi17s,zhangruqing,fanyixing,guojiafeng,cxq}@ict.ac.cn Abstract Recently, prompt tuning has achieved promis- ing results in a variety of natural language pro- cessing (NLP) tasks. The typical approach is to insert text pieces (i.e., templates) into the input and transform downstream tasks into the same form as pre-training. In essence, a high-quality template is the foundation of prompt tuning to support the performance of the converted cloze- style task. However, for sarcasm recognition, it is time-consuming and requires increasingly sophisticated domain knowledge to determine the appropriate templates and label words due to its highly figurative nature. In this work, we propose SarcPrompt, to incorporate the prior knowledge about contradictory intentions into prompt tuning for sarcasm recognition. Sar- cPrompt is inspired by that the speaker usually says the opposite of what they actually mean in the sarcastic text. Based on this idea, we explicitly mimic the actual intention by prompt construction and indicate whether the actual in- tention is contradictory to the literal content by verbalizer engineering. Experiments on three public datasets with standard and low-resource settings demonstrate the effectiveness of our SarcPrompt for sarcasm recognition. 1 Introduction Sarcasm is a sophisticated language phenomenon in which one conveys implicit intention with the opposite meaning of what is said or written literally (Campbell and Katz, 2012; Joshi et al., 2015). Due to its high ambivalence and figurative nature, sar- casm recognition which targets to predict a text as sarcastic or non-sarcastic, becomes a particularly challenging classification task. With the usage of sarcasm becoming prevalent on social media plat- forms like microblogs and online forums, sarcasm recognition has received growing research attention to facilitate sentiment analysis applications. Recent advances have shown that Pre-trained Language Models (PLMs), such as RoBERTa (Liu et al., 2019b), BERT (Devlin et al., 2019), can achieve promising performance in many downstream Nat- ural Language Processing (NLP) tasks (Xu et al., 2019; Liu et al., 2019a; Zhou et al., 2021). The success of PLMs has also attracted much attention for sarcasm recognition. Researchers mainly add extra classifiers on top of PLMs (Lou et al., 2021) to further train the models under classification ob- jectives, or re-train popular PLMs by incorporating sentiment knowledge and external sarcastic cor- pus (Babanejad et al., 2020). Despite fine-tuning has achieved satisfying re- sults, some recent studies (Schick and Schütze, 2021a,b) have found that one of its critical chal- lenges is the significant gap in objective forms be- tween pre-training and fine-tuning. This largely limits the transfer and adaptation of knowledge in PLMs to downstream tasks. Recently, a se- ries of studies propose to use prompt tuning (Han et al., 2021; …
Content truncated to 500 words
Labeled Text
AbstractThe recent explosion of question-answering (QA) datasets and models has increased the interest in the generalization of models across multiple domains and formats by either training on multiple datasets or combining multiple models. Despite the promising results of multi-dataset models, some domains or QA formats may require specific architectures, and thus the adaptability of these models might be limited. In addition, current approaches for combining models disregard cues such as question-answer compatibility. In this work, we propose to combine expert agents with a novel, flexible, and training-efficient architecture that considers questions, answer predictions, and answer-prediction confidence scores to select the best answer among a list of answer predictions. Through quantitative and qualitative experiments, we show that our model i) creates a collaboration between agents that outperforms previous multi-agent and multi-dataset approaches, ii) is highly data-efficient to train, and iii) can be adapted to any QA format. We release our code and a dataset of answer predictions from expert agents for 16 QA datasets to foster future research of multi-agent systems.
Publication Details
- Paper ID:
- 42736
- Year:
- 2023
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan