DRK: Discriminative Rule-based Knowledge for Relieving Prediction Confusions in Few-shot Relation Extraction
5028Authors
nan
Abstract
AbstractFew-shot relation extraction aims to identify the relation type between entities in a given text in the low-resource scenario. Albeit much progress, existing meta-learning methods still fall into prediction confusions owing to the limited inference ability over shallow text features. To relieve these confusions, this paper proposes a discriminative rule-based knowledge (DRK) method. Specifically, DRK adopts a logic-aware inference module to ease the word-overlap confusion, which introduces a logic rule to constrain the inference process, thereby avoiding the adverse effect of shallow text features. Also, DRK employs a discrimination finding module to alleviate the entity-type confusion, which explores distinguishable text features via a hierarchical contrastive learning. We conduct extensive experiments on four types of meta tasks and the results show promising improvements from DRK (6.0% accuracy gains on average). Besides, error analyses reveal the word-overlap and entity-type errors are the main courses of mispredictions in few-shot relation extraction.
Keywords
Full Content
Proceedings of the 29th International Conference on Computational Linguistics, pages 218–241 October 12–17, 2022. 218 LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging Andy Rosenbaum∗ Amazon, Cambridge, USA andros@amazon.com Saleh Soltan Amazon, New York, USA ssoltan@amazon.com Wael Hamza Amazon, Dallas, USA waelhamz@amazon.com Yannick Versley Amazon, Aachen, Germany yversley@amazon.de Markus Boese Amazon, Aachen, Germany boesem@amazon.de Abstract We present LINGUIST, a method for gener- ating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilin- gual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt. In a 10-shot novel intent setting for the SNIPS dataset, LINGUIST surpasses state-of-the-art approaches (Back-Translation and Example Extrapolation) by a wide margin, showing ab- solute improvement for the target intents of +1.9 points on IC Recall and +2.5 points on ST F1 Score. In the zero-shot cross-lingual set- ting of the mATIS++ dataset, LINGUIST out- performs a strong baseline of Machine Transla- tion with Slot Alignment by +4.14 points abso- lute on ST F1 Score across 6 languages, while matching performance on IC. Finally, we ver- ify our results on an internal large-scale multi- lingual dataset for conversational agent IC+ST and show significant improvements over a baseline which uses Back-Translation, Para- phrasing and Slot Catalog Resampling. To our knowledge, we are the first to demonstrate in- struction fine-tuning of a large-scale seq2seq model to control the outputs of multilingual intent- and slot-labeled data generation. 1 Introduction Conversational agents typically rely on large quantities of labeled training data to understand user requests through Intent Classification and Slot Tagging (IC+ST) (Tur and De Mori, 2011). Such data is plentiful for existing usage patterns (al- though costly to annotate), yet scarce for new in- tents/slots and new languages. A growing trend to address this problem is to generate synthetic train- ing data, e.g. via Paraphrasing, Back-Translation (BT), slot replacement, and Example Extrapola- tion (Ex2). (Jolly et al., 2020; Xie et al., 2020; ∗Correspondence Author: <andros@amazon.com>. Author contributions are listed in Appendix A. INPUT: <language> English </language> <intent> GetWeather </intent> <include> [1 * ] , [3 snow ] [5 tomorrow ] </include> <labels> [1=geographic_poi , [2=country , [3=condition_description , [4=city , [5=timeRange </labels> <examples> Will the weather be okay in [1 Yellowstone National Park ] [5 one week from now ] ? <br> will it [3 rain ] at the [1 Statue of Liberty ] at [5 noon ] <br> What’s the weather like at [1 Disneyworld ] in [5 november ] <br> I need the weather info for the [1 Guggenheim Museum ] in [2 Spain ] <br> What is the weather forecast for [5 October 12, 2022 ] in [4 Gyeongju ] </examples> OUTPUTS: 1. Give the [1 National Wildlife Refuge ] forecast for [3 snow ] [5 tomorrow ] 2. Can I get the [3 snow ] forecast for [1 Lake Tahoe ] [5 tomorrow ] ? 3. I want to know if it will [3 snow ] [5 tomorrow ] at [1 Mount Rainier State …
Content truncated to 500 words
Labeled Text
AbstractFew-shot relation extraction aims to identify the relation type between entities in a given text in the low-resource scenario. Albeit much progress, existing meta-learning methods still fall into prediction confusions owing to the limited inference ability over shallow text features. To relieve these confusions, this paper proposes a discriminative rule-based knowledge (DRK) method. Specifically, DRK adopts a logic-aware inference module to ease the word-overlap confusion, which introduces a logic rule to constrain the inference process, thereby avoiding the adverse effect of shallow text features. Also, DRK employs a discrimination finding module to alleviate the entity-type confusion, which explores distinguishable text features via a hierarchical contrastive learning. We conduct extensive experiments on four types of meta tasks and the results show promising improvements from DRK (6.0% accuracy gains on average). Besides, error analyses reveal the word-overlap and entity-type errors are the main courses of mispredictions in few-shot relation extraction.
Publication Details
- Paper ID:
- 5028
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan