ViNLI: A Vietnamese Corpus for Studies on Open-Domain Natural Language Inference

4877

Authors

nan

Abstract

AbstractOver a decade, the research field of computational linguistics has witnessed the growth of corpora and models for natural language inference (NLI) for rich-resource languages such as English and Chinese. A large-scale and high-quality corpus is necessary for studies on NLI for Vietnamese, which can be considered a low-resource language. In this paper, we introduce ViNLI (Vietnamese Natural Language Inference), an open-domain and high-quality corpus for evaluating Vietnamese NLI models, which is created and evaluated with a strict process of quality control. ViNLI comprises over 30,000 human-annotated premise-hypothesis sentence pairs extracted from more than 800 online news articles on 13 distinct topics. In this paper, we introduce the guidelines for corpus creation which take the specific characteristics of the Vietnamese language in expressing entailment and contradiction into account. To evaluate the challenging level of our corpus, we conduct experiments with state-of-the-art deep neural networks and pre-trained models on our dataset. The best system performance is still far from human performance (a 14.20% gap in accuracy). The ViNLI corpus is a challenging corpus to accelerate progress in Vietnamese computational linguistics. Our corpus is available publicly for research purposes.

Keywords

nan

Full Content

Proceedings of the 29th International Conference on Computational Linguistics, pages 412–423 October 12–17, 2022. 412 GRASP: Guiding model with RelAtional Semantics using Prompt for Dialogue Relation Extraction Junyoung Son∗, Jinsung Kim∗, Jungwoo Lim∗, Heuiseok Lim† Computer Science and Engineering, Korea University Republic of Korea {s0ny,jin62304,wjddn803,limhseok}@korea.ac.kr Abstract The dialogue-based relation extraction (Dialo- gRE) task aims to predict the relations be- tween argument pairs that appear in dialogue. Most previous studies utilize fine-tuning pre- trained language models (PLMs) only with extensive features to supplement the low in- formation density of the dialogue by multi- ple speakers. To effectively exploit inherent knowledge of PLMs without extra layers and consider scattered semantic cues on the re- lation between the arguments, we propose a Guiding model with RelAtional Semantics us- ing Prompt (GRASP). We adopt a prompt- based fine-tuning approach and capture rela- tional semantic clues of a given dialogue with 1) an argument-aware prompt marker strategy and 2) the relational clue detection task. In the experiments, GRASP achieves state-of-the- art performance in terms of both F1 and F1c scores on a DialogRE dataset even though our method only leverages PLMs without adding any extra layers. 1 Introduction The relation extraction (RE) task aims to extract semantic relations from unstructured text such as a sentence, a document, or even a dialogue. RE plays a critical role in information extraction and knowl- edge base construction as it can extract structured relational information (Ji et al., 2010; Swampil- lai and Stevenson, 2010). However, the utilization of sentence-level RE in a conversational setting is limited because numerous relational facts ap- pear across multiple sentences with more than one speaker in a dialogue (Yao et al., 2021). Thus, the dialogue-based relation extraction (DialogRE) task, which includes argument pairs and their corre- sponding relations, has been proposed to encourage building a model that captures the underlying se- ∗These authors have equally contributed to this work †Corresponding author Dialogue 1 S1: Hey guys! Hey! 2 S2: Hey Pheebs, guess who we saw today. ...3 S3: Ooh, ooh, fun! Okay... um, Liam Neeson. 7 S3: The woman who cuts my hair! 8 S4: Okay, look, this could be a really long game. 9 S5: Your sister Ursula. 10 S3: Oh, really. 11 S5: Yeah, yeah, she works over at that place, uh... ...12 S3: Rift’s. Yeah, I know. 17 S6: Um, Pheebs, so, you guys just don’t get along? 18 S3: It’s mostly just dumb sister stuff, you know, I mean, like, everyone always thought of her as the pretty one, you know... ... Argument pair Trigger Relation Type R1 (Pheebs, PER) sister per:siblings (Ursula, PER) R2 (S3, PER) none per:alternate_names (Pheebs, PER) R3 (Rift’s, ORG) works over at per:employees_or_members (Ursula, PER) Table 1: Example of DialogRE data. The arguments are bold, and the triggers are underlined. The arguments and triggers are scattered throughout the dialogue, which leads to low information density. The triggers determine the direction of the proper relation indirectly. mantic spread in the dialogue, as presented in Table 1 …

Content truncated to 500 words

Labeled Text

AbstractOver a decade, the research field of computational linguistics has witnessed the growth of corpora and models for natural language inference (NLI) for rich-resource languages such as English and Chinese. A large-scale and high-quality corpus is necessary for studies on NLI for Vietnamese, which can be considered a low-resource language. In this paper, we introduce ViNLI (Vietnamese Natural Language Inference), an open-domain and high-quality corpus for evaluating Vietnamese NLI models, which is created and evaluated with a strict process of quality control. ViNLI comprises over 30,000 human-annotated premise-hypothesis sentence pairs extracted from more than 800 online news articles on 13 distinct topics. In this paper, we introduce the guidelines for corpus creation which take the specific characteristics of the Vietnamese language in expressing entailment and contradiction into account. To evaluate the challenging level of our corpus, we conduct experiments with state-of-the-art deep neural networks and pre-trained models on our dataset. The best system performance is still far from human performance (a 14.20% gap in accuracy). The ViNLI corpus is a challenging corpus to accelerate progress in Vietnamese computational linguistics. Our corpus is available publicly for research purposes.

Publication Details

Paper ID:
4877
Year:
2022
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan