UoR-NCL at SemEval-2022 Task 3: Fine-Tuning the BERT-Based Models for Validating Taxonomic Relations

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 260 - 265 July 14-15, 2022 ©2022 Association for Computational Linguistics UoR-NCL at SemEval-2022 Task 3: Fine-Tuning the BERT-Based Models for Validating Taxonomic Relations Thanet Markchom University of Reading, UK t.markchom@pgr.reading.ac.uk Huizhi Liang Newcastle University, UK huizhi.liang@newcastle.ac.uk Jiaoyan Chen University of Oxford, UK jiaoyan.chen@cs.ox.ac.uk Abstract In human languages, there are many presuppo- sitional constructions that impose a constrain on the taxonomic relations between two nouns depending on their order. These constructions create a challenge in validating taxonomic rela- tions in real-world contexts. In SemEval2022- Task3 Presupposed Taxonomies: Evaluating Neural Network Semantics (PreTENS), the or- ganizers introduced a task regarding validat- ing the taxonomic relations within a variety of presuppositional constructions. This task is divided into two subtasks: classification and re- gression. Each subtask contains three datasets in multiple languages, i.e., English, Italian and French. To tackle this task, this work pro- poses to fine-tune different BERT-based mod- els pre-trained on different languages. Accord- ing to the experimental results, the fine-tuned BERT-based models are effective compared to the baselines for classification. For regression, the fine-tuned models show promising perfor- mances with the possibility of improvement. 1 Introduction Taxonomic relations are one of the significant lex- ical relationships that have been used in many ap- plications such as question answering (Yih et al., 2013), sentiment analysis (Araque et al., 2019) and biomedical ontologies (Bodenreider, 2004). In natural languages, there are many constructions that constrain the taxonomic relation between two nouns based on the order of these two nouns. For instance, given a sentence “I have a dog, not a pet”. The construction “I have a ..., not a ...” implies that the taxonomic relation does not hold between “dog” and “pet”. This can be seen as a presupposition im- posed by the construction. However, this is not true since dogs are pets. Thus, with various presupposi- tional constructions, validating taxonomic relations becomes more complicated in the real world. To address this issue, SemEval2022-Task3 Pre- supposed Taxonomies: Evaluating Neural Network Semantics (PreTENS) (Zamparelli et al., 2022) in- troduces the task where taxonomic relations have to be validated in different presuppositional construc- tions. This task proposes novel datasets in multiple languages (i.e., English, Italian and French) con- taining sentences with different two-noun construc- tions. Each sentence is labeled by an acceptability label for classification and an acceptability score for regression. Two challenges have been raised in this task: (1) a taxonomic relation between two nouns in the sentence must be detected, and (2) the construction which embeds the two nouns must also be validated. To effectively validate taxonomic relations in such constructions, understanding the contexts or semantic meanings of these constructions are the key. Many previous studies have shown that pre- trained models comprise the prior knowledge of context comprehension (Yang et al., 2019). Re- cently, the language mode called BERT has been widely used in several tasks. The BERT model can be pre-trained with the self-supervised method to generate word/token or sentence …

Content truncated to 500 words