KaMiKla at SemEval-2022 Task 3: AlBERTo, BERT, and CamemBERT—Be(r)tween Taxonomy Detection and Prediction

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 282 - 290 July 14-15, 2022 ©2022 Association for Computational Linguistics KaMiKla at SemEval-2022 Task 3: AlBERTo, BERT, and CamemBERT—Be(r)tween Taxonomy Detection and Prediction Karl Vetter karl.vetter @student. uni-tuebingen.de Miriam Segiet miriam.segiet @student. uni-tuebingen.de Klara Lennermann klara.lennermann @student. uni-tuebingen.de Abstract This paper describes our system submitted for SemEval Task 3: Presupposed Taxonomies: Evaluating Neural Network Semantics (Zam- parelli et al., 2022). We participated in both the binary classification and the regression sub- task. Target sentences are classified according to their taxonomical relation in subtask 1 and according to their acceptability judgment in subtask 2. Our approach in both subtasks is based on a neural network BERT model. We used separate models for the three languages covered by the task, English, French, and Ital- ian. For the second subtask, we used median averaging to construct an ensemble model. We ranked 15th out of 21 groups for subtask 1 (F1- score: 77.38%) and 11th out of 17 groups for subtask 2 (RHO: 0.078). 1 Introduction The recognition of lexical relationships between words and the corresponding generalization has attracted increasing attention in computational lin- guistics. Today, there already exist resources cover- ing manually marked semantic relationships, e.g., taxonomic relations, such as the lexical database WordNet (Miller, 1992) or the multilingual dictio- nary and semantic network BabelNet (Navigli and Ponzetto, 2010). Luu et al. (2016) define taxonomic relations be- tween two terms as an is–a relation. In such a relation there is a hypernym, i.e., a supertype, and a hyponym, i.e., a subtype. Both the supertype and subtype are sets covering, in our task, certain semantic categories. In a relation such as animal– dog, the animal is the superordinate category and the dog is the subordinate term. Furthermore, those specific sets are included in a relation forming a spe- cial hierarchy (Kay, 1971). As stated by Nguyen et al. (2017), in such an is–a relation the supertype necessarily implies the subtype, but not vice versa. SemEval 2022 Task 3 is a taxonomy detection and prediction task consisting of two subtasks: a binary classification and a regression task, both covering the languages English, French, and Ital- ian. We propose an approach based on the transformer-based machine learning model BERT. Since BERT is a bidirectional model producing state-of-the-art results (Devlin et al., 2019), we used this pre-trained model for our analysis. For the three different languages, the corresponding BERT models were used (Devlin et al., 2019; Polignano et al., 2019; Martin et al., 2020).1 2 Task Description In the present shared task (Zamparelli et al., 2022), the taxonomic sentence structures in the given files are composed of different artificially generated con- structions enforcing presuppositions. Table 1 shows example sentences provided in the English test set. According to the task description page, the French and Italian datasets are translated versions of the English dataset that were slightly adapted. The argument nouns in the given files come from 30 semantic categories including, among oth- ers, dogs, birds, …

Content truncated to 500 words