Igevorse at SemEval-2018 Task 10: Exploring an Impact of Word Embeddings Concatenation for Capturing Discriminative Attributes

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 995–998 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Igevorse at SemEval-2018 Task 10: Exploring an Impact of Word Embeddings Concatenation for Capturing Discriminative Attributes Maxim Grishin Innopois University Innopolis, Russia m.grishin@innopolis.ru igevorse@gmail.com Abstract Semantic differences extraction is a challeng- ing problem in Natural Language Processing and its solution is necessary for a realistic se- mantic representation as similarity informa- tion is not sufficient to capture individual as- pects of meaning. This paper presents a com- parison of several approaches for capturing discriminative attributes and considers an im- pact of concatenation of several word embed- dings of different nature on the classification performance. A similarity-based method is proposed and compared with machine learn- ing approaches. It is shown that this method outperforms others on all the considered word vector models and there is a performance in- crease when concatenated datasets are used. 1 Introduction Detecting semantic similarity is done well by state-of-the-art models. However, if the model is only good at similarity detection, it would have a limited practical usage (Krebs et al., 2018), since the task of understanding the semantics of words cannot be done without capturing semantic differ- ences. Semantic difference is a ternary relation be- tween two concepts (apple, banana) and a dis- criminative feature (red) that characterizes the first concept but not the other. Semantic differ- ence detection is a binary classification task: given a triple (apple, banana, red), the task is to deter- mine whether it exemplifies a semantic difference or not (Krebs et al., 2018). In this paper two con- cepts and a discriminative attribute attr are repre- sented as (word1, word2, attr). This research was done during the participation in ”Capturing Discriminative Attributes” task of SemEval 2018 competition. The paper is organized as follows. Section 2 describes the methods used. Section 3 shows the results and analyzes them. Section 4 mentions fu- ture directions. Section 5 concludes the paper. 2 Methods There were several approaches considered: classi- cal machine learning algorithms and a similarity- based model. 2.1 Data Preparation Dataset is provided by SemEval 2018 challenge organizers. • The train set consists of 17501 auto- matically generated samples of the form (word1, word2, attr, y), where y is a bi- nary target variable indicating whether attr is a discriminative attribute for word1 and word2. Classes are imbalanced: there are 63.83% samples for class 0 (not a discrimi- native attribute) and 36.16% for class 1 (is a discriminative attribute). • The validation set contains 2722 manually curated samples of the same form. Classes are almost balanced: 50.1% of samples have class 0, 49.9% - class 1. • There are 2340 samples in the test set, 55.3% for class 0, 44.7% for class 1. Each triple (word1, word2, attr) was con- verted to a numeric vector using pre-trained word embeddings. These vectors form a vector space, such that words that share common contexts in the corpus are located in close proximity …

Content truncated to 500 words