Francis Bacon at SemEval-2023 Task 4: Ensembling BERT and GloVe for Value Identification in Arguments

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2039–2042 July 13-14, 2023 ©2023 Association for Computational Linguistics Francis Bacon at SemEval-2023 Task 4: Ensembling BERT and GloVe for Value Identification in Arguments Kenan Hasanaliyev Stanford University kenanhas@stanford.edu Kevin Li Stanford University kevinli7@stanford.edu Saanvi Chawla Stanford University saanvic@stanford.edu Michael Nath Stanford University mnath@stanford.edu Rohan Sanda Stanford University rsanda@stanford.edu Justin Wu Stanford Univeristy justwu@stanford.edu William Huang Stanford University willsh@stanford.edu Daniel Yang Stanford University dy92634@stanford.edu Shane Mion Stanford University smion@stanford.edu Kiran Bhat Stanford University kvbhat@stanford.edu Abstract In this paper, we discuss our efforts on SemEval-2023 Task 4, a task to classify the human value categories that an argument draws on. Arguments consist of a premise, con- clusion, and the premise’s stance on the con- clusion. Our team experimented with GloVe embeddings and fine-tuning BERT. We found that an ensembling of BERT and GloVe with RidgeRegression worked the best. 1 Introduction Identifying the values (e.g., humility and dom- inance) in an argument is a key part of under- standing the psychology of the argument’s ideator. Present-day NLP models allow us to more accu- rately identify arguments for their human values. These models can power better emotional under- standing for conversational agents. Based on the values you are exhibiting, a conversational agent would be able to change its response to brighten your mood or de-escalate the situation, resulting in a more engaging experience. To help advance the NLP effort in value identifi- cation, we participated in the SemEval-2023 Task 4 (Kiesel et al., 2023). The SemEval task focuses on classifying the human value categories that a textual argument draws on. The dataset for the task contains arguments consisting of a premise, con- clusion, and the premise’s stance on the conclusion (“in favor of” or “against”). The target label for each argument is one or more of 20 given human values. Our team experimented with two approaches for value identification, a baseline model using GloVe word embeddings, and a BERT model that we fine- tuned for multilabel classification of human values. We describe the implementation details of these models in this paper, as well as our experiments and results from these models. We found that en- sembling BERT and GloVe acheived a 10% im- provement in average F1-score over our baseline GloVe model. Our full implementation can be found at https: //github.com/claserken/MLabTask4SemEval. 2 Background SemEval-2023 Task 4 consisted of four inputs for each argument: an argument ID, the conclusion, the stance, and the premise. The argument ID is simply a marker for this specific argument (e.g., A01002). The premise is the full argument, such as “we should ban human cloning as it will only cause huge issues when you have a bunch of the same humans running around all acting the same." The stance is a binary choice of whether the premise is supported or not, with this specific one being “in favor of" while others may be “against." The con- clusion is drawn from the stance and the premise 2039 which …

Content truncated to 500 words