Stanford MLab at SemEval-2021 Task 8: 48 Hours Is All You Need

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1245–1248 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1245 Stanford MLab at SemEval-2021 Task 8: 48 Hours Is All You Need Patrick Liu, Niveditha Iyer, Erik Rozi, Ethan A. Chi Stanford University {pliu1, nivsiyer, erikrozi}@stanford.edu, ethanchi@cs.stanford.edu Abstract This paper presents our system for the Quan- tity span identification, Unit of measurement identification and Value modifier classification subtasks of the MeasEval 2021 task. The pur- pose of the Quantity span identification task was to locate spans of text that contain a count or measurement, consisting of a value, usually followed by a unit and occasionally additional modifiers. The goal of the modifier classifica- tion task was to determine whether an associ- ated text fragment served to indicate range, tol- erance, mean value, etc. of a quantity. The de- veloped systems used pre-trained BERT mod- els which were fine-tuned for the task at hand. We present our system, investigate how archi- tectural decisions affected model predictions, and conduct an error analysis. Overall, our sys- tem placed 12 / 19 in the shared task and in the 2nd place for the Unit subcategory. 1 Introduction The growing ease of access to large bodies of scien- tific information and research has not been accom- panied by an improvement in ease of analysis or understanding of scientific text. While the perfor- mance of natural language processing tasks such as part-of-speech (POS) tagging and dependency parsing have seen improvements, analyzing counts and measurements in scientific texts has remained a largely unaddressed problem, though some work has been done by (Berrahou et al., 2013). Measurements involve quantification (in units such as litres or kilograms), entities and the mea- sured properties (e.g. growth rate of a fungus) and value modifiers (e.g. mean, range, tolerance). Ex- tracting semantic relations between quantities, en- tities, value modifiers, and units of measurement and deriving meaning from those components is challenging because scientific communication can be ambiguous or inconsistent. In addition, the loca- tion of this information relative to the measurement tends to vary. The precise extraction and contextualization of measurements could enable accurate summariza- tion of large bodies of scientific literature. Addi- tionally, this could allow for unordered measure- ment and numeric data from scientific texts to be transformed into standardized ordered data for eas- ier analysis. In this paper, we describe a rapidly implemented, lightweight model that extracts measurement con- text in natural language. Our pipeline was imple- mented in approximately 48 hours by the first three authors, who had minimal formal neural NLP expe- rience. Despite this, we were able to achieve strong results in the two categories (Quantity and Unit) that we entered, achieving second place in the Unit category and 12 / 19 overall. 2 Background 2.1 MeasEval Task Setup Data for this task is made available through the MeasEval repository (Harper et al., 2021). Input data are formatted as a paragraph of scientific text in English from which measurements would be extracted. …

Content truncated to 500 words