KGP at SemEval-2021 Task 8: Leveraging Multi-Staged Language Models for Extracting Measurements, their Attributes and Relations

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 387–396 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 387 KGP at SemEval-2021 Task 8: Leveraging Multi-Staged Language Models for Extracting Measurements, their Attributes and Relations Neel Karia∗, Ayush Kaushal∗, Faraaz Mallick∗ Department of Computer Science and Engineering Indian Institute of Technology Kharagpur {karianeel, ayushkaushal, faraazrm}@iitkgp.ac.in Abstract SemEval-2021 Task 8: MeasEval aims at im- proving the machine understanding of mea- surements in scientific texts through a set of en- tity and semantic relation extraction sub-tasks on identifying quantity spans along with vari- ous attributes and relationships. This paper de- scribes our system, consisting of a three-stage pipeline, that leverages pre-trained language models to extract the quantity spans in the text, followed by intelligent templates to identify units and modifiers. Finally, it identifies the quantity attributes and their relations using lan- guage models boosted with a feature re-using hierarchical architecture and multi-task learn- ing. Our submission significantly outperforms the baseline, with the best model from the post- evaluation phase delivering more than 100% increase on F1 (Overall) from the baseline. 1 Introduction Most scientific experiments are accompanied by relevant measurements, which help researchers to quantify their observations and qualitative argu- ments. Measurements also play a pivotal role in summarizing large experiments, and provide a brief idea of the results obtained. It is customary for sci- entists to present their research in the form of scien- tific papers. Nowadays, with thousands of papers being published digitally every year, it is extremely difficult to go through every single paper in order to get the desired data. The most popular electronic open-access repository of e-prints, arXiv, currently has 1,867,929 articles1. The sheer vastness of this number suggests just how important it is for us to automate the task of extracting measurement- related information from research papers (Singh et al., 2016). ∗Equal contribution. 1as of 9th April, 2021 A thorough understanding of the measurements not only requires the numerals, but also the con- text in which the quantities occur. Moreover, the entities and the properties measured along with the qualifiers that condition the measurements are cru- cial for understanding the measurement. MeasEval (Harper et al., 2021) is a semantic relation extrac- tion task focused on obtaining 9 different entities pertaining to counts, measurements and qualify- ing attributes of these quantities in a collection of excerpts from research papers in English. Figure 1 shows an example of a quantity along with its attributes and relations from this dataset. We propose a three-stage pipeline to address this task. The first stage uses a pre-trained BERT model (Devlin et al., 2019) to detect quantity spans from sentences. Receiving the detected spans as inputs, the second stage obtains the units and modifiers us- ing extracted units and modifier keywords. Finally, the third stage receives the quantity spans from the first stage and uses another pre-trained language model over each quantity-span-conditioned sen- tence to obtain quantity-span-aware contextualized representations for each sub-token in the sentence. These representations …

Content truncated to 500 words