LIORI at SemEval-2021 Task 8: Ask Transformer for measurements

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1249–1254 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1249 LIORI at SemEval-2021 Task 8: Ask Transformer for measurements Adis Davletov RANEPA, Moscow, Russia Lomonosov Moscow State University, Moscow, Russia davletov-aa@ranepa.ru Denis Gordeev RANEPA, Moscow, Russia gordeev-di@ranepa.ru Nikolay Arefyev Lomonosov Moscow State University, Moscow, Russia Samsung Research Center Russia, Moscow, Russia HSE University, Moscow, Russia nick.arefyev@gmail.com Emil Davletov Katanov Khakas State University, Abakan, Russia edavletov@yandex.ru Abstract This work describes our approach for subtasks of SemEval-2021 Task 8: MeasEval: Counts and Measurements which took the official first place in the competition. To solve all sub- tasks we use multi-task learning in a question- answering-like manner. We also use learnable scalar weights to weight subtasks’ contribution to the final loss in multi-task training. We fine- tune LUKE to extract quantity spans and we fine-tune RoBERTa to extract everything re- lated to found quantities, including quantities themselves. 1 Introduction SemEval-2021 Task 8 consisted of five subtasks that covered span extraction, classification, and relation extraction tasks. This paper presents so- lutions to all five of them which showed the best results in the competition1. In the subtask 1(A) participants were asked to re- trieve Quantity (Q) spans from texts. For example, in the following text ”The soda can’s volume was 355 ml.”, the system should retrieve ”355 ml” as Q span. The rest of the subtasks were to extract in- formation related to retrieved Quantities (Qs) from subtask A. The subtask 2(B) was to extract the Unit of mea- surement (UoM) of the extracted Q and also to classify it into 10 classes: HasTolerance, IsAp- proximate, IsCount, IsList, IsMean, IsMeanHas- Tolerance, IsMeanIsRange, IsMedian, IsRange, Is- RangeHasTolerance. It should be noted that some 1https://github.com/davletov-aa/meas-eval Qs could be related to more than one type and there were ones which didn’t belong to any type. The subtask 3(C) was to extract Measured Entity (ME) and Measured Property (MP) spans. In the sub- task 4(D) additional Qualifier (Qlfr) spans, which helped to validate or understand the extracted Q, were asked to be extracted. And finally, sub- task 5(E) was to extract relations between Qs, MEs, MPs and Qlfrs. More detailed information about the competition could be found in the Harper et al. (2021)’s shared task description paper. 2 Related Work Span extraction and classification problems have a long history of studies and are often studied as a part of Named Entity Recognition (NER). For ex- ample, the NER dataset Ontonotes v5 (Weischedel et al., 2013) contains such entities as ”Quantity”, which also includes measurements, and ”Money”. However, the general NER approach used in Ontonotes or ConLL 2003 (Sang and De Meul- der, 2003) datasets is not so fine-grained as the one that is used in the task under study. Most state-of-the-art models for named en- tity recognition and relation extraction are based on Transformer architecture by Vaswani et al. (2017). For example, the top three best models for Ontonotes v5 according to paperswithcode.com use …

Content truncated to 500 words