PTST-UoM at SemEval-2021 Task 10: Parsimonious Transfer for Sequence Tagging

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 445–451 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 445 PTST-UoM at SemEval-2021 Task 10: Parsimonious Transfer for Sequence Tagging Kemal Kurniawan1 Lea Frermann1 Philip Schulz2∗ Trevor Cohn1 1School of Computing and Information Systems, University of Melbourne 2Amazon Research kemal.kurniawan@student.unimelb.edu.au {lea.frermann,tcohn}@unimelb.edu.au phschulz@amazon.com Abstract This paper describes PTST, a source-free un- supervised domain adaptation technique for sequence tagging, and its application to the SemEval-2021 Task 10 on time expression recognition. PTST is an extension of the cross- lingual parsimonious parser transfer frame- work (Kurniawan et al., 2021), which uses high-probability predictions of the source model as a supervision signal in self-training. We extend the framework to a sequence pre- diction setting, and demonstrate its applicabil- ity to unsupervised domain adaptation. PTST achieves F1 score of 79.6 % on the official test set, with the precision of 90.1 %, the highest out of 14 submissions.1 1 Introduction SemEval-2021 Task 10 presents source-free unsu- pervised domain adaptation (SFUDA) for semantic processing.2 The goal of unsupervised domain adaptation is to transfer a model from a source domain to another, different domain — called tar- get domain — using only unlabelled data in the target domain. Source-free unsupervised domain adaptation additionally assumes no access to source domain data: only the source model pre-trained on the source domain data is available. This situation may occur when the source domain data contains protected information that cannot be shared, or even if it can, requires signing a complex data use agreement. While there are numerous works on SFUDA outside NLP (Hou and Zheng, 2020; Kim et al., 2020; Liang et al., 2020; Yang et al., 2020), SFUDA research for NLP is severely lacking in spite of its importance in, for example, clinical ∗Work done outside Amazon. 1Our code is available at https://github.com/ kmkurn/ptst-semeval2021. 2https://competitions.codalab.org/ competitions/26152 NLP (Laparra et al., 2020). There are two tasks involved in SemEval-2021 Task 10: negation de- tection and time expression recognition. We partic- ipate only in the latter. Our approach is an extension of the parsimo- nious parser transer framework (PPT; Kurniawan et al. (2021)). PPT allows cross-lingual trans- fer of dependency parsers in a source-free man- ner, requiring only unlabelled data in the target side. It leverages the output distribution of the source model to build a chart containing high prob- ability trees for each sentence in the target data. We extend this work by (1) formulating PPT for chain structures and evaluating it on a semantic sequence tagging task; and (2) demonstrating its ef- fectiveness in a domain adaptation setting. We call our method Parsimonious Transfer for Sequence Tagging (PTST). We find PTST effective for improving the preci- sion of the system in the target domain. It ranks 7th out of 14 submissions in the official leaderboard in terms of F1 score, but 1st in precision, with a gap of 3 points from the second best. Drawing on the model calibration literature, we provide a way to …

Content truncated to 500 words