Multilingual Normalization of Temporal Expressions with Masked Language Models

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1174–1186 May 2-6, 2023 ©2023 Association for Computational Linguistics Multilingual Normalization of Temporal Expressions with Masked Language Models Lukas Lange1 Jannik Strötgen1 1 Bosch Center for Artificial Intelligence, Renningen, Germany 2 Spoken Language Systems (LSV), Saarland University, Saarbrücken, Germany {Lukas.Lange,Jannik.Stroetgen,Heike.Adel}@de.bosch.com dietrich.klakow@lsv.uni-saarland.de Heike Adel1 Dietrich Klakow2 Abstract The detection and normalization of temporal expressions is an important task and prepro- cessing step for many applications. However, prior work on normalization is rule-based, which severely limits the applicability in real- world multilingual settings, due to the costly creation of new rules. We propose a novel neural method for normalizing temporal ex- pressions based on masked language model- ing. Our multilingual method outperforms prior rule-based systems in many languages, and in particular, for low-resource languages with performance improvements of up to 33 F1 on average compared to the state of the art. 1 Introduction Temporal tagging consists of the extraction of temporal expressions (TE) from texts and their normalization to a standard format (e.g., May ’22: 2022-05). While there are deep-learning ap- proaches for the extraction, temporal tagging as a whole is usually solved with highly specific rule- based systems, such as SUTime (Chang and Man- ning, 2012) or HeidelTime (Strötgen and Gertz, 2013). However, transferring rule-based methods to new languages or text domains requires a large manual effort to create rules specific to the target language. Although work on the automatic gen- eration of rules for many languages (Strötgen and Gertz, 2015) exists, the rule quality typically does not match the high accuracy of hand-crafted rules. In contrast to rule-based systems, neural net- works are known for their ability to generalize to new targets, in particular, for cross- and multilin- gual applications (Rahimi et al., 2019; Artetxe and Schwenk, 2019). In the context of temporal tag- ging, recent works have shown promising results of neural networks for TE extraction in monolin- gual (Laparra et al., 2018) and multilingual settings (Lange et al., 2020) where a single neural model is trained on many languages at once. However, TE ... appear in <TIMEX3 type="DATE" value=MASK>May</TIMEX3>. CIR: UNDEF-year-05 Slots: ... The Eta Auariids meteor shower will appear in May . (1) Extraction (2) Normalization to CIR UNDEF- year 05 PAD PAD (3) Anchoring Anchor: 2022-03-15 FUTURE 2022-05 PAD with MLM model Figure 1: Overview of our 3-step pipeline for tempo- ral tagging consisting of extraction of temporal expres- sions (step 1), normalization to a context-independent representation (CIR) using a slot-based masked lan- guage model (step 2) and anchoring given a reference time and further contextual cues (step 3). normalization remains challenging, and no solution for the normalization across languages exists yet. We propose a new multilingual normalization method which can make use of labeled data from many languages by training a neural transformer model with a masked language modeling (MLM) objective. Thus, we adopt the MLM objective func- tion for a new purpose: TE normalization. To the best of …

Content truncated to 500 words