Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1255–1263 May 2-6, 2023 ©2023 Association for Computational Linguistics Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding Janvijay Singh♣ Fan Bai♣ Zhen Wang♢ ♣ School of Interactive Computing, Georgia Institute of Technology ♢ Department of Computer Science and Engineering, The Ohio State University iamjanvijay@gatech.edu fan.bai@cc.gatech.edu wang.9215@osu.edu Abstract Cross-task knowledge transfer via multi-task learning has recently made remarkable progress in general NLP tasks. However, entity track- ing on the procedural text has not benefited from such knowledge transfer because of its dis- tinct formulation, i.e., tracking the event flow while following structural constraints. State- of-the-art entity tracking approaches either de- sign complicated model architectures or rely on task-specific pre-training to achieve good results. To this end, we propose MEET, a Multi-task learning-enabled entity Tracking ap- proach, which utilizes knowledge gained from general domain tasks to improve entity tracking. Specifically, MEET first fine-tunes T5, a pre- trained multi-task learning model, with entity tracking-specialized QA formats, and then em- ploys our customized decoding strategy to sat- isfy the structural constraints. MEET achieves state-of-the-art performances on two popular entity tracking datasets, even though it does not require any task-specific architecture design or pre-training.1 1 Introduction Pre-trained language models have revolutionized the NLP field in recent years (Devlin et al., 2019; Liu et al., 2019; Brown et al., 2020) and also become more versatile with the novel encoder- decoder architecture (Raffel et al., 2020; Lewis et al., 2020), which allows them to handle differ- ent types of NLP tasks without further architec- tural changes. This versatility inherently facilitates cross-task knowledge transfer via multi-task learn- ing (Raffel et al., 2020; Aribandi et al., 2022), and thus helps push the boundary of many popular NLP tasks such as question answering (Khashabi et al., 2020) and semantic parsing (Xie et al., 2022). How- ever, entity tracking, which tracks the states and locations of an entity throughout the procedural 1Our code and data are available at https://github. com/iamjanvijay/MeeT. Procedural text: How is hydroelectricity generated? 1. Water flows downwards thanks to gravity. 2. Enters the dam at high pressure. 3. Moving water spins the turbines in power plant. 4. … Entity Tracking Input Query Entity: Water Water step 1 exist step 2 move step 3 move … … State Prediction Location Prediction Entity Tracking Output Water step 1 unknown step 2 dam step 3 turbine … … Mention-guided CRF decoding State Prediction T5 step 3 step 2 step 1 exist move move exist destroy move unknown dam turbine Location Prediction T5 step 3 step 2 step 1 MeeT Figure 1: Overview of MEET (Multi-task learning- enabled entity Tracking). MEET utilizes the multi-task learning in T5 to boost entity tracking performance, with a customized decoding strategy addressing the structural constraints in state prediction (e.g., "move" cannot hap- pen after "destroy"). text, like scientific processes or recipes, has not been impacted by this multi-task learning wave for two main reasons. First, entity …

Content truncated to 500 words