Improving Event Duration Prediction via Time-aware Pre-training

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3370–3378 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3370 Improving Event Duration Prediction via Time-aware Pre-training Zonglin Yang Xinya Du Alexander Rush Claire Cardie Department of Computer Science Cornell University {zy223, arush}@cornell.edu {xdu, cardie}@cs.cornell.edu Abstract End-to-end models in NLP rarely encode ex- ternal world knowledge about length of time. We introduce two effective models for du- ration prediction, which incorporate external knowledge by reading temporal-related news sentences (time-aware pre-training). Specifi- cally, one model predicts the range/unit where the duration value falls in (R-PRED); and the other predicts the exact duration value (E- PRED). Our best model – E-PRED, substan- tially outperforms previous work, and captures duration information more accurately than R- PRED. We also demonstrate our models are capable of duration prediction in the unsuper- vised setting, outperforming the baselines. 1 Introduction Understanding duration of event expressed in text is a crucial task in NLP (Pustejovsky and Verhagen, 2009; Zhou et al., 2019). It facilitates downstream tasks such as story timeline construction (Ning et al., 2018; Leeuwenberg and Moens, 2019) and temporal question answering (Llorens et al., 2015). It is challenging to make accurate prediction mainly due to two reasons: (1) duration is not only asso- ciated with event word but also the context. For example, “watch a movie” takes around 2 hours, while “watch a bird fly” only takes about 10 sec- onds; (2) the compositional nature of events makes it difficult to train a learning-based system only based on hand annotated data (since it’s hard to cover all the possible events). Thus, external knowl- edge and commonsense are needed to make further progress on the task. However, most current approaches (Pan et al., 2011; Gusev et al., 2011; Vempala et al., 2018) focus on developing features and cannot utilize ex- ternal textual knowledge. The only exception is the web count based method proposed by Gusev et al. (2011), which queries search engine with event word (e.g., “watch”) and temporal units, and make predictions based on hitting times. However, this method achieves better performance when query only with the event word in the sentence, which means it does not enable contextualized understand- ing. To benefit from the generalizability of learning- based methods and utilizing external temporal knowledge, we introduce a framework, which in- cludes (1) a procedure for collecting duration- related news sentences, and automatic labeling the duration unit in it (Section 2.1); 1 (2) two effective end-to-end models that leverage external temporal knowledge via pre-training (Section 2.2). Specifi- cally, our first model (R-PRED) predicts the most likely temporal unit/range for the event, with a classification output layer; and the other model (E- PRED) predicts the exact duration value, with a regression output layer. Our best model (E-PRED) achieves state-of-the-art performance on the Time- Bank dataset and the McTACO duration prediction task. In addition, in the unsupervised setting, our model (E-PRED) trained with only collected web data outperforms the supervised BERT baseline by 10.24 F1 score …

Content truncated to 500 words