Dependency Enhanced Contextual Representations for Japanese Temporal Relation Classification

Dependency Enhanced Contextual Representations for Japanese Temporal Relation Classification Chenjing Geng† †Ochanomizu University geng.chenjing@is.ocha.ac.jp Fei Cheng‡ ‡Kyoto Univresity feicheng@i.kyoto-u.ac.jp Masayuki Asahara§ §NINJAL masayu-a@ninjal.ac.jp Lis Kanashiro Pereira† †Ochanomizu University kanashiro.pereira@ocha.ac.jp Ichiro Kobayashi† †Ochanomizu University koba@is.ocha.ac.jp Abstract Recently, quite a few studies have been progressive for temporal relation extraction, which is an important work used in sev- eral natural language processing applications. However, less concentration has been paid to corpora of Asian languages. In this work, we explored the feasibility of applying neural net- works to temporal relation identification in the non-English corpora, especially Japanese cor- pora, BCCWJ-TimeBank. We explored the strength of combining contextual word rep- resentations (CWR) such as BERT (Devlin et al., 2019) and shortest dependency paths (SDP) for Japanese temporal relation extrac- tion. We carefully designed a set of exper- iments to gradually reveal the improvements contributed by CWR and SDP. The empirical results suggested the following conclusions: 1) SDP offers richer information for beating the experiments with only source and target mentions. 2) CWR significantly outperforms fastText. 3) In most cases, the model applied CWR + SDP + Fine-tuning achieves the best performance overall. 1 Introduction Temporal relation extraction is the task to iden- tify temporal relationships between pairs of men- tions, namely temporal expressions, and events and is useful in various Natural Language Processing (NLP) applications, such as question answering, storytelling, text summarization, etc. Many stud- ies have so far proposed methods to extract tem- poral relation from English corpora (Pustejovsky et al., 2003a; UzZaman et al., 2012; Cheng and Miyao, 2017; Dligach et al., 2017; Tourille et al., 2017; Lin et al., 2017; Lin et al., 2018; Lin et al., 2019; Lin et al., 2020), however, unlike En- glish, there are few temporal relation identification studies in Asian languages, due to limited data re- sources. On the other hand, Asahara et al. (2013) proposed the first Japanese temporal information corpus, BCCWJ-TimeBank (see, section 3 for more detail). Yoshikawa et al. (2014) used this Japanese corpus for temporal relation identification, however, there is still much space to explore for the temporal relation identification in Japanese. We, therefore, at- tempt to tackle it with the Japanese corpora in this study. Temporal relation extraction is a kind of study of relation extraction. Bunescu and Mooney (2005) showed that there is a shortest path between the two entities in the dependency structure of a sentence, called shortest dependency path (SDP), which works well to extract the relation between the two entities. Xu et al. (2015) applied SDP in the BiLSTM frame- work for relation extraction and got the state-of-the- art results of those days. The SDP was also used in the deep learning framework for temporal rela- tion extraction (Cheng and Miyao, 2017; Cheng and Miyao, 2018; Jiang et al., 2019; Li et al., 2019) and provided good results. Recently, contextual word embedding models, such as ELMo (Peters et al., 2018) and BERT (De- vlin et al., 2019) have been effectively used as con- textual word representations. Differently from …

Content truncated to 500 words