An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution

Proceedings of the 28th International Conference on Computational Linguistics, pages 4956–4968 Barcelona, Spain (Online), December 8-13, 2020 4956 An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution Ryuto Konno ♠ Yuichiroh Matsubayashi ♠♦ Shun Kiyono ♦♠ Hiroki Ouchi ♦ Ryo Takahashi ♠♦ Kentaro Inui ♠♦ ♠Tohoku University ♦RIKEN {ryuto, ryo.t, inui}@ecei.tohoku.ac.jp y.m@tohoku.ac.jp {shun.kiyono, hiroki.ouchi}@riken.jp Abstract One critical issue of zero anaphora resolution (ZAR) is the scarcity of labeled data. This study explores how effectively this problem can be alleviated by data augmentation. We adopt a state-of- the-art data augmentation method, called the contextual data augmentation (CDA), that generates labeled training instances using a pretrained language model. The CDA has been reported to work well for several other natural language processing tasks, including text classification and machine translation (Kobayashi, 2018; Wu et al., 2019; Gao et al., 2019). This study addresses two underexplored issues on CDA, that is, how to reduce the computational cost of data augmentation and how to ensure the quality of the generated data. We also propose two methods to adapt CDA to ZAR: [MASK]-based augmentation and linguistically-controlled masking. Consequently, the experimental results on Japanese ZAR show that our methods contribute to both the accuracy gain and the computation cost reduction. Our closer analysis reveals that the proposed method can improve the quality of the augmented training data when compared to the conventional CDA. 1 Introduction In pro-drop languages, such as Japanese and Chinese, the arguments of a predicate are frequently omitted in sentences. Automatic recognition of such omitted arguments plays an important role in understanding the predicate-argument structure of a sentence. This task is referred to as zero anaphora resolution (ZAR). Figure 1a depicts an example of the task. In the sentence, the nominative argument of the predicate was acquitted is omitted. Such an omitted argument is referred to as a zero pronoun often represented as φ. In Figure 1a, φ refers to the man. One critical issue of ZAR is the scarcity of labeled data. To compensate for the scarcity, several studies have explored the direction of exploiting unlabeled data (Sasano et al., 2008; Sasano and Kurohashi, 2011; Chen and Ng, 2014; Chen and Ng, 2015; Liu et al., 2017; Yamashiro et al., 2018; Kurita et al., 2018). Another promising direction is to automatically augment labeled data (data augmentation). One state-of-the-art data augmentation method is contextual data augmentation (CDA), which augments labeled data by replacing an arbitrary token(s) with another token(s) predicted to be plausible in the surrounding context by a pretrained language model (LM). CDA works well in several natural language processing (NLP) tasks, such as text classification (Kobayashi, 2018; Wu et al., 2019) and machine translation (Gao et al., 2019). However, unlike the direction of exploiting unlabeled data, the potential of data augmentation in the context of ZAR has largely remained underexplored. This study investigates how the idea of CDA can be effectively employed in ZAR, taking the task of Japanese ZAR as a case study. Figure 1 illustrates a straightforward …

Content truncated to 500 words