BERT-based Cohesion Analysis of Japanese Texts

Proceedings of the 28th International Conference on Computational Linguistics, pages 1323–1333 Barcelona, Spain (Online), December 8-13, 2020 1323 BERT-based Cohesion Analysis of Japanese Texts Nobuhiro Ueda Kyoto University ueda@nlp.ist.i.kyoto-u.ac.jp Daisuke Kawahara Waseda University dkw@waseda.jp Sadao Kurohashi Kyoto University, CREST kuro@nlp.ist.i.kyoto-u.ac.jp Abstract The meaning of natural language text is supported by cohesion among various kinds of entities, including coreference relations, predicate-argument structures, and bridging anaphora relations. However, predicate-argument structures for nominal predicates and bridging anaphora relations have not been studied well, and their analyses have been still very difficult. Recent advances in neural networks, in particular, self training-based language models including BERT (Devlin et al., 2019), have significantly improved many natural language processing tasks, making it possible to dive into the study on analysis of cohesion in the whole text. In this study, we tackle an integrated analysis of cohesion in Japanese texts. Our results significantly outperformed existing studies in each task, especially about 10 to 20 point improvement both for zero anaphora and coreference resolution. Furthermore, we also showed that coreference resolution is different in nature from the other tasks and should be treated specially. 1 Introduction The meaning of natural language text is supported by cohesion among various kinds of entities. Such cohesion includes coreference relations, predicate-argument structures, and bridging anaphora relations. For example, there are various relations between entities in the Japanese text (two sentences) of Figure 1. Clarifying these relations is indispensable for computers to understand natural language texts. Among these relations, coreference relations and predicate-argument structures for verbs have been actively studied. However, predicate-argument structures for nominal predicates and bridging anaphora relations have not been studied well, and their analyses have been still very difficult. Recent advances in neural networks, in particular self training-based language models including BERT (Devlin et al., 2019), have significantly improved many natural language processing (NLP) tasks. By using these techniques, it is now possible to dive into the study on analysis of cohesion in the whole text, including nominal predicate-argument structures and bridging anaphora relations. In this study, we tackle an integrated analysis of cohesion in Japanese texts. To the best of our knowledge, no study has focused on such an integrated analysis of text cohesion. First, we explain the characteristics of each relation in Japanese texts. A coreference relation is a relation between nouns that refer to the same real-world entity, and the task of revealing this relation is called coreference resolution (CR). In Figure 1, “author” and “he” refer to the same entity. There is no Japanese specific phenomenon in CR. A predicate-argument structure consists of a predicate and its arguments that fill each case of the predicate, such as who does/did what to whom. The task of clarifying these relations is called predicate- argument structure analysis (PAS analysis). In Japanese, an argument is usually labeled by a case maker This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 1324 Figure 1: An example of Japanese semantic relations. Its English translation is …

Content truncated to 500 words