Fast End-to-end Coreference Resolution for Korean

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2610–2624 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2610 Fast End-to-end Coreference Resolution for Korean Cheoneum Park1, Jamin Shin2, Sungjoon Park3,4, Joonho Lim5, Changki Lee6 1AIRS Company, Hyundai Motor Group, Republic of Korea 2Riiid AI Research, Republic of Korea 3Korea Advanced Institute of Science and Technology, Republic of Korea 4Upstage AI Research, Republic of Korea 5Electronics and Telecommunications Research Institute, Republic of Korea 6Kangwon National University, Republic of Korea cheoneum.park@hyundai.com, jshin49@gmail.com, sungjoon.park@kaist.ac.kr, joonho.lim@etri.re.kr, leeck@kangwon.ac.kr Abstract Recently, end-to-end neural network-based approaches have shown significant improve- ments over traditional pipeline-based models in English coreference resolution. However, such advancements came at a cost of com- putational complexity and recent works have not focused on tackling this problem. Hence, in this paper, to cope with this issue, we pro- pose BERT-SRU-based Pointer Networks that leverages the linguistic property of head-final languages. Applying this model to the Korean coreference resolution, we significantly reduce the coreference linking search space. Combin- ing this with Ensemble Knowledge Distilla- tion, we maintain state-of-the-art performance 66.9% of CoNLL F1 on ETRI test set while achieving 2x speedup (30 doc/sec) in docu- ment processing time. 1 Introduction Coreference resolution is one of the fundamen- tal sub-tasks for Machine Reading Comprehen- sion and Dialogue Systems that groups mentions of a same entity in a given sentence or docu- ment (Soon et al., 2001; Raghunathan et al., 2010; Ng, 2010; Lee et al., 2013). Recently, for En- glish coreference resolution, span-based end-to- end trained models such as e2e-coref (Lee et al., 2017), c2f-coref (Lee et al., 2018), and BERT-coref (Joshi et al., 2019b) have shown to outperform previous rule-based or mention-pairing approaches. However, such approaches suffer from the com- putational complexity effectively-being O(n4), where n is the length of the input document. Fur- thermore, as coreference resolution is a very im- portant and complicated task, most of the research efforts have been focused on how to solve the prob- lem through better modeling, such as higher-order coreference resolution (Lee et al., 2018). Inevitably, [훈민정음을 창제한 세종]은 [조선의 4대 왕]이다. [Sejong, who created the Hunminjeongeum,] is [Joseon’s fourth king.] Korean: head-final language English: mixed head-directional language Figure 1: The brackets are mention boundaries, bold- faced words are the nouns, and underlined words are the heads of each mention. The red line is a dependency relation arc and Korean (top) shows the left-branching property (Dryer, 2009) where the heads are always at the end of the mention. On the other hand, the head locations for English is different across mentions. these approaches lead to more complicated mod- els that are more computation heavy, but there are not many studies on solving this complexity issue. Hence, this paper aims to cope with this problem by infusing relevant linguistic features into the model. One of the underlying reasons for such high com- putational complexity was the creation of O(n2) spans caused by the mixed head directionality of English, as shown in Figure 1. This …

Content truncated to 500 words