Cost-sensitive Regularization for Label Confusion-aware Event Detection

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5278–5283 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5278 Cost-sensitive Regularization for Label Confusion-aware Event Detection Hongyu Lin1,3, Yaojie Lu1,3, Xianpei Han1,2,∗, Le Sun1,2 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China {hongyu2016,yaojie2017,xianpei,sunle}@iscas.ac.cn Abstract In supervised event detection, most of the mis- labeling occurs between a small number of confusing type pairs, including trigger-NIL pairs and sibling sub-types of the same coarse type. To address this label confusion problem, this paper proposes cost-sensitive regulariza- tion, which can force the training procedure to concentrate more on optimizing confusing type pairs. Specifically, we introduce a cost- weighted term into the training loss, which pe- nalizes more on mislabeling between confus- ing label pairs. Furthermore, we also propose two estimators which can effectively measure such label confusion based on instance-level or population-level statistics. Experiments on TAC-KBP 2017 datasets demonstrate that the proposed method can significantly improve the performances of different models in both En- glish and Chinese event detection. 1 Introduction Automatic event extraction is a fundamental task in information extraction. Event detection, aim- ing to identify trigger words of specific types of events, is a vital step of event extraction. For ex- ample, from sentence “Mary was injured, and then she died”, an event detection system is required to detect a Life:Injure event triggered by “injured” and a Life:Die event triggered by “died”. Recently, neural network-based supervised models have achieved promising progress in even- t detection (Nguyen and Grishman, 2015; Chen et al., 2015; Ghaeini et al., 2016). Commonly, these methods regard event detection as a word- wise classification task with one NIL class for to- kens do not trigger any event. Specifically, a neu- ral network automatically extracts high-level fea- tures and then feed them into a classifier to cate- gorize words into their corresponding event sub- ∗Corresponding author. BC CT CR MT NIL CC BC 41.3 14.4 2.1 1.6 39.0 1.7 CT 8.5 42.7 4.7 2.6 40.6 0.9 CR 5.7 7.3 50.0 1.1 32.3 2.9 MT 3.0 7.7 6.1 28.7 51.3 3.2 Table 1: Prediction percentage heatmap of triggers with Contact coarse type. Row labels are the golden la- bel and the column labels indicate the prediction. BC: Broadcast; CT: Conctact(sub-type); CR: Correspon- dence; MT: Meet; CC: Other cross coarse-type errors. types (or NIL). Optimization criteria of such mod- els often involves in minimizing cross-entropy loss, which equals to maximize the likelihood of making correct predictions on the training data. However, we find that in supervised even- t detection, most of the mislabeling occurs be- tween a small number of confusing type pairs. We refer to this phenomenon as label confusion. Specifically, there are mainly two types of la- bel confusion in event detection: 1) trigger/NIL confusion; 2) sibling sub-types confusion. For example, both Transaction:Transfer-money and Transaction:Transfer-ownership events are fre- quently triggered …

Content truncated to 500 words