Using a Penalty-based Loss Re-estimation Method to Improve Implicit Discourse Relation Classification

Proceedings of the 28th International Conference on Computational Linguistics, pages 1513–1518 Barcelona, Spain (Online), December 8-13, 2020 1513 Using a Penalty-based Loss Re-estimation Method to Improve Implicit Discourse Relation Classification Xiao Li† Yu Hong†∗ Huibin Ruan† Zhen Huang‡ † School of Computer Science and Technology, Soochow University, 1 Shizi, Suzhou, CHN ‡ School of Computer Science, National University of Defense Technology, Changsha, CHN †{emilyxiao0512, tianxianer, hbr416}@gmail.com; ‡huangzhen@nudt.edu.cn Abstract We tackle implicit discourse relation classification, a task of automatically determining semantic relationships between arguments. The attention-worthy words in arguments are crucial clues for classifying the discourse relations. Attention mechanisms have been proven effective in high- lighting the attention-worthy words during encoding. However, our survey shows that some inessential words are unintentionally misjudged as the attention-worthy words and, therefore, as- signed heavier attention weights than should be. We propose a penalty-based loss re-estimation method to regulate the attention learning process, integrating penalty coefficients into the compu- tation of loss by means of overstability of attention weight distributions. We conduct experiments on the Penn Discourse TreeBank (PDTB) corpus. The test results show that our loss re-estimation method leads to substantial improvements for a variety of attention mechanisms. 1 Introduction The goal of pairwise sentence-level discourse analysis is to determine the relation that is held by a pair of arguments (Prasad et al., 2008), where an argument generally stands for a narrative sentence. Implicit discourse relation classification is a challenging subtask. It is required to determine the relation on the condition that the explicit connective (i.e., a syntactic conjunction) is not given. For example, the arguments in Figure 1 are a pair of semantically-related arguments, where the possible connective “but” that may signal the comparison relation has been omitted. Argument 1 (Arg1): Robert S. Enrlich resigned as chairman, president and chief executive. Argument 2 (Arg2): Mr. Enrlich will continue as a director and a consultant. Ground-truth relation: Comparison (sub-relation—concession) Omitted connective: but 0.00E+00 5.00E-03 1.00E-02 1.50E-02 Robert S . Enrlich resigned as chairman , president and chief executive 0.00E+00 5.00E-03 1.00E-02 1.50E-02 Mr . Enrlich will continue as a director and a consultant 0.00E+00 5.00E-02 1.00E-01 1.50E-01 Robert S . Enrlich resigned as chairman , president and chief executive 0.00E+00 3.00E-02 6.00E-02 9.00E-02 Mr . Enrlich will continue as a director and a consultant Attention distribution for Arg1 (existing attention mechanism) Attention distribution for Arg2 (existing attention mechanism) Attention distribution for Arg1 (after penalty) Attention distribution for Arg2 (after penalty) Figure 1: Example of arguments, connective and relationship, as well as attention weight distributions Detecting the content words that imply correlations between arguments contributes to the relation determination (Marcu and Echihabi, 2002). We refer such kind of content words to attention-worthy words, such as the words shown in bold in Figure 1. The current attention mechanisms have been proven effective in recognizing and utilizing attention-worthy words. They generally assign heavier weights to attention-worthy words conditioned on either internal (Lin et al., 2017) or external context ∗ Corresponding author: Yu Hong (tianxianer@gmail.com) This work is …

Content truncated to 500 words