High-order Refining for End-to-end Chinese Semantic Role Labeling

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 100–105 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 100 High-order Refining for End-to-end Chinese Semantic Role Labeling Hao Fei1, Yafeng Ren2 and Donghong Ji1∗ 1. Department of Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, China 2. Guangdong University of Foreign Studies, China {hao.fei,renyafeng,dhji}@whu.edu.cn Abstract Current end-to-end semantic role labeling is mostly accomplished via graph-based neural models. However, these all are first-order models, where each decision for detecting any predicate-argument pair is made in isolation with local features. In this paper, we present a high-order refining mechanism to perform in- teraction between all predicate-argument pairs. Based on the baseline graph model, our high- order refining module learns higher-order fea- tures between all candidate pairs via atten- tion calculation, which are later used to up- date the original token representations. After several iterations of refinement, the underly- ing token representations can be enriched with globally interacted features. Our high-order model achieves state-of-the-art results on Chi- nese SRL data, including CoNLL09 and Uni- versal Proposition Bank, meanwhile relieving the long-range dependency issues. 1 Introduction Semantic role labeling (SRL), as the shallow se- mantic parsing aiming to detect the semantic predi- cates and their argument roles in texts, plays a core role in natural language processing (NLP) commu- nity (Pradhan et al., 2005; Zhao et al., 2009; Lei et al., 2015; Xia et al., 2019b). SRL is traditionally handled by two pipeline steps: predicate identifica- tion (Scheible, 2010) and argument role labeling (Pradhan et al., 2005). More recently, growing interests are paid for developing end-to-end SRL, achieving both two subtasks, i.e., recognizing all possible predicates together with their arguments jointly, via one single model (He et al., 2018a). The end-to-end joint architecture can greatly al- leviate the error propagation problem, thus helping to achieve better task performance. Currently, the end-to-end SRL methods largely are graph-based ∗Corresponding author. neural models, enumerating all possible predicates and their arguments exhaustively (He et al., 2018a; Cai et al., 2018; Li et al., 2019). However, these first-order models that only consider one predicate- argument pair at a time can be limited to short-term features and local decisions, thus being subjective to long-range dependency issues existing at large surface distances between arguments (Chen et al., 2019; Lyu et al., 2019). This makes it imperative to capture the global interactions between multiple predicates and arguments. In this paper, based on the graph-based model architecture, we propose to further learn the higher- order interaction between all predicate-argument pairs by performing iterative refining for the un- derlying token representations. Figure 1 illustrates the overall framework of our method. The BiL- STM encoder (Hochreiter and Schmidhuber, 1997) first encodes the inputs into the initial token rep- resentations for producing predicate and argument representations, respectively. The biaffine attention then exhaustively calculates the score …

Content truncated to 500 words