High-Order Low-Rank Tensors for Semantic Role Labeling

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1150–1160, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics High-Order Low-Rank Tensors for Semantic Role Labeling Tao Lei1, Yuan Zhang1, Llu´ıs M`arquez2, Alessandro Moschitti2, and Regina Barzilay1 1Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology 2ALT Research Group, Qatar Computing Research Institute 1{taolei, yuanzh, regina}@csail.mit.edu 2{lmarquez, amoschitti}@qf.org.qa Abstract This paper introduces a tensor-based approach to semantic role labeling (SRL). The motiva- tion behind the approach is to automatically induce a compact feature representation for words and their relations, tailoring them to the task. In this sense, our dimensionality reduction method provides a clear alternative to the traditional feature engineering approach used in SRL. To capture meaningful interac- tions between the argument, predicate, their syntactic path and the corresponding role la- bel, we compress each feature representation first to a lower dimensional space prior to as- sessing their interactions. This corresponds to using an overall cross-product feature repre- sentation and maintaining associated parame- ters as a four-way low-rank tensor. The tensor parameters are optimized for the SRL perfor- mance using standard online algorithms. Our tensor-based approach rivals the best perform- ing system on the CoNLL-2009 shared task. In addition, we demonstrate that adding the representation tensor to a competitive tensor- free model yields 2% absolute increase in F- score.1 1 Introduction The accuracy of Semantic Role Labeling (SRL) sys- tems depends strongly on the features used by the underlying classifiers. For instance, the top perform- ing system on the CoNLL–2009 shared task em- ploys over 50 language-specific templates for fea- ture generation (Che et al., 2009). The templates 1Our code is available at https://github.com/ taolei87/SRLParser. are manually created and thus offer specific means of incorporating prior knowledge into the method. However, finding compact, informative templates is difficult since the relevant signal may be spread over many correlated features. Moreover, the use of lex- icalized features, which are inevitably sparse, leads to overfitting. In this case it is advantageous to try to automatically compress the feature set to use a small number of underlying co-varying dimensions. Dimensionality reduction of this kind can be incor- porated into the classifier directly by utilizing tensor calculus. In this paper, we adopt this strategy. We start by building high-dimensional feature vectors that are subsequently mapped into a low- dimensional representation. Since this high- dimensional representation has to reflect the inter- action between different indicators of semantic rela- tions, we construct it as a cross-product of smaller feature vectors that capture distinct facets of seman- tic dependence: predicate, argument, syntactic path and role label. By compressing this sparse repre- sentation into lower dimensions, we obtain dense representations for words (predicate, argument) and their connecting paths, uncovering meaningful inter- actions. The associated parameters are maintained as a four-way low-rank tensor, and optimized for SRL performance. Tensor modularity enables us to employ standard online algorithms for training. Our approach to SRL is inspired by recent suc- …

Content truncated to 500 words