Randomized Deep Structured Prediction for Discourse-Level Processing

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1174–1184 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1174 Randomized Deep Structured Prediction for Discourse-Level Processing Manuel Widmoser∗ 1 Maria Leonor Pacheco∗ 2 Jean Honorio2 Dan Goldwasser2 1University of Salzburg, Austria 2Purdue University, USA mwidmoser@cs.uni-salzburg.at, {pachecog, jhonorio, dgoldwas}@purdue.edu Abstract Expressive text encoders such as RNNs and Transformer Networks have been at the cen- ter of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. How- ever, certain tasks, such as argumentation min- ing, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a gen- eral framework to combine the complemen- tary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to al- leviate this concern and show that we can effi- ciently leverage deep structured prediction and expressive neural encoders for a set of tasks in- volving complicated argumentative structures. 1 Introduction Many discourse-level NLP tasks require modeling complex interactions between multiple sentences, paragraphs or even documents. For example, ana- lyzing opinions in online conversations (Hasan and Ng, 2013; Sridhar et al., 2015) requires modeling the dependencies between the opinions in individ- ual posts, the disagreement between posts in long conversational threads and the overall view of users, given all their posts. Learning in these settings is extremely challeng- ing. It requires highly expressive models that can capture the claims made in each document, either by using a rich, manually crafted feature set, or by ∗Contributed equally to this work as first authors using neural architectures to learn an expressive representation (Ji and Eisenstein, 2014; Niculae et al., 2017). In addition, reasoning about the in- teraction between these decisions is often compu- tationally challenging, as it requires incorporating domain-specific constraints into the search proce- dure, making exact inference intractable. As a re- sult, most current work relies on highly engineered solutions, which are difficult to adapt. Instead of training structured predictors that model the in- teraction between decisions during training, they combine locally trained classifiers at test time (Stab and Gurevych, 2017). Our goal in this paper is to study realistic set- tings, in which discourse-level problems can be learned efficiently when leveraging deep structured prediction, a framework for combining rich neural representation with an inference-layer, forcing con- sistency between them (Zheng et al., 2015). These models were applied successfully to simpler NLP tagging tasks (Lample et al., 2016), in which infer- ence is tractable. However, as shown in a recent argumentation mining work (Niculae et al., 2017), their applicability to more complex learning tasks is …

Content truncated to 500 words