Structured Prediction

Recent speed-ups for training large-scale models like those found in statistical NLP exploit distributed computing (either on multicore or “cloud” architectures) and rapidly converging online learning algo- rithms. Here we …...

In this paper, we consider the problem of un- supervised morphological analysis from a new angle. Past work has endeavored to design un- supervised learning methods which explicitly or implicitly …...

In this paper, we present a supervised learning approach to training submodu- lar scoring functions for extractive multi- document summarization. By taking a structured prediction approach, we pro- vide a …...

This paper deals with the problem of predict- ing structures in the context of NLP. Typically, in structured prediction, an inference proce- dure is applied to each example independently of …...

Most existing theory of structured prediction assumes exact inference, which is often in- tractable in many practical problems. This leads to the routine use of approximate infer- ence such as …...

Online learning algorithms like the percep- tron are widely used for structured predic- tion tasks. For sequential search problems, like left-to-right tagging and parsing, beam search has been successfully combined …...

Meaning Representation Jeffrey Flanigan Sam Thomson Jaime Carbonell Chris Dyer Noah A. Smith Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213, USA {jflanigan,sthomson,jgc,cdyer,nasmith}@cs.cmu.edu Abstract Abstract Meaning Representation (AMR) is …...

In this paper, we propose a new frame- work that unifies the output of three infor- mation extraction (IE) tasks - entity men- tions, relations and events as an informa- …...

AbstractExtracting instances of sentiment-oriented relations from user-generated web documents is important for online marketing analysis. Unlike previous work, we formulate this extraction task as a structured prediction problem and design …...

Hypernymy relation acquisition has been widely investigated, especially because tax- onomies, which often constitute the backbone structure of semantic resources are structured using this type of relations. Although lots of …...

We present a new algorithm for transforming dependency parse trees into phrase-structure parse trees. We cast the problem as struc- tured prediction and learn a statistical model. Our algorithm is …...

AbstractAspect-level analysis of sentiments contained in a review text is important to reveal a detailed picture of consumer opinions. While a plethora of methods have been traditionally employed for this …...

Structured prediction from bandit feed- back describes a learning scenario where instead of having access to a gold standard structure, a learner only receives partial feedback in form of the …...

Domain-independent meaning representa- tion of text has received a renewed in- terest in the NLP community. Compar- ison plays a crucial role in shaping ob- jective and subjective opinion and …...

AbstractThere has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a …...

. Structured prediction lies at the heart of modern Natural language Processing (NLP). In this paper, we study a specific family of structured learning algorithms, loosely refered to as “Learning-to-search” …...

AbstractThe best solution of structured prediction models in NLP is often inaccurate because of limited expressive power of the model or to non-exact parameter estimation. One way to mitigate this …...

AbstractBuilding an effective adversarial attacker and elaborating on countermeasures for adversarial attacks for natural language processing (NLP) have attracted a lot of research in recent years. However, most of the …...

AbstractRecently, pre-training contextualized encoders with language model (LM) objectives has been shown an effective semi-supervised method for structured prediction. In this work, we empirically explore an alternative pre-training method for …...

AbstractInterpretable rationales for model predictions play a critical role in practical applications. In this study, we develop models possessing interpretable inference process for structured prediction. Specifically, we present a method …...

AbstractInteractive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. In contrast to previous text games with mostly synthetic texts, IF …...

AbstractVarious natural language processing tasks are structured prediction problems where outputs are constructed with multiple interdependent decisions. Past work has shown that domain knowledge, framed as constraints over the output …...

AbstractIn structured prediction problems, cross-lingual transfer learning is an efficient way to train quality models for low-resource languages, and further improvement can be obtained by learning from multiple source languages. …...

AbstractKnowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge …...

AbstractWith the increase in availability of large pre-trained language models (LMs) in Natural Language Processing (NLP), it becomes critical to assess their fit for a specific target task a priori—as …...

AbstractProviding technologies to communities or domains where training data is scarce or protected e.g., for privacy reasons, is becoming increasingly important. To that end, we generalise methods for unsupervised transfer …...

AbstractIn this paper, we prove that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, that is minimizing these losses may not result in a model that …...