Better call Saul: Flexible Programming for Learning and Inference in NLP

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3030–3040, Osaka, Japan, December 11-17 2016. Better call Saul: Flexible Programming for Learning and Inference in NLP Parisa Kordjamshidi∗† Daniel Khashabi‡ Christos Christodoulopoulos‡ Bhargav Mangipudi‡ Sameer Singh§ Dan Roth‡ † Tulane University ‡ University of Illinois at Urbana-Champaign § University of California, Irvine † pkordjam@tulane.edu § sameer@uci.edu ‡ {khashab2,christod,mangipu2,danr}@illinois.edu Abstract We present a novel way for designing complex joint inference and learning models using Saul (Ko- rdjamshidi et al., 2015), a recently-introduced declarative learning-based programming language (DeLBP). We enrich Saul with components that are necessary for a broad range of learning based Natural Language Processing tasks at various levels of granularity. We illustrate these advances using three different, well-known NLP problems, and show how these generic learning and inference modules can directly exploit Saul’s graph-based data representation. These properties allow the programmer to easily switch between different model formulations and configurations, and consider various kinds of dependencies and correlations among variables of interest with minimal programming effort. We argue that Saul provides an extremely useful paradigm both for the design of advanced NLP systems and for supporting advanced research in NLP. 1 Introduction Most of the problems in natural language processing domain can be viewed as a mapping from an input structure to an output structure that represents lexical, syntactical or semantic aspects of the text. For example, Part-of-Speech (POS) tagging provides a syntactic representation, Semantic Role Labeling (SRL) is a (shallow) semantic representation, and all variations of information extraction such as Entity-Relation (ER) extraction provide a lightweight semantic representation of unstructured textual data. Even though the text data looks initially unstructured, solving such problems requires one to consider various kinds of relationships between linguistic components at multiple levels of granularity. However, designing machine learning models that deal with structured representations is a challenging problem. Using such representations is challenging in that designing a new learning model or tackling a new task requires a significant amount of task-specific and model-specific programming effort for learning and inference. Additionally, global background knowledge in these models is usually hard-coded, and changing or augmenting it is extremely time-consuming. There are several formal frameworks (Tsochan- taridis et al., 2004; Chang et al., 2013) for training structured output models but these provide no generic solution for doing inference on arbitrary structures. Likewise, probabilistic programming languages (Pfeffer (2009), McCallum et al. (2009) inter alia) try to provide generic probabilistic solutions to arbitrary inference problems but using them in the context of training arbitrary structured output prediction models for real world problems is a challenge; in addition, encoding structured features and high-level background knowledge necessary for these problems is not a component of those frameworks. In this paper we build on the abstractions made available in Saul programming language (Kordjamshidi et al., 2015), in order to create a unified and flexible machine learning programming framework using the generic Constrained-Conditional-Model (CCM) paradigm (Chang et al., 2012). Specifically, we build upon Saul’s graph-based data representation and …

Content truncated to 500 words