Data-driven learning of symbolic constraints for a log-linear model in a phonological setting

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2217–2226, Osaka, Japan, December 11-17 2016. Data-driven learning of symbolic constraints for a log-linear model in a phonological setting Gabriel Doyle Department of Psychology Stanford University Stanford, CA 94305 gdoyle@stanford.edu Roger Levy Department of Brain and Cognitive Sciences Massachusetts Institute of Techonology Cambridge, MA 02139 rplevy@mit.edu Abstract We propose a non-parametric Bayesian model for learning and weighting symbolically-defined constraints to populate a log-linear model. The model jointly infers a vector of binary con- straint values for each candidate output and likely definitions for these constraints, combining observations of the output classes with a (potentially infinite) grammar over potential constraint definitions. We present results on a small morphophonological system, English regular plurals, as a test case. The inferred constraints, based on a grammar of articulatory features, perform as well as theoretically-defined constraints on both observed and novel forms of English regular plurals. The learned constraint values and definitions also closely resemble standard constraints defined within phonological theory. 1 Introduction Constraint-based models of language, often in the form of “maximum entropy” or “log-linear” models, are prominent in many applications and theoretical analyses in computational linguistics and psycholin- guistics, including in text segmentation (Beeferman et al., 1999; Poon et al., 2009), machine translation (Och and Ney, 2002), syntactic alternation choice (Bresnan et al., 2007), and phonology (Goldwater and Johnson, 2003). Building successful models – and learning about human behavior from them – relies on the ability to identify relevant constraints, and this can be a difficult problem. In this paper, we propose a system for learning both the values of and symbolic definitions for such constraints. We present a framework that combines observed data about linguistic outcomes with a flexible probabilistic context-free grammar of constraint structure to jointly infer (binary) feature values for multiple constraints and likely symbolic definitions for those constraints. We ground the model in a morphophonological setting, using the model to infer what phonological constraints affect the output form of regular English plurals, although it can be applied to other problems for which a constraint grammar can be defined. The inference procedure moves beyond existing methods for learning extensional definitions of con- straint values (Griffiths and Ghahramani, 2005; G¨or¨ur et al., 2006; Doyle et al., 2014) from observational data to incorporate top-down information about likely intensional constraint definitions, improving both the applicability of the constraints and the theoretical basis for their values. We show that learning the constraints through this model performs as well as using pre-specified phonologically-standard con- straints in explaining both observed and novel regular plural morphophonology. In addition, the structure of the learned constraints is similar to standard phonological constraints, showing that the model can be useful in both applications and theory-building. 2 Constraint-based models and the phonological test case Our core problem is how to learn appropriate identities and weights for log-linear features in linguistic applications. In general, we assume some set of input types {xi}, with ni· instances of each type …

Content truncated to 500 words