How Symbolic Learning Can Help Statistical Learning (and vice versa)

Proceedings of Recent Advances in Natural Language Processing, pages 649–658, Hissar, Bulgaria, 7-13 September 2013. How Symbolic Learning Can Help Statistical Learning (and vice versa) Isabelle Tellier Lattice / Lattice - UMR 8094 Lattice / 1 rue Maurice Arnoux Lattice / 92120 Montrouge isabelle.tellier@univ-paris3.fr Yoann Dupont Lattice / Lattice - UMR 8094 Lattice / 1 rue Maurice Arnoux Lattice / 92120 Montrouge yoa.dupont@gmail.com Abstract We describe in this paper how different learning strategies can be applied on the same NLP task, namely chunking. The reference corpus is extracted from the French Treebank, the symbolic learning strategy used is grammatical inference and the statistical one is CRFs (Conditional Random Fields). As expected, the sym- bolic approach allows readability but is less effective than the statistical one. We then propose two distinct ways to combine both approaches and show that in both cases they benefit from one another. 1 Introduction Supervised machine learning approaches, espe- cially when they have access to huge amounts of data, have now extensively proved their effective- ness for a lot of text mining tasks like text classi- fication, sentence annotation and information ex- traction. Most effective learning approaches rely on a theoretical background which is either opti- mization (SVM), statistics (Naive Bayes) or both (HMMs, MaxEnt models, CRFs). But, however effective they may be, the main drawback of these techniques is that they usually do not provide any human-readable model. There also exists other branches of Machine Learning, referred to as symbolic, whose partic- ularity is to provide a more human-readable out- put. This is the case of decision trees, Inductive Logic Programming (ILP) or Grammatical Infer- ence (GI in the following). The latter is our main interest here. It can be defined as the study of how it is possible to automatically learn a formal grammar or any other device able to represent a language (such as an automaton, a regular expres- sion...) from a sample of (possibly enriched) se- quences known to belong (or not) to this language (de la Higuera, 2010). This domain is often not very well known due to its roots in theoretical computer science and formal language theory. GI algorithms’ known drawback is their lack of ef- ficiency on real data: they are often time consum- ing, sensitive to errors and do not behave well with large alphabets (for example alphabets containing every word of a natural language). In this article, we want to give some GI algo- rithms a chance to compete with the state of the art of statistical machine learning approaches. The task we deal with for this purpose is chunking (Ab- ney, 1991) for French, which can be done with hand-made automata (Antoine et al., 2008; Blanc et al., 2010). To our knowledge, trying to auto- matically learn these automata instead of writing them by hand has never been tested for any lan- guage before. On the other hand, chunking can also be treated as an annotation task (cf. shared task of CoNLL2000) and thus been efficiently …

Content truncated to 500 words