Bielefeld SC: Orthonormal Topic Modelling for Grammar Induction

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 119–122, Dublin, Ireland, August 23-24, 2014. Bielefeld SC: Orthonormal Topic Modelling for Grammar Induction John P. McCrae CITEC, Bielefeld University Inspiration 1 Bielefeld, Germany jmccrae@cit-ec.uni-bielefeld.de Philipp Cimiano CITEC, Bielefeld University Inspiration 1 Bielefeld, Germany cimiano@cit-ec.uni-bielefeld.de Abstract In this paper, we consider the application of topic modelling to the task of induct- ing grammar rules. In particular, we look at the use of a recently developed method called orthonormal explicit topic analysis, which combines explicit and latent models of semantics. Although, it remains unclear how topic model may be applied to the case of grammar induction, we show that it is not impossible and that this may allow the capture of subtle semantic distinctions that are not captured by other methods. 1 Introduction Grammar induction is the task of inducing high- level rules for application of grammars in spoken dialogue systems. In practice, we can extract rel- evant rules and the task of grammar induction re- duces to finding similar rules between two strings. As these strings are not necessarily similar in sur- face form, what we really wish to calculate is the semantic similarity between these strings. As such, we could think of applying a semantic anal- ysis method. As such we attempt to apply topic modelling, that is methods such as Latent Dirich- let Allocation (Blei et al., 2003), Latent Seman- tic Analysis (Deerwester et al., 1990) or Explicit Semantic Analysis (Gabrilovich and Markovitch, 2007). In particular we build on the recent work to unify latent and explicit methods by means of orthonormal explicit topics. In topic modelling the key choice is the docu- ment space that will act as the corpus and hence topic space. The standard choice is to regard all articles from a background document collection – Wikipedia articles are a typical choice – as the This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ topic space. However, it is crucial to ensure that these topics cover the semantic space evenly and completely. Following McCrae et al. (McCrae et al., 2013) we remap the semantic space defined by the topics in such a manner that it is orthonormal. In this way, each document is mapped to a topic that is distinct from all other topics. The structure of the paper is as follows: we de- scribe our method in three parts, first the method in section 2, followed by approximation method in section 3, the normalization methods in section 4 and finally the application to grammar induction in section 5, we finish with some conclusions in section 6. 2 Orthonormal explicit topic analysis ONETA (McCrae et al., 2013, Orthonormal ex- plicit topic analysis) follows Explicit Semantic Analysis in the sense that it assumes the avail- ability of a background document collection B = {b1, b2, ..., bN} consisting of textual representa- tions. The mapping into the …

Content truncated to 500 words