Phrase-Based Statistical Language Generation Using Graphical Models and Active Learning

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1552–1561, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Phrase-based Statistical Language Generation using Graphical Models and Active Learning Franc¸ois Mairesse, Milica Gaˇsi´c, Filip Jurˇc´ıˇcek, Simon Keizer, Blaise Thomson, Kai Yu and Steve Young∗ Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK {f.mairesse, mg436, fj228, sk561, brmt2, ky219, sjy}@eng.cam.ac.uk Abstract Most previous work on trainable language generation has focused on two paradigms: (a) using a statistical model to rank a set of generated utterances, or (b) using statistics to inform the generation deci- sion process. Both approaches rely on the existence of a handcrafted generator, which limits their scalability to new do- mains. This paper presents BAGEL, a sta- tistical language generator which uses dy- namic Bayesian networks to learn from semantically-aligned data produced by 42 untrained annotators. A human evalua- tion shows that BAGEL can generate nat- ural and informative utterances from un- seen inputs in the information presentation domain. Additionally, generation perfor- mance on sparse datasets is improved sig- nificantly by using certainty-based active learning, yielding ratings close to the hu- man gold standard with a fraction of the data. 1 Introduction The field of natural language generation (NLG) is one of the last areas of computational linguistics to embrace statistical methods. Over the past decade, statistical NLG has followed two lines of research. The first one, pioneered by Langkilde and Knight (1998), introduces statistics in the generation pro- cess by training a model which reranks candi- date outputs of a handcrafted generator. While their HALOGEN system uses an n-gram language model trained on news articles, other systems have used hierarchical syntactic models (Bangalore and Rambow, 2000), models trained on user ratings of ∗This research was partly funded by the UK EPSRC un- der grant agreement EP/F013930/1 and funded by the EU FP7 Programme under grant agreement 216594 (CLASSiC project: www.classic-project.org). utterance quality (Walker et al., 2002), or align- ment models trained on speaker-specific corpora (Isard et al., 2006). A second line of research has focused on intro- ducing statistics at the generation decision level, by training models that find the set of genera- tion parameters maximising an objective function, e.g. producing a target linguistic style (Paiva and Evans, 2005; Mairesse and Walker, 2008), gener- ating the most likely context-free derivations given a corpus (Belz, 2008), or maximising the expected reward using reinforcement learning (Rieser and Lemon, 2009). While such methods do not suffer from the computational cost of an overgeneration phase, they still require a handcrafted generator to define the generation decision space within which statistics can be used to find an optimal solution. This paper presents BAGEL (Bayesian networks for generation using active learning), an NLG sys- tem that can be fully trained from aligned data. While the main requirement of the generator is to produce natural utterances within a dialogue sys- tem domain, a second objective is to minimise the overall development effort. In this regard, a major …

Content truncated to 500 words