Spring Cleaning and Grammar Compression: Two Techniques for Detection of Redundancy in HPSG Grammars

Spring Cleaning and Grammar Compression: Two Techniques for Detection of Redundancy in HPSG Grammars Antske Fokkens a, Yi Zhang b and Emily M. Bender c aDepartment of Computational Linguistics, Saarland University Saarland University Campus, 66123 Saarbr¨ucken, Germany afokkens@coli.uni-saarland.de bLT-Lab, German Research Center for Artificial Intelligence Stuhlsatzenhausweg 3 / Building D3, 66123 Saarbr¨ucken, Germany yizhang@dfki.de cDepartment of Linguistics, University of Washington, Box 354340, Seattle, WA 98195-4340, USA ebender@uw.edu Abstract. This paper presents two approaches that identify which parts of an implemented grammar are used and which parts are computationally inactive. Our results lead to the following insights: even small grammars contain noise due to revised analyses, removing superfluous types from a grammar may help to detect errors in the original grammar and at least half of the types defined in the grammars we investigated do not play a role in the computational process of the grammar. Keywords: Implemented grammars, HPSG, typed feature structures. 1 Introduction Grammars of natural language are highly complex objects. This complexity is reflected in imple- mentations of linguistically motivated precision grammars. Understanding the role of a specific element in a broad coverage precision grammar is therefore not always straightforward, even for the engineer who implemented the grammar. Implementations for different phenomena interact, and revisions to the grammar may change the role of existing parts of the grammar. In this paper, we present two techniques for investigating the role that specific types play in implemented HPSG (Pollard and Sag, 1994) grammars. The first, dubbed ‘spring cleaning’, focuses on identifying portions of the grammar that do not play any role in the set of sentences it recognizes or the struc- tures it assigns to them. Such artifacts can accrue in a grammar because abandoned analyses are not completely removed or because the grammar is built on a cross-linguistic resource but does not use all of the infrastructure that resource provides. Spring cleaning is intended to be used in the course of grammar development and as such must leave the grammar in a state that is still easy to maintain. In contrast, the second technique, ‘grammar compression’, computes the smallest subset of a type hierarchy that can assign the same structures to the same sentences as the original grammar. In grammar compression, we remove not only the types taken out in spring cleaning, but also those that exist only to express generalizations over their subtypes. We use these techniques to explore the degree of redundancy in a range of DELPH-IN1 gram- mars, including the two grammars of Wambaya (Bender, 2010), the BURGER grammar of Bul- garian (Osenova, 2010), the ManGO grammar2 of Mandarin Chinese, all built with the LinGO Copyright 2011 by Antske Fokkens, Yi Zhang and Emily M. Bender 1 http://www.delph-in.net/ 2 http://wiki.delph-in.net/moin/MandarinGrammarOnline 25th Pacific Asia Conference on Language, Information and Computation, pages 236–244 236 Grammar Matrix (Bender et al., 2002; Bender et al., 2010), and two much larger grammars, the English Resource Grammar (Flickinger, 2000) and German Grammar (M¨uller and Kasper, 2000; Crysmann, 2005). This paper is structured …

Content truncated to 500 words