NorGramBank: A ‘Deep’ Treebank for Norwegian

NorGramBank: A ‘Deep’ Treebank for Norwegian Helge Dyvik1, Paul Meurer2, Victoria Rosén1, Koenraad De Smedt1, Petter Haugereid1, Gyri Smørdal Losnegaard1, Gunn I. Lyse1, Martha Thunes1 University of Bergen1, Uni Research Computing2 dyvik@uib.no, paul.meurer@uni.no, desmedt@uib.no, victoria@uib.no, petterha@gmail.com, gyri.losnegaard@uib.no, gunn.lyse@uib.no, martha.thunes@uib.no Abstract We present NorGramBank, a treebank for Norwegian with highly detailed LFG analyses. It is one of many treebanks made avail- able through the INESS treebanking infrastructure. NorGramBank was constructed as a parsebank, i.e. by automatically parsing a corpus, using the wide coverage grammar NorGram. One part consisting of 350,000 words has been manually disambiguated using computer-generated discriminants. A larger part of 50 M words has been stochastically disambiguated. The treebank is dynamic: by global reparsing at certain intervals it is kept compatible with the latest versions of the grammar and the lexicon, which are continually further developed in interaction with the annotators. A powerful query language, INESS Search, has been developed for search across formalisms in the INESS treebanks, including LFG c- and f-structures. Evaluation shows that the grammar provides about 85% of randomly selected sentences with good analyses. Agreement among the annotators responsible for manual disambiguation is satisfactory, but also suggests desirable simplifications of the grammar. Keywords: treebanks, Norwegian, Lexical Functional Grammar 1. Motivation and Design NorGramBank is the first treebank for written Norwe- gian (Bokmål and Nynorsk)1 based on ‘deep’ parsing. It is constructed through parsing with NorGram, a manually written grammar within the framework of Lexical Func- tional Grammar (LFG). NorGram contains about 380 com- plex syntactic rules (mostly valid for both Bokmål and Nynorsk), corresponding to a transition network with more than 160,000 states and more than 4.7 M arcs. The lexicon comprises around 180,000 lemmas for Bokmål and 110,000 lemmas for Nynorsk and has more than 200 different verb frames. Approximately 350,000 words of parsed text have been manually disambiguated and checked using computer- generated discriminants. Through stochastic disambigua- tion the corpus has been extended to about 50 M word to- kens. NorGramBank and many other treebanks are made avail- able through the INESS platform (Rosén et al., 2012), which enables advanced exploration. Here authenticated re- searchers have access to a fully web-based interface for tree- bank selection, browsing and search with visualization of syntactic structures. Since the treebank was designed as a parsed corpus, the grammar used by the parser is decisive for the annotation. The design of NorGram is based partly on the principles developed within the Parallel Grammar Project (Butt et al., 2002), which define a cross-linguistic set of features to be used in the f-structures. The ParGram features have been established through a long-term project involving grammar development for a wide range of languages and thus en- sures a degree of cross-linguistic validity for the descrip- 1Bokmål and Nynorsk are the two written standard varieties of Norwegian tive choices. Furthermore, the design is based partly on the principles for X-bar syntax and its integration in LFG pro- posed by Joan Bresnan (Bresnan, 2001), which govern the design of the c-structures and …

Content truncated to 500 words