Improvements in Unsupervised Co-Occurrence Based Parsing

Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 1–8, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Improvements in unsupervised co-occurrence based parsing Christian H¨anig Daimler AG Research and Technology 89081 Ulm, Germany christian.haenig@daimler.com Abstract This paper presents an algorithm for unsu- pervised co-occurrence based parsing that improves and extends existing approaches. The proposed algorithm induces a context- free grammar of the language in question in an iterative manner. The resulting struc- ture of a sentence will be given as a hier- archical arrangement of constituents. Al- though this algorithm does not use any a priori knowledge about the language, it is able to detect heads, modifiers and a phrase type’s different compound compo- sition possibilities. For evaluation pur- poses, the algorithm is applied to manually annotated part-of-speech tags (POS tags) as well as to word classes induced by an unsupervised part-of-speech tagger. 1 Introduction With the growing amount of textual data available in the Internet, unsupervised methods for natural language processing gain a considerable amount of interest. Due to the very special usage of lan- guage, supervised methods trained on high quality corpora (e. g. containing newspaper texts) do not achieve comparable accuracy when being applied to data from fora or blogs. Huge annotated corpora consisting of sentences extracted from the Internet barely exist until now. Consequential a lot of effort has been put into unsupervised grammar induction during the last years and results and performance of unsuper- vised parsers improved steadily. Klein and Man- ning (2002)’s constituent context model (CCM) obtains 51.2% f-score on ATIS part-of-speech strings. The same model achieves 71.1% on Wall Street Journal corpus sentences with length of at most 10 POS tags. In (Klein and Manning, 2004) an approach combining constituency and dependency models yields 77.6% f-score. Bod (2006)’s all-subtree approach — known as Data- Oriented Parsing (DOP) — reports 82.9% for UML-DOP. Seginer (2007)’s common cover links model (CCL) does not need any prior tagging and is applied on word strings directly. The f-score for English is 75.9%, and for German (NEGRA10) 59% is achieved. H¨anig et al. (2008) present a co- occurrence based constituent detection algorithm which is applied to word forms, too (unsupervised POS tags are induced using unsuPOS, see (Bie- mann, 2006)). An f-score of 63.4% is reported for German data. In this paper, we want to present a new unsu- pervised co-occurrence based grammar induction model based on H¨anig et al. (2008). In the fol- lowing section, we give a short introduction to the base algorithm unsuParse. Afterwards, we present improvements to this algorithm. In the final sec- tion, we evaluate the proposed model against ex- isting ones and discuss the results. 2 Co-occurrence based parsing It has been shown in (H¨anig et al., 2008) that statistical methods like calculating significant co- occurrences and context clustering are applicable to grammar induction from raw text. The underly- ing assumption states that each word prefers a cer- tain position within a phrase. Two particular cases are …

Content truncated to 500 words