Identifying Urdu Complex Predication via Bigram Extraction

Proceedings of COLING 2012: Technical Papers, pages 409–424, COLING 2012, Mumbai, December 2012. Identifying Urdu Complex Predication via Bigram Extraction Miriam Butt1 Tina B¨ogel1 Annette Hautli1 Sebastian Sul ger1 Taf seer Ahmed2 (1) Universität Konstanz, Germany (2) University of Karachi, Pakistan firstname.lastname@uni-konstanz.de1, tafseer@uok.edu.pk2 ABSTRACT A problem that crops up repeatedly in shallow and deep syntactic parsing approaches to South Asian languages like Urdu/Hindi is the proper treatment of complex predications. Problems for the NLP of complex predications are posed by their productiveness and the ill understood nature of the range of their combinatorial possibilities. This paper presents an investigation into whether fine-grained information about the distributional properties of nouns in N+V CPs can be identified by the comparatively simple process of extracting bigrams from a large “raw” corpus of Urdu. In gathering the relevant properties, we were aided by visual analytics in that we coupled our computational data analysis with interactive visual components in the analysis of the large data sets. The visualization component proved to be an essential part of our data analysis, particular for the easy visual identification of outliers and false positives. Another essential component turned out to be our language-particular knowledge and access to existing language-particular resources. Overall, we were indeed able to identify high frequency N-V complex predications as well as pick out combinations we had not been aware of before. However, a manual inspection of our results also pointed to a problem of data sparsity, despite the use of a large corpus. KEYWORDS: Urdu, complex predicates, visualization, bigrams, corpus. 409 1 Introduction A problem that crops up repeatedly in shallow and deep syntactic parsing approaches to South Asian languages like Urdu/Hindi1 is the proper treatment of complex predications. Whereas verbal expressions in European languages are mostly realized with a single predicate (e.g. ‘to remember’), South Asian languages tend to use combinations of more than one element to express an action (e.g., memory+do = ‘remember’). In Urdu, only about 700 simple verbs exist (Humayoun 2006) — the remaining verbal inventory consists of complex predications. Complex predicates (CPs) are encountered frequently in general language use, as well as in newspaper corpora. Thus, any NLP application, whether shallow or deep, whether its goal be parsing, generation, question-answering or the construction of lexical resources like WordNet (cf. Bhattacharyya 2010), encounters complex predication sooner rather than later. While the productive capability as well as constraints on V+V combinations are now compara- tively well understood from a theoretical point of view (e.g, see Hook 1974; Butt 1995, 2010 and references therein), the constraints governing N+V, ADJ+V as well as P+V combinations are less well understood (the standard reference for N+V is Mohanan 1994, there is no standard reference for the other types). This is despite the fact that these latter three are very productive. Indeed, very little is known overall about P+V combinations (with Raza 2011 providing a first description), but as our results show, they do occur in newspaper corpora (section 5). With respect to NLP applications the frequency …

Content truncated to 500 words