Automatic Clause Boundary Annotation in the Hindi Treebank

Automatic Clause Boundary Annotation in the Hindi Treebank Rahul Sharma, Soma Paul, Riyaz Ahmad Bhat and Sambhav Jain Language Technology Research Centre, IIIT-Hyderabad, India {rahul.sharma, riyaz.bhat, sambhav.jain}@research.iiit.ac.in, soma@iiit.ac.in Abstract In this paper, we propose a method for au- tomatic clause boundary annotation in the Hindi Dependency Treebank. We show that the clausal information implicitly encoded in a dependency structure can be made explicit with no or less human interven- tion. We exercised the proposed approach on 16,000 sentences of Hindi Dependency Treebank. Our approach gives an accuracy of 94.44% for clause boundary identifica- tion evaluated over 238 clauses. The resul- tant corpus has varied usages and can be utilized for developing a statistical clause boundary identifier. 1 Introduction Clause boundary is important for various NLP systems like machine translation, parallel corpora alignment, parsing etc. (Leffa, 1998; Gadde et al., 2010; Ejerhed, 1988). This information is fur- nished by an automatic tool often called clause boundary identifier. Both data driven (Puscasu, 2004) and rule based (Leffa, 1998) approaches have been explored in past for building such a sys- tem, however recent inclination has been towards the data-driven approaches due to their robustness. In order to built a clause boundary identifier, using data driven approach, one needs to have a good clause boundary annotated corpus for training. At present, such a resource is not available in Hindi. However, the syntactic treebank with dependency relations annotated has been developed. We wish to expand this manually annotated treebank with the clause boundary annotation in this work. Several insightful approaches, in past, have en- riched existing resources by first utilizing the ex- plicit information available to derive new implicit information (Klein and Manning, 2003; Kosaraju et al., 2012) and then explicitly annotating it back into the original resource. Conversion of a tree- bank from one grammatical formalism to the other serves as a good example of how an implicit in- formation can be mapped and extracted (Xia and Palmer, 2001). Instead of starting from scratch, an already existing treebank is transformed into a new grammatical formalism. Bhatt et al. (2009) is one such effort for Hindi. They have automati- cally transformed dependency structures to phrase structure utilizing Hindi Dependency Treebank and Hindi PropBank (Palmer et al., 2009). Fol- lowing such insights, we attempt to automatically generate the clause marked data from existing re- sources for Hindi. We propose that the clause in- formation is implicitly encoded in the Hindi De- pendency Treebank and thus, can be extracted and explicitly specified as an additional layer of anno- tation in the treebank. This paper presents a sys- tematic approach towards incorporating clausal in- formation in the Hindi Dependency Treebank uti- lizing the information (mopho-syntactic, depen- dency etc.) already available in the treebank. This paper is structured as follows: In Section 2, we discuss the related works that have been done earlier on clause identification and classification. In Section 3, we talk about clause and its types. In Section 4, we discuss about Hindi-Urdu treebank. Section 5 describes …

Content truncated to 500 words