Using lexical and Dependency Features to Disambiguate Discourse Connectives in Hindi

Using Lexical and Dependency Features to Disambiguate Discourse Connectives in Hindi Rohit Jain, Himanshu Sharma, Dipti Misra Sharma FC Kohli Center on Intelligent Systems (KCIS) International Institute of Information Technology - Hyderabad Telangana, India rohit.jain@research.iiit.ac.in, himanshu.sharma@research.iiit.ac.in, dipti@iiit.ac.in Abstract Discourse parsing is a challenging task in NLP and plays a crucial role in discourse analysis. To enable discourse analysis for Hindi, Hindi Discourse Relations Bank was created on a subset of Hindi TreeBank. The benefits of a discourse analyzer in auto- mated discourse analysis, question summarization and question answering domains has motivated us to begin work on a discourse analyzer for Hindi. In this paper, we focus on discourse connective identification for Hindi. We explore various available syntactic features for this task. We also explore the use of dependency tree parses present in the Hindi TreeBank and study the impact of the same on the performance of the system. We report that the novel dependency features introduced have a higher impact on precision, in comparison to the syntactic features previously used for this task. In addition, we report a high accuracy of 96% for this task. Keywords: Discourse analysis, Discourse connective identification, Hindi 1. Introduction Units within a piece of text are not meant to be understood independently but understood by linking them with other units in the text. These units may be clauses, sentences or even complete paragraphs. Establishing relations between units present in a text allows the text to be semantically well structured and understandable. Understanding the internal structure of text and the identification of discourse relations is called discourse analysis. The Hindi Discourse Relation Bank (Prasad et al., 2008b; Oza et al., 2009; Kolachina et al., 2012) was created to en- able discourse analysis and research beyond sentence-level for Hindi. Similar discourse annotation projects for En- glish (Miltsakaki et al., 2004; Prasad et al., 2008a), Chi- nese (Xue, 2005; Zhou and Xue, 2012), Czech (Mladov´a et al., 2008; Pol´akov´a et al., 2013) and Turkish (Zeyrek and Webber, 2008; Zeyrek et al., 2010) have also been carried out. A fully automated discourse analyzer would thus greatly aid discourse analysis and NLP applications such as text summarization and question answering. The benefits of a fully automated discourse analyzer has led us to work to- wards the same for Hindi. Given a text, a discourse analyzer would mark the pres- ence discourse relations between two spans of text in a discourse. The presence of discourse relations are often marked by the presence of discourse connectives. Such dis- course relations are called Explicit discourse relations, and those whose presence is not marked by discourse connec- tives are called Implicit discourse relations. Thus the first component towards a complete discourse an- alyzer would be discourse connective identifier. Our work involves identification of discourse connectives which are explicitly realized i.e explicit connectives. In this paper we experiment with lexical and syntactic fea- tures and study their performance for this task. In addition to these features we attempt to make use of the dependency tree …

Content truncated to 500 words