Two-Step Classification using Recasted Data for Low Resource Settings

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 706–719 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 706 Two-Step Classification using Recasted Data for Low Resource Settings Shagun Uppal1, Vivek Gupta2, Avinash Swaminathan1, Debanjan Mahata 3, Rakesh Gosangi 3, Haimin Zhang 3 Rajiv Ratn Shah1, Amanda Stent3 1 IIIT-Delhi, India, 2 University of Utah, 3 Bloomberg, New York shagun16088@iiitd.ac.in, vgupta@cs.utah.edu, s.avinash.it.17@nsit.net.in, {dmahata, rgosangi, hzhang449, astent}@bloomberg.net, rajivratn@iiitd.ac.in Abstract An NLP model’s ability to reason should be independent of language. Previous works utilize Natural Language Inference (NLI) to understand the reasoning abil- ity of models, mostly focusing on high resource languages like English. To ad- dress scarcity of data in low-resource lan- guages such as Hindi, we use data recast- ing to create four NLI datasets from ex- isting four text classification datasets in Hindi language. Through experiments, we show that our recasted dataset1 is devoid of statistical irregularities and spurious pat- terns. We study the consistency in pre- dictions of the textual entailment models and propose a consistency regulariser to remove pairwise-inconsistencies in predic- tions. Furthermore, we propose a novel two-step classification method which uses textual-entailment predictions for classifi- cation task. We further improve the classi- fication performance by jointly training the classification and textual entailment tasks together. We therefore highlight the ben- efits of data recasting and our approach 2 with supporting experimental results. 1 Introduction Textual entailment (TE) is the task of deter- mining if a hypothesis sentence can be inferred from a given context sentence. Figure 1 shows examples of context-hypothesis pairs for TE. Previous works (Wang and Zhang, 2009; Tatu and Moldovan, 2005; Sammons et al., 2010) in- vestigated several semantic approaches for TE and demonstrated how they can be used to evaluate inference-related tasks such as Ques- 1https://github.com/midas-research/ hindi-nli-data 2https://github.com/midas-research/ hindi-nli-code tion Answering (QA), reading comprehension (RC) and paraphrase acquisition (PA). Context-Hypothesis Label p : The kid exclaimed with joy. entailed h : The kid is happy. p : I am feeling happy. not-entailed h : I am angry. (contradictory) Table 1: Example illustrating context (c) - hypoth- esis (h) pairs for the task of textual entailment. Researchers have curated many resources3 and benchmark datasets for TE in English (Bowman et al., 2015; Williams et al., 2018; Khot et al., 2018). However, to our knowledge, there is only one TE dataset (XNLI) in Hindi, which was created by translating English data (Conneau et al., 2018) and another in Hindi- English code-switched setting (Khanuja et al., 2020). Hindi is the language with the fourth most native speakers in the world4. Despite its wide prevalence, Hindi is still considered a low-resource language by NLP practitioners because there are a rather limited number of publicly available annotated datasets. Devel- oping models that can accurately process text from low-resource languages, such as Hindi, is critical for the proliferation and broader adop- tion of NLP technologies. Creating a high-quality labeled corpus …

Content truncated to 500 words