Bilingual Tabular Inference: A Case Study on Indic Languages

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4018 - 4037 July 10-15, 2022 ©2022 Association for Computational Linguistics Bilingual Tabular Inference: A Case Study on Indic Languages Chaitanya Agarwal1∗, Vivek Gupta2*†, Anoop Kunchukuttan3,4, Manish Shrivastava1 1LTRC, IIIT Hyderabad; 2University of Utah; 3AI4Bharat; 4Microsoft chaitanya.agarwal@research.iiit.ac.in; vgupta@cs.utah.edu; ankunchu@microsoft.com; m.shrivastava@iiit.ac.in Abstract Existing research on Tabular Natural Language Inference (TNLI) exclusively examines the task in a monolingual setting where the tabular premise and hypothesis are in the same language. However, due to the uneven distribution of text resources on the web across languages, it is common to have the tabular premise in a high resource language and the hypothesis in a low resource language. As a result, we present the challenging task of bilingual Tabular Natural Language Inference (bTNLI), in which the tabular premise and a hypothesis over it are in two separate languages. We construct EI-INFOTABS: an English-Indic bTNLI dataset by translating the textual hypotheses of the English TNLI dataset INFOTABS into eleven major Indian languages. We thoroughly investigate how pre- trained multilingual models learn and perform on EI-INFOTABS. Our study shows that the performance on bTNLI can be close to its monolingual counterpart, with translate-train, translate-test and unified-train being strongly competitive baselines. 1 Introduction Tabular Natural Language Inference (TNLI) is the task of classifying whether a textual hypothesis is an entailment, contradiction or a neutral extension of the given tabular premise. The task requires a broad range of reasoning abilities, including but not limited to the ability to make lexical, spatio- temporal, and semantic deductions. Recently published datasets, TabFact (Chen et al., 2020b) and INFOTABS (Gupta et al., 2020), have enabled the examination of the TNLI task. Moreover, sophisticated models based on deep contextual embeddings like BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), etc. trained under ∗Equal Contribution †Corresponding Author Joe Strummer Birth Name John Graham Mellor Born 1952-08-21 Ankara, Turkey Died 2002-12-22 Broomfield, Somerset, England Genres Punk Rock, Post Punk Occupation(s) Musician, Songwriter, Radio Host, Actor Instruments Vocals, Guitar, Piano Years Active 1970-2002 Labels CBS, Sony, Hellcat, Mercury Associated Acts The 101ers, The Clash H1: John Graham Mellor plays less instruments than the number of labels he has worked for. H2: Joe Strummer changed his surname after he became a guitar player. H3: Joe Strummer was active in the sports industry for over three decades. H1hi−trl: jon gr¯aham melar un lebaloan k¯ı sankhy¯a k¯ı tulan¯a mean kam v¯adya baj¯ate haian jinake lie unhoanne k¯am kiy¯a hai H2hi−trl: jo st.ramar ne ek git.¯ar v¯adak banane ke b¯ad apan¯a upan¯am badal liy¯a H3hi−trl: jo st.ramar t¯ın dashakoan se khel udyog mean sakriya the Figure 1: Tabular premise followed by human written hypotheses (H1, H2, H3). H1 is entailed entirely from the premise, H2 is neither entailed nor contradictory, and H3 is contradictory. H1hi−trl, H2hi−trl, and H3hi−trl are the transliterations of Hindi translations of the former, released as a part of our EI-INFOTABS dataset. supervision on heuristic adaptations …

Content truncated to 500 words