A Deep Transfer Learning Method for Cross-Lingual Natural Language Inference

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3084–3092 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3084 A Deep Transfer Learning Method for Cross-Lingual Natural Language Inference Dibyanayan Bandyopadhyay1 Arkadipta De2∗ Baban Gain1 Tanik Saikh1 Asif Ekbal1 1Department of Computer Science and Engineering, 1Indian Institute of Technology Patna, India 1Department of Artificial Intelligence, 2Indian Institute of Technology Hyderabad, India 1{dibyanayan 2111cs02, baban 2111cs21, 1821cs08, asif}@iitp.ac.in 2{ai20mtech14002}@iith.ac.in Abstract Natural Language Inference (NLI), also known as Recognizing Textual Entailment (RTE), has been one of the central tasks in Artificial Intelligence (AI) and Natural Language Processing (NLP). RTE between the two pieces of texts is a crucial problem, and it adds further challenges when involving two different languages, i.e., in the cross-lingual scenario. This paper proposes an effective transfer learning approach for cross-lingual NLI. We perform experiments on English-Hindi language pairs in the cross-lingual setting to find out that our novel loss formulation could enhance the performance of the baseline model by up to 2%. To assess the effectiveness of our method further, we perform additional experiments on every possible language pair using four European languages, namely French, German, Bulgarian, and Turkish, on top of XNLI dataset. Evaluation results yield up to 10% performance improvement over the respective baseline models, in some cases surpassing the state-of-the-art (SOTA). It is also to be noted that our proposed model has 110M parameters which is much lesser than the SOTA model having 220M parameters. Finally, we argue that our transfer learning-based loss objective is model agnostic and thus can be used with other deep learning-based architectures for cross-lingual NLI. Keywords: Natural Language Inference, Textual Entailment, Cross-lingual 1. Introduction Textual Entailment (TE) is one of the fundamental prob- lems of Natural Language Understanding (NLU). Un- derstanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is vital for developing se- mantic representations of natural languages. For two sentences, premise (P) and hypothesis (H), the Natural Language Inference (NLI) task is to understand the rela- tionship between these two sentences (viz. Entailment, Contradiction, Unknown/Neutral). It is an elusive and frontier problem in Artificial Intelligence (AI). More specifically, H entails P if it strictly follows the state- ments: • Hypothesis (H) is a logical consequence of Premise (P). • Hypothesis (H) is true in every circumstance in which the Premise (P) is true. If both P and H are in different languages, the problem is known as Cross-Lingual Textual Entailment (CLTE). Over the years, Natural Language Inference (NLI) has been addressed using a large variety of techniques, including those based on symbolic logic, knowledge bases, machine learning algorithms, and neural net- works. While the field of Textual Entailment is quite popular and developed, limited research has been done *Work done while he was an intern at IIT Patna. in the field of CLTE. The main challenge of CLTE is that the fragments of texts are in different languages having different semantic and …

Content truncated to 500 words