Hypernymy Detection for Low-Resource Languages via Meta Learning

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3651–3656 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3651 Hypernymy Detection for Low-Resource Languages via Meta Learning Changlong Yu1∗ Jialong Han2∗ Haisong Zhang3 Wilfred Ng1 1HKUST, Hong Kong, China 2Amazon, USA {cyuaq, wilfred}@cse.ust.hk3Tencent AI Lab, China jialonghan@gmail.com, hansonzhang@tencent.com Abstract Hypernymy detection, a.k.a. lexical entail- ment, is a fundamental sub-task of many natu- ral language understanding tasks. Previous ex- plorations mostly focus on monolingual hyper- nymy detection on high-resource languages, e.g., English, but few investigate the low- resource scenarios. This paper addresses the problem of low-resource hypernymy detec- tion by combining high-resource languages. We extensively compare three joint training paradigms and for the first time propose apply- ing meta learning to relieve the low-resource issue. Experiments demonstrate the superior- ity of our method among the three settings, which substantially improves the performance of extremely low-resource languages by pre- venting over-fitting on small datasets. 1 Introduction Hypernymy is a fundamental asymmetric lexico- semantic relation. It expresses is-a relationship between concepts and is widely used to build tax- onomies (Miller, 1995) or large-scale knowledge bases (Wu et al., 2012; Seitner et al., 2016). Lexico- semantic patterns (e.g., X such as Y) are gener- ally employed to harvest benchmark datasets or resources from large English corpus due to their high precision (Hearst, 1992). However, Hearst- like patterns of English can not be easily trans- ferred to other languages such as Chinese. Creat- ing high-quality hypernymy benchmarks for other languages requires much more human-annotation efforts and hypernymy detection in those languages remains low-resource tasks (Vuli´c et al., 2019). In this paper, we focus on the question: how could we make full use of hypernymy pairs of high-resource languages such as English for other low-resource languages, e.g., Japanese and Thai? ∗ Work done when C. Yu and J. Han were with Tencent AI Lab. To investigate this question, we firstly assume a strong feasibility of semantic relation transfer across languages, which is in line with existing findings on human cognition. Youn et al. (2016) uncovered the universal conceptual structure of hu- man lexical semantics among cross-lingual dictio- naries and revealed the language-independent at- tribute for semantic similarity of concepts. Wang et al. (2019) studied cross-lingual training by sim- ply merging high-resource language pairs and low- resource language ones, which is prone to over- fitting to low-resource ones. Based on the above interesting findings and the datasets in Wang et al. (2019), we study three training paradigms of com- bining training data from multiple different lan- guages, i.e., cross-lingual training, multilingual training, as well as meta learning. To the best of our knowledge, meta learning algorithms have not been previously applied to hy- pernymy detection. We propose applying meta learning algorithms in low-resource hypernymy de- tection and perform extensive comparisons with multilingual training. Meta-learning algorithms aim at learning language-independent models and then fine-tuning on multiple languages with min- imal training instances. In our experiments, we further explore the two following questions: …

Content truncated to 500 words