Improving Word Translation via Two-Stage Contrastive Learning
39082Authors
nan
Abstract
AbstractWord translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust and effective two-stage contrastive learning framework for the BLI task. At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps. In Stage C2, we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word translation capability. We also show that static WEs induced from the ‘C2-tuned’ mBERT complement static WEs from Stage C1. Comprehensive experiments on standard BLI datasets for diverse languages and different experimental setups demonstrate substantial gains achieved by our framework. While the BLI method from Stage C1 already yields substantial gains over all state-of-the-art BLI methods in our comparison, even stronger improvements are met with the full two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28 language pairs.
Keywords
Full Content
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 373 - 390 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering Yu-Jung Heo1,4, Eun-Sol Kim2, Woo Suk Choi1, and Byoung-Tak Zhang1,3 1 Seoul National University 2 Department of Computer Science, Hanyang University 3 AI Institute (AIIS), Seoul National University 4 Surromind yjheo@bi.snu.ac.kr, eunsolkim@hanyang.ac.kr, {wschoi, btzhang}@bi.snu.ac.kr Abstract Knowledge-based visual question answering (QA) aims to answer a question which re- quires visually-grounded external knowledge beyond image content itself. Answering com- plex questions that require multi-hop reason- ing under weak supervision is considered as a challenging problem since i) no supervision is given to the reasoning process and ii) high- order semantics of multi-hop knowledge facts need to be captured. In this paper, we intro- duce a concept of hypergraph to encode high- level semantics of a question and a knowl- edge base, and to learn high-order associations between them. The proposed model, Hyper- graph Transformer, constructs a question hy- pergraph and a query-aware knowledge hy- pergraph, and infers an answer by encoding inter-associations between two hypergraphs and intra-associations in both hypergraph it- self. Extensive experiments on two knowledge- based visual QA and two knowledge-based textual QA demonstrate the effectiveness of our method, especially for multi-hop reason- ing problem. Our source code is available at https://github.com/yujungheo/ kbvqa-public. 1 Introduction Visual question answering (VQA) is a semantic rea- soning task that aims to answer questions about vi- sual content depicted in images (Antol et al., 2015; Zhu et al., 2016; Hudson and Manning, 2019), and has become one of the most active areas of research with advances in natural language processing and computer vision. Recently, researches for VQA have advanced, from inferring visual properties on entities in a given image, to inferring commonsense or world knowledge about those entities (Wang et al., 2017, 2018; Marino et al., 2019; Shah et al., 2019; Zellers et al., 2019). In this paper, we focus on the task which is called knowledge-based visual question answering, Figure 1: An example of knowledge-based visual ques- tion answering. The rectangles and arrows between the rectangles represent the entities and relations from KB. To answer the given question, the multiple reasoning evidences (marked as orange) are required. where a massive number of knowledge facts from a general knowledge base (KB) is given with an image-question pair. To answer the given question as shown in Figure 1, a model should understand the semantics of the given question, link visual enti- ties appearing in the given image to the KB, extract a number of evidences from the KB and predict an answer by aggregating semantics of both the ques- tion and the extracted evidences. Following these, there are two fundamental challenges in this task. i) To answer a complex question, multi-hop reasoning over multiple knowledge evidences is necessary. ii) Learning a complex reasoning process is difficult especially in a condition where only QA is pro- vided without …
Content truncated to 500 words
Labeled Text
AbstractWord translation or bilingual lexicon induction (BLI) is a key cross-lingual task, aiming to bridge the lexical gap between different languages. In this work, we propose a robust and effective two-stage contrastive learning framework for the BLI task. At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps. In Stage C2, we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word translation capability. We also show that static WEs induced from the ‘C2-tuned’ mBERT complement static WEs from Stage C1. Comprehensive experiments on standard BLI datasets for diverse languages and different experimental setups demonstrate substantial gains achieved by our framework. While the BLI method from Stage C1 already yields substantial gains over all state-of-the-art BLI methods in our comparison, even stronger improvements are met with the full two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28 language pairs.
Publication Details
- Paper ID:
- 39082
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan