CHOLAN: A Modular Approach for Neural Entity Linking on Wikipedia and Wikidata

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 42–48 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 42 On the (In)Effectiveness of Images for Text Classification Chunpeng Ma1, Aili Shen2, Hiyori Yoshikawa1, Tomoya Iwakura1, Daniel Beck2, Timothy Baldwin2 1 Fujitsu Laboratories LTD., Japan 2 The University of Melbourne, Australia {ma.chunpeng, y.hiyori, iwakura.tomoya}@fujitsu.com ailis@student.unimelb.edu.au, {beck.d, tbaldwin}@unimelb.edu.au Abstract Images are core components of multi-modal learning in natural language processing (NLP), and results have varied substantially as to whether images improve NLP tasks or not. One confounding effect has been that previous NLP research has generally focused on sophis- ticated tasks (in varying settings), generally ap- plied to English only. We focus on text classi- fication, in the context of assigning named en- tity classes to a given Wikipedia page, where images generally complement the text and the Wikipedia page can be in one of a number of different languages. Our experiments across a range of languages show that images comple- ment NLP models (including BERT) trained without external pre-training, but when com- bined with BERT models pre-trained on large- scale external data, images contribute nothing. 1 Introduction Combining data from multiple modalities (e.g., text, images, categorical metadata, or user inter- action features) has become commonplace in artifi- cial intelligence. In NLP, examples include multi- modal machine translation (MMT) (Elliott et al., 2016; Elliott, 2018), visual question answering (VQA) (Goyal et al., 2017; Johnson et al., 2017), vi- sual commonsense reasoning (VCR) (Zellers et al., 2019; Geva et al., 2019), and multi-modal pre- training (Lu et al., 2019; Chen et al., 2019). While tasks such as VQA and VCR are multi- modal in nature, there has been research on tradi- tionally text-based tasks such as text classification (Shen et al., 2020; Huang, 2018) and word em- bedding learning (Bruni et al., 2014) which has demonstrated that the addition of images boosts performance. At the same time, however, there is evidence of images providing no additional infor- mation, e.g. Caglayan et al. (2019) show that MMT models learn to ignore visual content when trained on a parallel corpus of image captions (Elliott et al., 2016). These mixed findings raise the question of when visual context is actually useful in NLP. In this work, we take a first step towards an- swering this question, in focusing on the task of text classification, which has traditionally been ad- dressed using textual data only. We identify two gaps in the literature on multi-modal NLP: (1) no results for pre-trained language models (LMs); and (2) no results for languages other than English. The first is important in terms of updating the research relative to state-of-the-art approaches, while the second relates to the question of how “language- independent” systems actually are (Bender, 2011). We fill these gaps via a text classification task over Wikipedia articles (Sekine et al., 2019). Our main findings are: (1) while images do help in a tradi- tional supervised learning setting, their utility dis- appears …

Content truncated to 500 words