WMD at SemEval-2020 Tasks 7 and 11: Assessing Humor and Propaganda Using Unsupervised Data Augmentation

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1865–1874 Barcelona, Spain (Online), December 12, 2020. 1865 WMD at SemEval-2020 Tasks 7 and 11: Assessing humor and propaganda using Unsupervised Data Augmentation Guillaume Daval-Frerot Paris, France gdavalfrerot@gmail.com Yannick Weis Karlsruhe, Germany yannick.weis@freenet.de Abstract In this work, we combine the state-of-the-art BERT architecture with the semi-supervised learn- ing technique UDA in order to exploit unlabeled raw data to assess humor and detect propaganda in the tasks 7 and 11 of the SemEval-2020 competition. The use of UDA shows promising results with a systematic improvement of the performances over the four different subtasks, and even outperforms supervised learning with the additional labels of the Funlines dataset. 1 Introduction In times when information is able to circulate freely via the internet with limited to non-existing qual- ity control, organized groups and governments are using various tools of propaganda to spread miss- information and push their own agenda to achieve financial and political gains (Miller, 1939). With millions of press articles written every month, it became unrealistic for humans to perform any form of regulation. Machine Learning (ML) can help with the automatic processing of large volumes of data, but paradoxically also requires a significant amount of examples in order to apprehend high-level con- cepts such as humor or propaganda. More and more solutions to this problem are emerging, for example Transfer Learning (Pan and Yang, 2010) which allows the use of already existing resources to increase the performance of a given task. It can be achieved by using a larger and closely related dataset, or simply by fine-tuning a pre-trained model. A recent neural architecture proposed for natural language processing in (Devlin et al., 2018) and achieving state-of-the-art results in numerous tasks is called Bidirectional Encoder Representations from Transformers (BERT). One of the reasons for its popularity is the availability of models pre-trained with large resources in different languages. It is based on the combination of bidirectional Long Short-Term Memory (LSTM) networks, described in (Hochreiter and Schmidhuber, 1997) and Attention Mecha- nisms, described in (Bahdanau et al., 2015). Both techniques have been discussed and considered as promising subjects of study in previous works (Daval-Frerot et al., 2018) on sentiment analysis during the SemEval-2018 challenge. Another solution to deal with the data collection is to exploit raw data with semi-supervised ap- proaches. Both humor assessment in Task 7 (Hossain et al., 2020a) and propaganda detection in Task 11 (Da San Martino et al., 2020) have the advantage of a wide availability of samples in online press articles. By avoiding the use of labels, a lot more resources can be used through techniques like Un- supervised Data Augmentation (UDA) proposed in (Xie et al., 2019). The idea is to draw similarities between labeled and unlabeled data during a semi-supervised training, making the overwhelming mil- lions of un-regulated articles into a strength as an exploitable resource. In this paper, we propose to combine the two solutions along with other practices with the objective of using …

Content truncated to 500 words