HuaAMS at SemEval-2022 Task 8: Combining Translation and Domain Pre-training for Cross-lingual News Article Similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1151 - 1156 July 14-15, 2022 ©2022 Association for Computational Linguistics HuaAMS at SemEval-2022 Task 8: Combining Translation and Domain Pre-training for Cross-lingual News Article Similarity Sai Sandeep Sharma Chittilla and Talaat Khalil Huawei Technologies R&D Amsterdam, Netherlands {sandeep.chittilla, talaat.khalil}@huawei.com Abstract This paper describes our submission to SemEval-2022 Multilingual News Article Sim- ilarity task. We experiment with different ap- proaches that utilize a pre-trained language model fitted with a regression head to predict similarity scores for a given pair of news arti- cles. Our best performing systems include 2 key steps: 1) pre-training with in-domain data 2) training data enrichment through machine translation. Our final submission is an ensem- ble of predictions from our top systems. While we show the significance of pre-training and augmentation, we believe the issue of language coverage calls for more attention. 1 Introduction In the recent few years, there has been a growing interest towards automating news understanding tasks thanks to the continuous demand by down- stream applications. One of the most important aspects of news understanding is identifying news articles that cover the same stories. Grouping such similar articles can be useful in multiple appli- cation scenarios including news recommendation, news stories analysis, news retrieval and ranking amongst others. Task 8 in SemEval 2022 (Chen et al., 2022) pro- vides an experimental setup to address and identify the challenges of assessing the similarity between two news articles. Unlike standard document simi- larity tasks (Agirre et al., 2015), this task is focused on a more challenging multilingual setting where the systems are not only expected to evaluate pairs of long text articles in the same language but in different languages as well. Moreover, the task as- sumes that accurately identifying articles that share the same story cannot be solely captured by textual similarity since the underlying similarity function is hypothetically a combination of a set of other features like time, geo-location, mentions of named entities and narratives. Participating teams are required to provide a sim- ilarity score for each pair of news articles. The scores should range between 1 and 4 where 1 in- dicates that two stories are almost identical and 4 indicates no similarity at all. The systems are evaluated based on the Pearson’s correlation with ground truth scores which are provided in a test set annotated by human evaluators. In our submission, we hypothesize that the sub- dimensions (that are assumed to be story similarity predictors by the task organizers) do not have to be independently modeled and present an approach that assumes that such sub-dimensions can be rep- resented in the model’s latent space while directly optimizing to learn article similarity scores. For- mally speaking, we model the similarity task as a supervised regression problem to optimize the similarity score between two news articles. To test our hypothesis, we experiment with different pre-trained language models and evaluate multiple methods of boosting the model’s performance via domain pre-training and data augmentation. …

Content truncated to 500 words