TextLearner at SemEval-2020 Task 10: A Contextualized Ranking System in Solving Emphasis Selection in Text

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1691–1697 Barcelona, Spain (Online), December 12, 2020. 1691 TextLearner at SemEval-2020 Task 10: A Contextualized Ranking System in Solving Emphasis Selection in Text Zhishen Yang ∗ Tokyo Institute of Technology zhishen.yang @nlp.c.titech.ac.jp Lars Wolfsteller ∗ Technical University of Munich (TUM) lars.wolfsteller @tum.de Naoaki Okazaki Tokyo Institute of Technology okazaki @c.titech.ac.jp Abstract This paper describes the emphasis selection system of the team TextLearner for SemEval 2020 Task 10: Emphasis Selection For Written Text in Visual Media. The system aims to learn the em- phasis selection distribution using contextual representations extracted from pre-trained language models and a two-staged ranking model. The experimental results demonstrate the strong contex- tual representation power of the recent advanced transformer-based language model RoBERTa, which can be exploited using a simple but effective architecture on top. 1 Introduction Visual communication aims at conveying intended information from the author to the audience with the help of visual elements. One of its essential design principles is to reduce the ambiguity and increase the effectiveness of communication. Visual communication usually consists of two elements: text and image. For text, the author often emphasizes some parts of the text by changing the visual representation of that part to better convey the intention. Visual communication design tools often fail in understanding the meaning of text and user intention. They rely solely on visual attributes when suggesting emphasized textual parts and their visual representations, which often leads to the wrong text emphasis (Shirani et al., 2020). Therefore, a system that could help design tools to understand the semantic meaning and common human interpretation of the text would better assist the user to design effective visual communication. SemEval 2020 Task 10: Emphasis Selection For Written Text in Visual Media is a shared task that invites participants to propose methods that can model human emphasis selection for short written English text (Shirani et al., 2020). The task contains several challenges. First, emphasis selection is highly subjective; different annotator may emphasize different parts of the text. Second, there is a lack of additional context information; the model therefore has to rely only on the text. Third, there is a relatively small amount of training data; the training set provided by the organizers contains only 2742 instances of short written text. Our strategy to tackle these three challenges is to model the emphasis selection distribution of annotators by a combination of contextual representations of text extracted from a pre-trained language model and a two-staged ranking model to suggest possible tokens for emphasis. From experimental results, we demonstrate that our simple two-staged ranking model built on top of RoBERTa (large model, fine-tuned on MNLI) (Liu et al., 2019) outperforms the LSTM-based baseline system (Shirani et al., 2020), which indicates transformer-based pre-trained language models such as RoBERTa have a strong contextual representation power and the combination of such language models with a simple ranking model is beneficial to the problem of emphasis selection. 1.1 Task Definition The task organizers formalized …

Content truncated to 500 words