Randomseed19 at SemEval-2020 Task 10: Emphasis Selection for Written Text in Visual Media

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 1685–1690 Barcelona, Spain (Online), December 12, 2020. 1685 randomseed19 at SemEval-2020 Task 10: Emphasis Selection For Written Text in Visual Media Aleksandr Shatilov Denis Gordeev Alexey Rey The Russian Presidential Academy of National Economy and Public Administration (RANEPA), Moscow, Russia {shatilov-aa,gordeev-di,rey-ai}@ranepa.ru Abstract This paper describes our approach to emphasis selection for written text in visual media as a solution for SemEval 2020 Task 10. We used an ensemble of several different Transformer-based models and cast the task as a sequence labeling problem with two tags: ’I’ as ’emphasized’ and ’O’ as ’non-emphasized’ for each token in the text. Our approach ranked fourth in this competition. 1 Introduction The purpose of SemEval 2020 Task 10 is to design automatic methods for emphasis selection, i.e. choosing candidates for emphasis in short written texts or enabling automated design assistance in authoring. The given task is different from related ones in that word emphasis patterns are person- and domain-specific, making different selections valid depending on the audience and the intent. Examples of different emphases are presented in Figure 1. The dataset for this shared task includes short sentences from the following two datasets: 1. Spark dataset: This dataset is collected from Adobe Spark and is a collection of short texts containing a variety of subjects featured in flyers, posters, advertisements or motivational memes on social media. 2. Quotes dataset: This dataset is a collection of quotes from well-known authors collected from Wisdom Quotes. The data was labeled by 9 annotators, for further details please refer to the task paper (Shirani et al., 2020). Figure 1: Two examples with different emphasis This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http:// creativecommons.org/licenses/by/4.0/. 1686 2 System overview 2.1 Dataset Each token in the dataset is labeled using the BIO notation. The main target of the task is to predict emphasis probability for each token in a given sentence, which is calculated as a share of ’B’ and ’I’ labels from 9 annotators. An example of training and validation data is presented in Table 1. In the test data only columns ’word id’ and ’word’ are available. word id word BIO annotations BIO frequencies emphasis probability pos tags Q 2 0 What O|O|O|O|O|O|O|O|O 0|0|9 0.0 WP Q 2 1 ’s O|O|O|O|O|O|O|O|O 0|0|9 0.0 VBZ Q 2 2 dangerous O|O|B|B|O|O|B|B|B 5|0|4 0.5556 JJ Q 2 3 is O|O|O|O|O|O|O|O|O 0|0|9 0.0 VBZ Q 2 4 not O|B|B|O|O|O|O|O|O 2|0|7 0.2222 RB Q 2 5 to O|I|I|O|O|O|O|O|O 0|2|7 0.2222 TO Q 2 6 evolve B|I|I|O|B|B|O|O|O 3|2|4 0.5556 VB Q 2 7 . O|O|O|O|I|O|O|O|O 0|1|8 0.1111 . Table 1: Example of training and validation data. We approach the task as a sequence labeling problem with two tags: ’I’ as ’emphasized’ (the original ’B’ was transformed to ’I’) and ’O’ as ’non-emphasized’ for each token in the text. Subsequently, the probability of predicted ’I’ token was taken as emphasis probability. We considered two ways to convert the given …

Content truncated to 500 words