HashSet - A Dataset For Hashtag Segmentation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7215–7219 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7215 HashSet - A Dataset For Hashtag Segmentation Prashant Kodali†, Akshala Bhatnagar∗, Naman Ahuja† Manish Shrivastava†, Ponnurangam Kumaraguru† †IIIT Hyderabad prashant.kodali@research.iiit.ac.in, naman.ahuja@students.iiit.ac.in, {m.shrivastava, pk.guru}@iiit.ac.in ∗IIIT Delhi akshala18012@iiitd.ac.in Abstract Hashtag segmentation is the task of breaking a hashtag into its constituent tokens. Hashtags often encode the essence of user- generated posts, along with information like topic and sentiment, which are useful in downstream tasks. Hashtags prioritize brevity and are written in unique ways - transliterating and mixing languages, spelling variations, creative named entities. Benchmark datasets used for the hashtag segmentation task - STAN, BOUN - are small in size and extracted from a single set of tweets. However, datasets should reflect the variations in writing styles of hashtags and also account for domain and language specificity, failing which the results will misrepresent model performance. We argue that model performance should be assessed on a wider variety of hashtags, and datasets should be carefully curated. To this end, we propose HashSet, a dataset comprising of: a) 1.9k manually annotated dataset; b) 3.3M loosely supervised dataset. HashSet dataset is sampled from a different set of tweets when compared to existing datasets and provides an alternate distribution of hashtags to build and validate hashtag segmentation models. We show that the performance of SOTA models for Hashtag Segmentation drops substantially on proposed dataset, indicating that the proposed dataset provides an alternate set of hashtags to train and assess models. Datasets and results are released publicly and can be accessed from https://github.com/prashantkodali/HashSet Keywords: Hashtag Segmentation 1. Introduction Hashtags have become ubiquitous across user- generated content on the Internet. Hashtags often encapsulate the gist, emotion, and sentiment cues of the text (Qadir and Riloff, 2014), and have been demonstrated to be useful in downstream tasks like text classification (Belainine et al., 2016). Hashtags, however, pose a challenge in automatic processing because of their unsegmented nature. To leverage hashtags in downstream task, hashtags have to be bro- ken into their constituent tokens, a task which is called Hashtag Segmentation or Hashtag Decomposition. Segmenting a hashtag is akin to a word segmentation problem. Hashtags show specific quirks like spelling variations (e.g., #letzgooo), romanization of native language words (e.g., #sabkasaath), camel case (e.g., #WeLoveApples) and presence of special characters (e.g., #We love apples@1). Presence of such quirks in hashtags makes the solution of hashtag segmentation non-trivial and slightly different from word segmenta- tion e.g., ”letsgo” can be segmented easily compared to ”letzgoo” since the later has non-cannonical spellings. Recently proposed methods (Maddela et al., 2019; Ro- drigues et al., 2021) have leveraged the power of lan- guage models and combined them with neural rank- ing models to segment hashtags. STAN (Bansal et al., 2015), BOUN (C¸ elebi and ¨Ozg¨ur, 2016) are the popu- lar benchmark datasets for Hashtag segmentation. Test set sizes for STAN and BOUN datasets are 1012, and 999 respectively. Small-sized …

Content truncated to 500 words