Now, It’s Personal : The Need for Personalized Word Sense Disambiguation

Proceedings of Recent Advances in Natural Language Processing, pages 53–62 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_007 53 Towards a Better Understanding of Noise in Natural Language Processing Khetam Al Sharou∗, Zhenhao Li∗, Lucia Specia Language and Multimodal AI (LAMA) Lab, Imperial College London, UK {k.al-sharou, zhenhao.li18, l.specia}@imperial.ac.uk Abstract In this paper, we propose a definition and tax- onomy of various types of non-standard tex- tual content – generally referred to as “noise” – in Natural Language Processing (NLP). While data pre-processing is undoubtedly important in NLP, especially when dealing with user- generated content, a broader understanding of different sources of noise and how to deal with them is an aspect that has been largely ne- glected. We provide a comprehensive list of potential sources of noise, categorise and de- scribe them, and show the impact of a subset of standard pre-processing strategies on differ- ent tasks. Our main goal is to raise aware- ness of non-standard content – which should not always be considered as “noise” – and of the need for careful, task-dependent pre- processing. This is an alternative to blan- ket, all-encompassing solutions generally ap- plied by researchers through “standard” pre- processing pipelines. The intention is for this categorisation to serve as a point of refer- ence to support NLP researchers in devising strategies to clean, normalise or embrace non- standard content. 1 Introduction In Natural Language Processing (NLP), “noise” as a concept is not well understood and often de- scribed as hard to define (Taghipour et al., 2011). In this paper, we attempt to outline what different types of noise mean in order to support NLP re- searchers in devising strategies to clean, normalise or embrace unexpected content at either training or inference time. The term “noise” has been generally used to cover both harmful and meaningful types of non- standard content. We however believe that a distinc- tion should be made between content that should ∗Equal contribution be removed/normalised because it harms NLP sys- tems – henceforth harmful noise – and content that should be kept to improve the performance of such systems – henceforth useful noise1. To clar- ify further, the usual approach to handling “noise” is to attempt to clean or normalise the data as much as possible, but this content can be as important as any other elements in the data because it carries meaning or intentions. In text classification appli- cations, for example, some types of noise should be kept and taken into account, e.g. certain punctu- ation patterns in sentiment analysis. Furthermore, in generation applications, noise often needs to be transferred to the output, such as emojis in machine translation to preserve sentiment. Moreover, in tasks such as correction of second language learn- ers’ essays, errors such as lack of cohesion and incorrect punctuation need to be retained. Handling noise in NLP has been attracting sig- nificant attention especially with the widespread availability of data from social media platforms such as Twitter, Reddit, among others, that created not only new opportunities but also …

Content truncated to 500 words