SEND: A Simple and Efficient Noise Detection Algorithm for Vietnamese Real Estate Posts

SEND: A Simple and Efficient Noise Detection Algorithm for Vietnamese Real Estate Posts Khanh Quoc Tran1,3,4, An Tran-Hoai Le1,3,4, An Trong Nguyen1,3,4, Tung Tran Nguyen Doan4, Son Thanh Huynh2,3,4, Bao Le Hoang2,3,4, Hoang Nguyen Minh2,3,4, Triet Minh Thai1,3,4, Hoang Le Huy2,3,4, Dang T. Huynh2,3,4, Binh T. Nguyen2,3,4∗, Nhi Ho5, Trung T. Nguyen5 1 University of Information Technology, Ho Chi Minh City, Vietnam 2 University of Science, Ho Chi Minh City, Vietnam 3 Vietnam National University, Ho Chi Minh City, Vietnam 4 AISIA Research Lab, Vietnam 5 Hung Thinh Corporation, Ho Chi Minh City, Vietnam Abstract One of the emerging research fields in Natural Language Processing is Noise Detection (ND), the process of identifying posts containing noise information on textual data. While nu- merous datasets and approaches are developed for ND research in other languages, equiv- alent resources for the Vietnamese are lim- ited. To the best of our knowledge, no dataset or method has been investigated or proposed to address the noise Detection tasks in the Vietnamese language. In reality, noise data is constantly present in datasets and some- times hurts relevant model performance. To overcome this limitation, we propose ViND, a first human-annotated dataset that is available to the scientific community as a benchmark for the task of Vietnamese Noise Detection. The ViND dataset contains 12,862 posts col- lected from five major Vietnamese real es- tate news websites. This paper provides an overview of the Vietnamese Noise Detec- tion task, the process of creating the ViND dataset, and the techniques for carrying out the baseline experiments. On the ViND dataset, the PhoBERTlarge model outperforms robust baseline models such as LSTM, Bi-LSTM, BERT, RoBERTa, XLM-R, and DistilBERT and achieves a macro F1-score of 0.9024. In addition, our proposed method also success- fully improves the related task’s performance, mainly Vietnamese Named Entity Recognition (NER) for real estate posts, about 0.0239 in terms of macro F1-score. 1 Introduction In Natural Language Processing (NLP) and Ma- chine Learning, data and data processing play a significant role, especially when working with ∗*Corresponding author: Binh T. Nguyen (e-mail: ngt- binh@hcmus.edu.vn). user-generated or social network content formed in non-standardized text. Furthermore, data in this aspect generally contains meaningless or useless, and this kind of information often impacts nega- tively on purpose but is easily ignored. Therefore, to increase the quality of data and the performance of the NLP models, removing all of the noisy in- formation from the dataset (Subramaniam et al., 2009) is necessary. The explosion of data available from social net- working and e-commerce platforms has opened the need and opportunities for noise information processing in NLP (Al Sharou et al., 2021). How- ever, for real estate, the posts could be disturbed by the wrong input from creators, missing essen- tial descriptions, and the mistakes made by real estate agents. Besides, there is a wide range of helpful values with complete information. Never- theless, their definitions are still ambiguous and complicated, making them difficult to distinguish from the actual noise (Subramaniam et al., 2009). …

Content truncated to 500 words