Neural text normalization leveraging similarities of strings and sounds

Proceedings of the 28th International Conference on Computational Linguistics, pages 2126–2131 Barcelona, Spain (Online), December 8-13, 2020 2126 Neural text normalization leveraging similarities of strings and sounds Riku Kawamura† Tatsuya Aoki† Hidetaka Kamigaito† Hiroya Takamura† ‡ Manabu Okumura† †Tokyo Institute of Technology ‡National Institute of Advanced Industrial Science and Technology {riku, aoki, kamigaito}@lr.pi.titech.ac.jp {takamura, oku}@pi.titech.ac.jp Abstract We propose neural models that can normalize text by considering the similarities of word strings and sounds. We experimentally compared a model that considers the similarities of both word strings and sounds, a model that considers only the similarity of word strings or of sounds, and a model without the similarities as a baseline. Results showed that leveraging the word string similarity succeeded in dealing with misspellings and abbreviations, and taking into account the sound similarity succeeded in dealing with phonetic substitutions and emphasized characters. So that the proposed models achieved higher F1 scores than the baseline. 1 Introduction Non-standard words such as misspellings or abbreviations are often used in social media such as Twitter. It is difficult to understand the meaning of such words without any extra knowledge, and it is challenging to perform natural language processing over sentences including them. Text normalization plays an important role in dealing with these broken texts by correcting such a sentence into a standard one. The following is an example of text normalization. before normalization: r ↓u cuming 2 midcorner dis sunday? ↓ ↓ ↓ ↓ after normalization: are you coming to midcorner this sunday? In text normalization, as shown in the example, we need to correct several types, including misspellings (‘cuming’ to ‘coming’), abbreviations (‘convo’ to ‘conversation’), phonetic substitutions (‘dis’ to ‘this’ or ‘r u’ to ‘are you’), and emphasized characters (‘yeeeees’ to ‘yes’). Even though the similarities of character surfaces and phonemes is important for a human to under- stand non-standard words, current neural network-based text normalization methods do not consider this information. We assume that text normalization more intuitive to humans is possible by explicitly con- sidering such features in a neural network-based method. Based on this assumption, in this work, we propose neural text normalization models that leverage both string and sound similarities. Experimental results show that our proposed models outperformed a baseline and achieved state-of-the-art results in the text normalization track on WNUT-2015. 2 Related Work Han et al. (2012) proposed a ranking-based text normalization method that incorporates the matching degree of surrounding word n-grams to the target word and the edit distance from existing words. Li and Liu (2012) proposed a text normalization method leveraging phonetic information to translate non- standard words into standard ones. Ansari et al. (2017) proposed an automatic optimization-based nearest neighbor matching approach leveraging string and phonetic similarity. Jin (2015) achieved the best This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 2127 LSTM LSTM Embedding layer Word: x LSTM LSTM Embedding layer Word: y Cosine similarity Encoder + 【 Architecture of Seq2Seq 】 【 Architecture of Deep Levenshtein …

Content truncated to 500 words