Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities

Proceedings of the 28th International Conference on Computational Linguistics, pages 4667–4675 Barcelona, Spain (Online), December 8-13, 2020 4667 Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities Hao Zhang and Jae Ro and Richard Sproat Google Research {haozhang,jaero,rws}@google.com Abstract Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search. We link this problem to the classic problem of Chinese word segmentation and show the effectiveness of a tagging model based on Recurrent Neural Networks (RNNs) using characters as input. To compensate for the lack of training data, we propose a pre-training method on concatenated entity names in a large knowledge database. Pre-training improves the model by 33% and brings the sequence accuracy to 85%. 1 Introduction Word segmentation is a fundamental NLP analysis problem for written languages with no space delimiters between words such as Chinese and Japanese. In the age of digital communications, new URLs (e.g. www.openresearch.org) and hashtags (e.g. #photooftheday), which often include strings of concatenated words (openresearch, photooftheday) are being added every day to a growing set of tokens that an NLP system may need to deal with, and they pose challenges for language and speech applications. For example, a Text-to-Speech (TTS) synthesis system will struggle to pronounce these concatenated tokens, since simply applying a grapheme-to-phoneme system out of the box to something like photooftheday will usually yield poor results. This suggests the need for a model that can split such tokens into the component words. So-called “end-to-end” neural TTS systems (Sotelo et al., 2017; Wang et al., 2017), which learn to map directly from character sequences to speech might seem to hold out the hope of avoiding treating this problem separately. However, the fact that URLs occur relatively rarely in most TTS training data limits the promise of such models on this long-tail problem. The problem of analyzing URLs does differ in one useful way from more general text normalization problems. For a token such as 123 in a text, one typically needs to know what context it occurs in in order to know how to read it: is it one hundred twenty three or one twenty three; see (Sproat et al., 2001), inter alia. In the case of URLs, these are largely context-independent since the output segmentation is usually unaffected by the surrounding words. Hence the problem can be treated as a standalone one that does not require the system to be trained as part of broader text normalization training. Our training data comes from camel case URLs that naturally define the segment boundaries (e.g. NYTimes.com maps to N Y Times . com) along with manual corrections for non-trivial bound- aries. We release our training and evaluation data sets to promote research on this problem. By drawing an analogy with Chinese word segmentation, we cast the URL segmentation problem as a sequence tagging problem. We propose a simple Recurrent Neural Network (RNN) based tagger with an encoder and a decoder. …

Content truncated to 500 words