Morphologically Correct Tokenisations

Tokenisation is the first step in almost all NLP tasks, and state-of-the-art transformer-based language models all use subword tokenisation algorithms to process input text. Existing al- gorithms have problems, often …...