Моноязычные китайские токенизаторы

As tokenization is usually ambiguous for many natural languages such as Chinese and Korean, tokenization errors might po- tentially introduce translation mistakes for translation systems that rely on 1-best to- …...