Correcting Keyboard Layout Errors and Homoglyphs in Queries

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 621–626, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Correcting Keyboard Layout Errors and Homoglyphs in Queries Derek Barnes debarnes@ebay.com Mahesh Joshi mahesh.joshi@ebay.com eBay Inc., 2065 Hamilton Ave, San Jose, CA, 95125, USA Hassan Sawaf hsawaf@ebay.com Abstract Keyboard layout errors and homoglyphs in cross-language queries impact our abil- ity to correctly interpret user informa- tion needs and offer relevant results. We present a machine learning approach to correcting these errors, based largely on character-level n-gram features. We demonstrate superior performance over rule-based methods, as well as a signif- icant reduction in the number of queries that yield null search results. 1 Introduction The success of an eCommerce site depends on how well users are connected with products and services of interest. Users typically communi- cate their desires through search queries; however, queries are often incomplete and contain errors, which impact the quantity and quality of search results. New challenges arise for search engines in cross-border eCommerce. In this paper, we fo- cus on two cross-linguistic phenomena that make interpreting queries difficult: (i) Homoglyphs: (Miller, 2013): Tokens such as “case” (underlined letters Cyrillic), in which users mix characters from different character sets that are visually simi- lar or identical. For instance, English and Russian alphabets share homoglyphs such as c, a, e, o, y, k, etc. Although the letters are visually similar or in some cases identical, the underlying character codes are different. (ii) Keyboard Layout Errors (KLEs): (Baytin et al., 2013): When switching one’s keyboard between language modes, users at times enter terms in the wrong character set. For instance, “чехол шзфв” may appear to be a Rus- sian query. While “чехол” is the Russian word for “case”, “шзфв” is actually the user’s attempt to enter the characters “ipad” while leaving their keyboard in Russian language mode. Queries con- taining KLEs or homoglyphs are unlikely to pro- duce any search results, unless the intended ASCII sequences can be recovered. In a test set sam- pled from Russian/English queries with null (i.e. empty) search results (see Section 3.1), we found approximately 7.8% contained at least one KLE or homoglyph. In this paper, we present a machine learning approach to identifying and correcting query to- kens containing homoglyphs and KLEs. We show that the proposed method offers superior accuracy over rule-based methods, as well as significant im- provement in search recall. Although we focus our results on Russian/English queries, the techniques (particularly for KLEs) can be applied to other lan- guage pairs that use different character sets, such as Korean-English and Thai-English. 2 Methodology In cross-border trade at eBay, multilingual queries are translated into the inventory’s source language prior to search. A key application of this, and the focus of this paper, is the translation of Rus- sian queries into English, in order to provide Rus- sian users a more convenient interface to English- based inventory in North America. The presence of KLEs and homoglyphs in …

Content truncated to 500 words