Decoding Anagrammed Texts Written in an Unknown Language and Script

Decoding Anagrammed Texts Written in an Unknown Language and Script Bradley Hauer and Grzegorz Kondrak Department of Computing Science University of Alberta Edmonton, Canada {bmhauer,gkondrak}@ualberta.ca Abstract Algorithmic decipherment is a prime exam- ple of a truly unsupervised problem. The first step in the decipherment process is the iden- tification of the encrypted language. We pro- pose three methods for determining the source language of a document enciphered with a monoalphabetic substitution cipher. The best method achieves 97% accuracy on 380 lan- guages. We then present an approach to de- coding anagrammed substitution ciphers, in which the letters within words have been ar- bitrarily transposed. It obtains the average de- cryption word accuracy of 93% on a set of 50 ciphertexts in 5 languages. Finally, we report the results on the Voynich manuscript, an un- solved fifteenth century cipher, which suggest Hebrew as the language of the document. 1 Introduction The Voynich manuscript is a medieval codex1 con- sisting of 240 pages written in a unique script, which has been referred to as the world’s most important unsolved cipher (Schmeh, 2013). The type of ci- pher that was used to generate the text is unknown; a number of theories have been proposed, includ- ing substitution and transposition ciphers, an abjad (a writing system in which vowels are not written), steganography, semi-random schemes, and an elab- orate hoax. However, the biggest obstacle to deci- 1The manuscript was radiocarbon dated to 1404-1438 AD in the Arizona Accelerator Mass Spectrometry Labo- ratory (http://www.arizona.edu/crack-voynich-code, accessed Nov. 20, 2015). phering the manuscript is the lack of knowledge of what language it represents. Identification of the underlying language has been crucial for the decipherment of ancient scripts, in- cluding Egyptian hieroglyphics (Coptic), Linear B (Greek), and Mayan glyphs (Ch’olti’). On the other hand, the languages of many undeciphered scripts, such as Linear A, the Indus script, and the Phaistos Disc, remain unknown (Robinson, 2002). Even the order of characters within text may be in doubt; in Egyptian hieroglyphic inscriptions, for instance, the symbols were sometimes rearranged within a word in order to create a more elegant inscription (Singh, 2011). Another complicating factor is the omission of vowels in some writing systems. Applications of ciphertext language identification extend beyond secret ciphers and ancient scripts. Nagy et al. (1987) frame optical character recogni- tion as a decipherment task. Knight et al. (2006) note that for some languages, such as Hindi, there exist many different and incompatible encoding schemes for digital storage of text; the task of an- alyzing such an arbitrary encoding scheme can be viewed as a decipherment of a substitution cipher in an unknown language. Similarly, the unsupervised derivation of transliteration mappings between dif- ferent writing scripts lends itself to a cipher formu- lation (Ravi and Knight, 2009). The Voynich manuscript is written in an unknown script that encodes an unknown language, which is the most challenging type of a decipherment prob- lem (Robinson, 2002, p. 46). Inspired by the mys- tery of both the Voynich manuscript …

Content truncated to 500 words